NASA Earth Observation Data in the Cloud

Overview

Data rates from NASA’s Earth observing satellites are slated to grow by a factor of 10 in a few short years. In turn, this will cause 10-fold growth in the Earth Observing System Data and Information System (EOSDIS) archives. Eventually the problem will propagate to the Earth science research community, and simply downloading massive datasets to a local server for analysis will become infeasible. In response, EOSDIS data for large upcoming missions will be archived in the cloud.

Project Details

Storing NASA’s Earth observation data in the cloud carries significant benefits. The near-infinite scalability of cloud storage simplifies data management, freeing data curators from worrying about the hardware and commensurate space, power, and cooling requirements needed to store over 200 petabytes of data. The proximity of the data to highly scalable computing resources will allow users to analyze the data in place. Data transformed into Analysis-Ready form will be delivered to high-performance computing (HPC) users for data fusion and assimilation. The outputs of these operations will then return to the cloud for archive and distribution to the wider research community.

Results and Impact

NASA has prototyped an archive system in the cloud, which is on track to support two upcoming high-data-rate radar missions: Surface Water Ocean Topography (SWOT) and NASA-Indian Space Research Organization (ISRO) Synthetic Aperture Radar (NISAR). Other prototypes are exploring how to provision the necessary computing power to work with the cloud archive

Why HPC Matters

While cloud computing is useful for data-parallel analysis, supercomputers may be more suitable for other types of high-performance analysis, such as some forms of data fusion and modeling. At the same time, supercomputing centers are typically designed for use by a controlled group of expert users. On the other hand, delivering data to a diverse global community numbering in the millions is a core capability of cloud computing. Establishing a symbiotic relationship between the cloud-computing-based data archives and NASA HPC-based computation is thus critical to supporting high-volume data analysis and modeling.

More Information

• See the poster for this demonstration (PDF, 1.3MB). As shown in the NASA booth at SC18.