What Role Should Commercial Clouds Play in NASA HPC?

Overview

Every so often, the question arises: would it be more cost effective for NASA’s High-End Computing Capability (HECC) Project to use commercial cloud resources? To find out, HECC’s Application Performance and Productivity (APP) team undertook a performance and cost evaluation comparing three domains: two commercial cloud providers, Amazon Web Services (AWS) and Penguin Computing on Demand (POD), and HECC’s in-house supercomputers.

Project Details

In the study, the APP team used a combination of the NAS Parallel Benchmarks (NPB) and six full applications from NASA’s workload on the Pleiades and Electra supercomputers to compare the performance of nodes based on three different generations of Intel Xeon processors—Haswell, Broadwell, and Skylake.

The APP team also calculated costs for the runs (as of May 2018). In the case of work done on Pleiades and Electra, the team used the “full cost” of running, based on the HECC Project’s total annual budget (including all hardware, software, power, maintenance, staff, facility costs, and so on). In the case of the commercial cloud providers, the team used only the compute cost of each run, using published rates and including publicly known discounts as appropriate. Other infrastructure costs of running in the cloud were not considered in this study. These include: near-term storage; deep archive storage; network bandwidth and software licensing; plus staffing costs for security and security monitoring; application porting and support; problem tracking and resolution; program management and support; and sustaining engineering. These “full cloud costs” would likely be significant.

Results and Impact

Based on this analysis of performance and cost data, the study found that tightly coupled, multi-node applications from the NASA workload take somewhat more time when run on cloud-based nodes connected with high-performance computing (HPC)-level interconnects—and they take significantly more time when run on cloud-based nodes that use conventional, Ethernet-based interconnects. It also found that the per-hour full cost of HECC resources is less than the compute-only spot price of similar resources at AWS and significantly less than the compute-only price of similar resources at POD. In analyzing the results, the APP team found that while commercial clouds cannot compete on cost with the highly utilized HECC systems, they still have a place in providing access to resource types that would be otherwise unavailable.

As a result of this study, the APP team identified and has started preparing to broaden the range of services offered by HECC to enable a portion of its workload to run on commercial cloud resources. This work includes conducting a pilot study, providing some HECC users with cloud access, and developing a longer-term approach where HECC can act as a broker for HPC use of cloud resources, including user support and an integrated accounting model.

Why HPC Matters

Measuring the performance and cost of running the NASA HPC workload on different platforms, including commercial clouds, is critical to HECC remaining cost effective in meeting the current and future computing needs for NASA science and engineering missions.

What’s Next

The team will eventually expand its cloud pilot project to make it available to all HECC users at no charge, for those workloads that are most cost effective to run in the cloud. For other jobs, users may opt to be billed for cloud usage in order to guarantee faster job turnaround times. Following support for these two pilot services, the team will add mechanisms for non-HECC users to pay for access to cloud resources to meet their NASA science and engineering requirements.

More Information

• Evaluating the Suitability of Commercial Clouds for NASA’s High Performance Computing Applications: A Trade Study (PDF)
• See the poster for this demonstration (PDF, 1.9MB). As shown in the NASA booth at SC18.