Extreme Weather, Machine Learning, and High-Performance Computing

Overview

Extreme weather has enormous effects around the globe including potentially major impacts on human life and property. Being able to better predict the probabilities and onset of extreme weather events in an ever-changing climate has obvious benefits. However, these events happen within a global context with connections over large expanses of space and time that are not easily exposed. The NASA Center for Climate Simulation (NCCS) has been developing a machine learning (ML) algorithm to help better predict the onset of such events. We are now in a better position than ever to do this given the availability of the high-resolution reanalysis datasets.

Project Details

Our research team used association rule mining techniques to expose links between historical atmospheric patterns leading up to hurricanes. The locations of tropical storms occurring in the West Pacific storm basin between 1980 and 2016 came from the International Best Track Archive for Climate Stewardship (IBTrACS) for August and September. Using these locations, researchers extracted Single Level Diagnostics (SLD) from the Modern-Era Retrospective analysis for Research and Applications, version 2 (MERRA-2) dataset. Due to the amount of data, the SLDs were dimensionally reduced and passed through Symbolic Aggregate approXimation (SAX). Then, all events were passed through a clustering algorithm.

Results and Impact

The NCCS embarked on this project for two primary reasons: 1) to see if machine learning could be applied to large-scale, spatiotemporal datasets to expose connections between extreme weather events, and 2) to better understand the gaps in our current high-performance computing environment when used for ML applications.

• Four distinct clusters emerged. While additional analysis of these clusters is required, the initial clusters seem to be centered around the speed at which the hurricanes formed.
• When working on large, existing datasets, there is clearly a need for methods and infrastructure to extract relevant information and distill that information down to a manageable size for the ML algorithms.
• With the speed at which NASA and other agencies are generating new, high-resolution reanalysis data, there is also a need for running ML algorithms while data is being generated.

Why HPC Matters

The creation of high-resolution climate reanalysis data requires large-scale processing, data storage, and analytics. NASA's Global Modeling and Assimilation Office is using the NCCS Discover supercomputer to generate multiple reanalysis datasets with ever-increasing amounts and types of atmospheric data, including aerosols. The MERRA-2 dataset has a total size of approximately 400 terabytes, which makes it impractical, if not impossible, to download for this type of analysis. Therefore, scientists must do such analyses within the NCCS, where the data resides.

What's Next

These types of machine learning techniques open the door to emerging questions about the development of extreme weather events, their connections though space and time, and probabilities for these events to develop. Future work will include more parameter tuning across various aspects of the analysis. In addition, researchers will develop a technique to dynamically warp time for weather events and minimize the effects of variant time scales for weather events. These techniques will be useful for exploring other phenomena, such as mesoscale convective systems.