Rapid Analysis and Management of Large Spatiotemporal Data
Speaker: Ben O’Neill, RNET Technologies and Rohit Deshmukh, Florida Atlantic University
Presentation
Abstract
NASA's vast collection of simulated, experimental, and observational datasets has paved the way for groundbreaking advancements in addressing real-world physical challenges. Yet, the sheer size of high-fidelity data generated presents challenges in terms of data transfer, computer resources, and data volume, particularly for downstream users lacking direct access to high performance computing resources.
In this talk, we introduce a lossy reduce-then-analyze strategy that enables efficient data storage, transfer, and analysis. Our data reduction method, called the Data-informed Local Subspaces (DLS), uses data-driven feature learning and data-enriched finite element methods to transform high fidelity data into a reduced format that promotes rapid data analytics. Results demonstrating 20-100x data reduction for canonical fluid dynamics datasets, and 10-80x for earth science datasets will be discussed. An order of magnitude speedup in vorticity calculations and multiple orders of magnitude improvement in signal processing analyses will also be presented. Finally, we will discuss a pathway for how our tools can be integrated into NASAs existing aerospace and earth science workflows.
This work is made possible through the support of a NASA Phase I Small Business Innovation Research (SBIR) award.
Biography