Skip to main content
NASA Advanced Supercomputing (NAS) Division World-class modeling, simulation, and supercomputing in support of the NASA mission

A Scalable Platform for Rapid Data Analytics Through Data Driven Lossy Compression

AMS Seminar Series

Speaker: Ben O’Neill, RNET Technologies and Rohit Deshmukh, Florida Atlantic University

Presentation

Seminar Slide Deck (PDF-3.7MB)

Abstract

The increasing scale of NASA's high-fidelity earth science and fluid dynamics datasets poses significant storage, transfer, and analysis challenges. RNET, in collaboration with FAU, is addressing these challenges by developing a data-driven scientific data compression and analysis platform that enables efficient storage, transfer, and analysis of large-scale spatiotemporal datasets. At the heart of the platform is a collection of data-driven lossy compression algorithms based on Data-informed Local Subspaces (DLS). These algorithms transform high-fidelity data into a geometry-preserving reduced data format that supports embarrassingly parallel pointwise decompression, low-memory statistical calculations, and ultra-fast spatiotemporal analysis.

In this talk, we present recent advancements in data-driven lossy compression using DLS, including a “delayed” compression approach that minimizes online (in situ) compression costs, a “zone-based” compression algorithm that lowers compression costs and caps memory usage, and a new error-bounded lossy compression approach that can be applied on both structured and unstructured data. These new algorithms will be demonstrated on data from a recent GEOS-Forward Processing forecast (structured data defined on a C720 cubed sphere) and previously generated GEOS5-Nature Run data subsets (structured data defined on a lat-lon grid). Towards the end, we will present (and seek feedback on) our future plans for this ongoing SBIR Phase II project, including a description of planned features, data formats, and a PANGEO/XArray-based analytics API.

This work is made possible through the support of NASA Phase I & II Small Business Innovation Research (SBIR) awards.

Biography

Photo of Ben O’Neill
Ben O’Neill is a senior research scientist at RNET Technologies Inc. in Dayton Ohio. Ben received his Ph.D. in Applied Mathematics from the University of Colorado at Boulder with a specialization in high performance computing and parallel time integration. His research interests include HPC software usability tools, lossy data compression, and reduced order modeling.






Photo of Rohit Deshmukh
Rohit Deshmukh is an Assistant Professor at Florida Atlantic University (FAU). Rohit obtained his Ph.D. in Aerospace Engineering from The Ohio State University in 2016, with a focus on applying model order reduction to incompressible turbulent flows. Rohit has extensive research experience in computational mechanics in the areas of model reduction, data-driven-physics-based model development, and coupled fluid-structural-thermal simulation and analysis.