Machine learning, deep learning, and artificial intelligence have become essential tools for handling and gaining insight from the enormous amounts of data that are being generated via high-performance computing, modern modeling and simulation, and instrument technology.

These methods organize data in a way that is both systematic (collected, processed, and stored methodically according to a standard practice) and semantic (unambiguous and logical, in both order and representation, so that relationships and biases can be easily explored). Once the data is organized, it can then be analyzed using powerful statistical methods.

HECC Data Science Team: Pilot Projects

HECC offers a range of services for researchers moving into advanced data science, using the latest technologies in statistics-based data analytics, machine learning, and deep learning. For general information, see HECC Data Science Services.

The Data Science Team is currently working with NASA researchers on a number of projects, including:

• Carbon Nanotube Gas Sensor Using Neural Networks (PDF)
• Predicting Composition of Photo Voltaic Cells Using Neural Networks (PDF)

For more information about working with our team, please contact us at dataanalytics@nas.nasa.gov.

Getting Started with Machine Learning

In this section of the Knowledge Base, we provide information that can help you get started with a machine learning environment and short tutorials describing how to use the machine learning tools available on NAS systems. Please check back often, as new articles will become available soon.

Once you have a NAS account, you will have access to software and infrastructure for your machine learning project. If you are a new user, be sure to read through our New User Orientation and complete the first-time login process.

Conda Environments

We provide multiple conda environments that include basic machine learning packages, as well as common image processing and natural language processing packages, for your machine learning projects. 

The tables below show currently available conda environments that can be used with nodes containing either x86_64-Based CPUs (Table 1) or aarch64-based CPUs (Table 2). All of the environments include GPU support.

Notes:

• The conda environments contain many of the same software programs, but the package versions may be different. Also, the environments may contain additional packages to the ones listed here. To learn more, see Using Conda Environments for Machine Learning.
• The environments are provided via software modules in HECC system directories. For general information about NAS-provided software modules, see Software Directories and Using Software Modules.

NVIDIA rapids can be found in the rapids24_12 conda environment. PyTorch versions 2.4 and 2.5 can be found in the pyt2_4_gh and pyt2_5_gh conda environments, respectively. PyTorch 2.5 is also available in the pyt_keras_gh conda environment; this environment is also the host for Keras 3.7.

TensorFlow version 2.18 can be found in the tf2_18_gh conda environment. To use this version of TensorFlow across multiple nodes, you might need to set these environment variables in your code before calling TensorFlow, as follows:

os.environ["TF_USE_LEGACY_KERAS"] = "1"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

Or, you can set the environment variables at the command line before the code is run.

TensorFlow users who use the Keras API can use the pyt_keras_gh environment with PyTorch 2.5 as the backend. For most TensorFlow code bases, few or no changes to the code will be needed to use this environment. Contact NAS support if you need assistance in adapting TensorFlow code to run in this environment.

Note: Rendering graphics using OpenAI Gym is currently unavailable.

Packages

Miniconda version 4.12.0 is licensed from Anaconda, which directly impacts how environments can be cloned. Some of the previous cloning options cannot be used with the licensed Anaconda environments. For the new cloning instructions, see Managing and Installing Python Packages for Machine Learning.

Working with PBS

All NAS systems utilize the Portable Batch System (PBS) for batch job submission, job monitoring, and job management. To request examples of PBS scripts for Apache Spark and TensorFlow, please contact us at dataanalytics@nas.nasa.gov.

System Information

• Systems and Filesystems
• GPU nodes
• Current system status

Note: When you publish your results, please acknowledge the use of NAS resources.

Additional Resources

• HECC Webinars, including:
  • 2021 NAS Data Science Webinar
  • HECC Data Science Platform
• Machine Learning (Wikipedia)
• Machine Learning for Humans