A job might not start for one of the following reasons:
You can often determine the reason a job does not start by running one of the following commands to view job status and information:
Note: Users have no access to pbs05a which serves jobs submitted to run on Cabeus gh200 nodes.
See Common Reasons Why Jobs Won't Start for more detailed troubleshooting information.
Yes, you can recover deleted files if you deleted them from your home directory on the NAS compute resources or Lou. Call NAS User Services at (800) 331-8737 or (650) 604-4444, or send an email to support@nas.nasa.gov with the filenames, paths, and the dates they were last modified.
If you deleted the files from a /nobackup directory, the files cannot be restored. For more information on filesystem management, see NAS Lustre Filesystems.
To obtain resource usage information about all of the project group IDs (GIDs) that you belong to, you can:
The myNAS portal shows all the GIDs you belong to and provides a central location where you can easily check allocation and usage and monitor PBS jobs. For more information, see Using the myNAS Portal.
To use the acct_ytd utility to view the number of hours remaining for all of your projects on all systems, run:
acct_ytd -c all
To view the number of hours remaining for a specific project across all systems, specify the GID. For example:
acct_ytd -c all a0800
You can also obtain detailed billing information, such as the number of SBUs that have been used in the current operating year, by using the acct_query utility.
For more information about how to use these utilities, see Job Accounting Utilities, man acct_ytd, and man acct_query.
You can make special requests for various job-related changes, such as longer wall-clock time, higher job priority, or increased disk quotas.
For more information about how job priority and quotas are determined, see How PBS Schedules Jobs and Quota Policy on Disk Space and Files.
When the amount of data in your NAS compute resources home directory or your NAS /nobackup directory exceeds the soft quota, you will receive an email informing you of your current usage and how much grace period remains.
It is expected that your data will occasionally exceed the soft quota, as needed. However, if your data still exceeds the soft quota after 14 days—or if it exceeds the hard quota—you must delete unneeded files and archive infrequently used files until the amount of data is below the soft limit. To archive files, transfer them to the Lou mass storage system.
Once you exceed your hard quota, you will receive the following PBS error when attempting to submit a new job:
qsub: Filesystem quota exceeded! Please reduce your usage or request an increase before submitting.
If you have a long-term need for higher quota limits, send a justification to support@nas.nasa.gov. Your request will be reviewed by management for approval.
For more information, including quota limits, see:
You can use the dmget command to retrieve files that have been archived to tape. For more information about using the dmget command, and further explanation of data migration and storage, see Data Migration Facility (DMF) Commands.
In general, when you have problems logging in with your NAS password or RSA SecurID passcode, call the NAS Control Room at (800) 331-8737 or (650) 604-4444.
Note: Due to security requirements, Control Room staff will confirm your identity by asking you the security question that you submitted with your account request form, or by calling you back at your work phone number on record. If your phone number has changed due to office moves or reorganizations, the Principal Investigator (PI) for your project must contact the Control Room and provide the reason for the change, either by phone or by sending an email to support@nas.nasa.gov. If your PI is unavailable, your branch chief or division chief may do this for you.
You will be prompted to change your default password when you log in to a NAS system. See Password Creation Rules for guidance.
When you publish your research based on the use of NAS-provided high-end computing resources, please include the following language in your acknowledgments:
High-end computing (HEC) resources supporting this work were provided by the NASA Advanced Supercomputing (NAS) Division at Ames Research Center.
Your files may have been migrated offline, from disk to tape. Depending on the size of the files and how busy the tape drive is, it might take a long time for the Data Migration Facility (DMF) to retrieve them from tape.
You can use the dmget command to transfer the files back online, from tape to disk. To learn how to use this command most effectively with your files, see Dealing with Slow File Retrieval.
See also:
On the Lou mass storage system, the Data Migration Facility (DMF) manages disk space by moving unused files to storage on tape.
Use the dmls command on Lou to determine whether a file is on tape or disk. The output shows the state of each file. For example:
lou dmls -l
total 220
-rw-r----- 1 aeneuman madmag 20155 Mar 2 11:24 (REG) A
-rw-r----- 1 aeneuman madmag 1209300 Mar 3 11:24 (OFL) B
-rw-r----- 1 aeneuman madmag 201550 Mar 3 11:18 (DUL) C
In this example:
When a file is in dual state, the online (disk) copy will persist if there is no demand for free space in its filesystem. When free space is needed, the online copy of the file is removed, leaving just the offline copy—in this case, the file state will change from DUL to OFL. If you make any change to a dual-state file, the offline copy becomes out of date and invalid—in this case, DMF will treat it as a completely new file and the file state will change from DUL to REG.
For a list of other possible file states, see Data Migration Facility (DMF) Commands.
Passwords must:
Passwords must not contain:
For more guidance, see Password Creation Rules.
Software programs on NAS systems are managed as modules or packages. For a list of available modules and packages, see Software on NAS Systems.
To learn how to find detailed information about software packages in the pkgsrc collection, see Using Software Packages in pkgsrc.
To find the /nobackup system that is assigned to you, type:
pfe21% ls -ld /nobackup*/your_username
In the following example, the output shows that the /nobackupp17 directory is assigned to user zsmith:
pfe21% ls -ld /nobackup*/zsmith
lrwxrwxrwx 1d root root 19 Dec 23 2024 /nobackup/zsmith -> /nobackupp17/zsmith
For more information about how /nobackup directories are used, see NAS Lustre Filesystems.
You can view the project group IDs (GIDs) that you belong to by logging into the myNAS portal or by running the groups command.
The myNAS portal shows all the GIDs you belong to, along with their project names and descriptions, and provides a central location where you can easily monitor your PBS jobs and check allocation and usage. For more information, see Using the myNAS Portal.
You can run the groups command with your username to list all the GIDs that you belongs to. The following example shows that user zsmith is a member of the groups a0800, a0907, and e0720:
%groups zsmith
zsmith : a0800 a0907 e0720
If you do not know the project names that the GID numbers represent, you can check the myNAS portal or contact the NAS Control Room at (800) 331-8737, (650) 604-4444, or support@nas.nasa.gov.
For more information about project groups and GID charges, see Charging to a Project Group ID (GID).
Yes. When you change your password, the new password is propagated to all of the systems in the NAS high-performance computing (HPC) environment, including:
The password change will propagate to all the systems within a few minutes. If a system continues to request a password change after approximately 15 minutes, or if you see an error message, contact the NAS Control Room at (800) 331-8737 or (650) 604-4444.
For more information, see the HPC Environment Overview.
When you sign into the myNAS web portal, you can easily monitor all aspects of your NAS user account activity, including PBS jobs, resource allocations, and usage for the project group IDs (GIDs) you belong to. You can also check for upcoming downtimes and see the most recent HECC news—at a glance!
To get started, see Using the myNAS Portal.
You can use several methods to transfer files to and from the NAS supercomputers, depending on the size of your files, your system setup, and other considerations. See Remote File Transfer Commands for a list of these methods and how to use them.
There are many possible reasons for poor code performance. If your code is not performing well, see Optimization/Troubleshooting > Code Development for articles that can help you troubleshoot the issue.
If you cannot determine the problem, send an email to support@nas.nasa.gov with the following information:
Your request will be acknowledged and routed to our code optimization specialists for resolution. For more information on HECC code optimization services, see Application Optimization Services.
To check the disk quota on your NAS compute resources home directory (or nobackupnfs2 directory), run the quota ‑vs command.
To check the file quota on your Lou home directory, run the quota ‑vs command from a Lou front-end system (lfe).
To check the disk quota on your NAS Lustre (/nobackup) directory, use lfs quota as follows:
% lfs quota -hu your_username /nobackup/your_username
You can also check quotas and usage for both $Home and Lustre (nobackupp) filesystems on the Data page in the myNAS portal. For more information, see Using the myNAS Portal.
For more information about these filesystems and about quota policies, see the following articles:
To profile a Python application on a NAS system, we recommend using Intel's VTune software package.
For information about how to use VTune, see Finding Hotspots in Your Code with the Intel VTune Command-Line Interface.
Only principal investigators (PIs) can make allocation requests.
If you want to add more hours to your account, contact your PI.
For PIs: See Requesting an Allocation.
Depending on the type of RSA SecurID token you have, see one of the following articles for step-by-step instructions.
If you have any difficulty using your RSA SecurID token, contact the NAS Control Room at (800) 331-8737 or (650) 604-4444.