There are several settings on the NAS front-end systems and compute nodes that define hard and soft limits to control the amount of resources allowed per user, or per user process. You can change the default soft limits for your sessions, up to the maximum hard limits, as needed.
Keep in mind that there might be different default limits set for an SSH login session and a PBS session, and among the various front-end systems and compute nodes.
Viewing Soft and Hard Limits
To view the soft limit settings in a session, run:
For csh: % limit
For bash: $ ulimit -a
The output samples below show soft limit settings on an Aitken front-end (afe01 or afe02).
$ ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 2060780
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 16384
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 300000
cpu time (seconds, -t) unlimited
max user processes (-u) 2060780
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
To view the hard limits, run:
For csh: % limit -h
For bash: $ ulimit -Ha
Modifying Soft Limits
To modify the soft limit setting of a resource, type the name of the resource after limit (for csh) or type the option after ulimit (for bash). For example, to reset the stacksize limit to 200,000 KB, run:
For csh: % limit stacksize 200000
For bash: $ ulimit -Ss 200000
Common Issues with the Default Limits
The following list describes a few common issues with default limits.
stacksize (stack size)
Some plotting packages, such as Tecplot, require a small stack size to function properly. On systems where the stacksize is set to unlimited, you will have to decrease the stacksize in order to use them. However, if the stacksize is set too small when applications are run, segmentation faults (segfaults) commonly occur. If you run plotting packages and your own applications in the same session, you will need to alternate between a small and a large stack size accordingly.
maxproc (maximum user processes)
If you encounter one of the following error messages, this indicates that you have reached the maxproc soft limit:
can't fork process
or
fork: Resource temporarily unavailable
This may happen during compilation, especially if you run a parallel make operation using make -j.
Note: Each Tecplot session can consume more than 100 user processes. If you run multiple instances of Tecplot on the same front-end system where maxproc is set to a small value, you might reach the maxproc soft limit.
memoryuse and vmemoryuse (maximum memory size and virtual memory)
The vmemoryuse setting affects the outcome when you try to allocate memory—for example, when you use the allocate command as follows:
allocate (a(i),stat=ierror)
The memoryuse setting affects the referencing of array a. Keep in mind that running your applications on nodes with different settings will result in different behaviors.
coredumpsize (core file size)
Core files are useful for debugging your application. However, large core files may consume too much of your disk quota. In addition, the generation of large core files in a PBS job could occupy the buffer cache of the nodes, waiting to be flushed to disk. If the buffer cache on the nodes is not flushed by the PBS epilogue of the current job and the PBS prologue of the next job, the nodes will be unusable.
