The maximum number of OpenMP threads that an application can use on a compute node depends on:
The number of physical processor cores in the node
Whether hyperthreading is available and enabled
With hyperthreading, the operating system views each physical core as two logical processors, and can therefore assign two threads per core. This is beneficial only when one thread does not keep the functional units in the core busy all the time and can share the resources in the core with another thread. Running in this mode may take less than twice the wall time compared to running only one thread on the core.
Hyperthreading technology is available and enabled on all Electra and Aitken processor types. See the following table for the maximum number of threads possible on each processor type:
Processor Type
Maximum Threads
without Hyperthreading
Maximum Threads
with Hyperthreading
Broadwell
28
56
Skylake
40
80
Cascade Lake
40
80
Rome
128
256
Milan
128
256
TIP: Hyperthreading does not benefit all applications. Also, some applications may show improvement with some process counts but not with others, and there may be other unforeseen issues. Therefore, before using this technology in your production run, you should test your applications with and without hyperthreading. If your application runs more than twice as slow with hyperthreading than without, do not use it.
Here is sample PBS script for running OpenMP applications on a NAS Skylake node without hyperthreading:
#PBS -lselect=1:ncpus=40:ompthreads=40:model=sky_ele, walltime=1:00:00
module load comp-intel
cd $PBS_O_WORKDIR
./your_executable
Here is sample PBS script with hyperthreading:
#PBS -lselect=1:ncpus=40:ompthreads=80:model=sky_ele, walltime=1:00:00
module load comp-intel
cd $PBS_O_WORKDIR
./your_executable
