Hyperthreading is available and enabled on the Aitken and Electra compute nodes. With hyperthreading, each physical core can function as two logical processors. This means that the operating system can assign two threads per core by assigning one thread to each logical processor.

The following table shows the number of physical cores and potential logical processors available for each processor type:

If you use hyperthreading, you can run an MPI code using 2N processes per node instead of N process per node—so you can use half the number of nodes for your job. Each process will be assigned to run on one logical processor; in reality, two processes are running on the same physical core.

Running two processes per core can take less than twice the wall-clock time compared to running only one process per core—if one process does not keep the functional units in the core busy, and can share the resources in the core with another process.

Benefits and Drawbacks

Using hyperthreading can improve the overall throughput of your jobs, potentially saving standard billing unit (SBU) charges. Also, requesting half the usual number of nodes may allow your job to start running sooner—an added benefit when the systems are loaded with many jobs. However, using hyperthreading may not always result in better performance.

Using Hyperthreading

Hyperthreading can improve the overall throughput, as demonstrated in the following example.

Example 

Consider the following scenario with a job that uses 80 MPI ranks on Skylake. Without hyperthreading, we would specify:

#PBS -lselect=2:ncpus=40:mpiprocs=40:model=sky_ele

and the job will use 2 nodes with 40 processes per node. Suppose that the job takes 1000 seconds when run this way. If we run the job with hyperthreading, e.g.:

#PBS -lselect=1:ncpus=40:mpiprocs=80:model=sky_ele

then the job will use 1 node with all 80 processes running on that node. Suppose this job takes 1800 seconds to complete.

Without hyperthreading, we used 2 nodes for 1000 seconds (a total of 2000 node-seconds); with hyperthreading, we used 1 node for 1800 seconds (1800 node-seconds). Thus, under these circumstances, if you were interested in getting the best wall-clock time performance for a single job, you would use two nodes without hyperthreading. However, if you were interested in minimizing resource usage, especially with multiple jobs running simultaneously, using hyperthreading would save you 10% in SBU charges.

Mapping of Physical Core IDs and Logical Processor IDs

Mapping between the physical core IDs and the logical processor IDs is summarized in the following table. The value of N is 28, 40, and 128 for Broadwell, Skylake/Cascade Lake, and Rome/Milan processor types, respectively.

Note: For additional mapping details, see the configuration diagrams at the for each processor type, or run the cat /proc/cpuinfo command on the specific node type.