TotalView is a GUI-based debugging tool that provides control over processes and thread execution, as well as visibility into program state and variables, for C, C++ and Fortran applications. It also provides memory debugging to detect errors such as memory leaks, deadlocks, and race conditions. You can use TotalView to debug serial, OpenMP, or MPI codes.

Using TotalView to Debug Your Code

To find out which versions are available as modules, use the module avail command.

Before You Begin

Our current licenses allow using TotalView with MemoryScape, ReplayEngine, and CUDA support for up to a total of 256 processes on NAS x86_64 cluster systems and shared memory systems. Use the following command to find out whether there are unused licenses before you start TotalView:

pfe% /u/scicon/tools/bin/check_licenses -t

Ensure that X11 forwarding is set up on your system. Alternately, you can use the ssh -X or -Y options to enable X11 forwarding for your SSH session.

Note: If you are using a NAS-supported workstation or compute server, X11 forwarding should already be set up on your system. If the response of the GUI via X11 forwarding is slow, you will need to set up a VNC session.

Complete the following steps:

1. Compile your program using the -g option.
2. Start a PBS session.

   On on NAS x86_64 cluster systems:

   % qsub -I -X -q devel -lselect=2:ncpus=8:model=xxx,walltime=1:00:00

   where xxx is one of the following: bro_ele, sky_ele, cas_ait, rom_ait, or mil_ait.

   On Endeavour3/4:

   % qsub -I -X  -lselect=1:ncpus=28:mem=185GB:model=cas_end,walltime=1:00:00 \
   -q queue_name@pbspl4

   Note: The command line above is too long to be formatted as one line, so it is broken with a backslash (\).

3. Test the X11 forwarding with xclock:

   % xclock 

Debugging with TotalView

Load the module:

% module load totalview/2023.4.16

The method you use to run TotalView depends on the application you want to debug.

For Serial Applications

Launch TotalView by running the totalview command and specifying the application:

% totalview ./a.out 

Or, if your application requires arguments:

% totalview ./a.out -a arg_1 arg_2 

For MPI Applications Built with HPE MPT

The old -tv option of HPE MPT's mpiexec_mpt is no longer supported (since mpt.2.23). Use the following method instead:

1. Load the latest MPT module:

   % module load comp-intel
   % module load mpi-hpe/mpt

2. Launch your program as follows:

   % totalview mpiexec_mpt.real -a -np 16 ./a.out 

   If the MPI launcher fails to launch the executable and recommends setting MPI_SHEPHERD=true, then set that environment variable and try running TotalView again.

For more information, see the TotalView documentation.