Advanced SGE: GPGPU

The ARC3 system is the first cluster at Leeds to include GPU accelerator technologies.

In addition to the standard GPU nodes, the cluster provides 2 GPU nodes each with 24 cores and 128GB of memory, an 800GB hard disk drive and two NVIDIA K80 cards.


The scheduler has been extended to allow requests for GPU resources. To request K80 GPU resource you should use the flag:

#$ -l coproc_k80=<cards_per_compute_node>
Where <cards_per_compute_node> should be set to 1 or 2
#$ -l coproc_k80=1
Will allocate 1 x K80 card and half the other resources on the node:
  • 12 CPU cores
  • 64 GB memory
#$ -l coproc_k80=2Will allocate 2 x K80 cards and all resources on the node:
  • 24 CPU cores
  • 128 GB memory

To use the GPU cards you will need to ensure that the NVIDIA CUDA toolkit module is loaded into your environment:

module load cuda
This needs to be done before compiling or running GPU code.

Note that this version of the CUDA environment will only work with certain compiler versions:

  • INTEL versions 15 and 16
  • PGI versions >= 16.3
  • GNU version 4.8.2 (this is the gnu/native compiler

To confirm what cards you have been allocated use the command:

nvidia-smi -L
To compile on the K80 NVIDIA GPU you will need to load the cuda module.

module load cuda