mpiP

mpiP is a lightweight profiling library for MPI applications.

This page demonstrates usage on our systems and offers a short tour of some of the functionality – please see the website for more details:

http://mpip.sourceforge.net/

Basic usage

Before using, please execute the following to make the software available:


module add mpiP

Profile an application by modifying the normal mpirun command within a job script to read:


env LD_PRELOAD=${MPIP_HOME}/lib/libmpiP.so mpirun 

At the start of execution, mpiP will emit a header the terminal, so that it is clear it is working:


mpiP: 
mpiP: mpiP: mpiP V3.4.1 (Build Feb  9 2015/10:05:24)
mpiP: Direct questions and errors to mpip-help@lists.sourceforge.net
mpiP: 

A text file will be created, containing MPI information about the run, including aggregate information on number, size and duration of each MPI call in the application.

If the application has been compiled with -g , then mpiP will attempt to show what file and line number of each MPI call.

Configuration

The behaviour of mpiP can be influenced by setting the MPIP environment variable, enabling it to print more or less detail: see the project website for more information.

Example output


$ more swaparray.32.24013.1.mpiP
@ mpiP
@ Command : ./swaparray 
@ Version                  : 3.4.1
@ MPIP Build date          : Feb  9 2015, 10:05:24
@ Start time               : 2015 02 09 10:22:05
@ Stop time                : 2015 02 09 10:22:05
@ Timer Used               : gettimeofday
@ MPIP env var             : [null]
@ Collector Rank           : 0
@ Collector PID            : 24013
@ Final Output Dir         : .
@ Report generation        : Collective
@ MPI Task Assignment      : 0 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 1 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 2 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 3 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 4 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 5 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 6 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 7 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 8 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 9 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 10 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 11 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 12 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 13 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 14 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 15 h3s3b10.arc2.leeds.ac.uk
@ MPI Task Assignment      : 16 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 17 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 18 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 19 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 20 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 21 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 22 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 23 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 24 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 25 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 26 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 27 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 28 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 29 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 30 h3s3b11.arc2.leeds.ac.uk
@ MPI Task Assignment      : 31 h3s3b11.arc2.leeds.ac.uk

---------------------------------------------------------------------------
@--- MPI Time (seconds) ---------------------------------------------------
---------------------------------------------------------------------------
Task    AppTime    MPITime     MPI%
   0    0.00755    0.00521    69.05
   1    0.00905    0.00049     5.41
   2    6.9e-05          0     0.00
   3    6.4e-05          0     0.00
   4    6.4e-05          0     0.00
   5    6.2e-05          0     0.00
   6    6.2e-05          0     0.00
   7    6.7e-05          0     0.00
   8    6.2e-05          0     0.00
   9    7.8e-05          0     0.00
  10    6.2e-05          0     0.00
  11    9.9e-05          0     0.00
  12   0.000102          0     0.00
  13    7.4e-05          0     0.00
  14    6.4e-05          0     0.00
  15    6.6e-05          0     0.00
  16    6.3e-05          0     0.00
  17   0.000111          0     0.00
  18    6.4e-05          0     0.00
  19    9.3e-05          0     0.00
  20   0.000116          0     0.00
  21    7.1e-05          0     0.00
  22    6.3e-05          0     0.00
  23    6.4e-05          0     0.00
  24   0.000116          0     0.00
  25    6.7e-05          0     0.00
  26   0.000116          0     0.00
  27    6.3e-05          0     0.00
  28    6.7e-05          0     0.00
  29    6.5e-05          0     0.00
  30    6.7e-05          0     0.00
  31    6.7e-05          0     0.00
   *     0.0189     0.0057    30.23
---------------------------------------------------------------------------
@--- Callsites: 2 ---------------------------------------------------------
---------------------------------------------------------------------------
 ID Lev File/Address        Line Parent_Funct             MPI_Call
  1   0 swaparray.f90         46 swaparray                Ssend
  2   0 swaparray.f90         58 swaparray                Recv
---------------------------------------------------------------------------
@--- Aggregate Time (top twenty, descending, milliseconds) ----------------
---------------------------------------------------------------------------
Call                 Site       Time    App%    MPI%     COV
Ssend                   1       5.21   27.63   91.41    0.00
Recv                    2       0.49    2.60    8.59    0.00
---------------------------------------------------------------------------
@--- Aggregate Sent Message Size (top twenty, descending, bytes) ----------
---------------------------------------------------------------------------
Call                 Site      Count      Total       Avrg  Sent%
Ssend                   1          1    1.6e+06    1.6e+06 100.00
---------------------------------------------------------------------------
@--- Callsite Time statistics (all, milliseconds): 2 ----------------------
---------------------------------------------------------------------------
Name              Site Rank  Count      Max     Mean      Min   App%   MPI%
Recv                 2    1      1     0.49     0.49     0.49   5.41 100.00
Recv                 2    *      1     0.49     0.49        0   2.60   8.59

Ssend                1    0      1     5.21     5.21     5.21  69.05 100.00
Ssend                1    *      1     5.21     5.21        0  27.63  91.41

---------------------------------------------------------------------------
@--- Callsite Message Sent statistics (all, sent bytes) -------------------
---------------------------------------------------------------------------
Name              Site Rank   Count       Max      Mean       Min       Sum
Ssend                1    0       1   1.6e+06   1.6e+06   1.6e+06   1.6e+06
Ssend                1    *       1   1.6e+06   1.6e+06   1.6e+06   1.6e+06

---------------------------------------------------------------------------
@--- End of Report --------------------------------------------------------