1

I want to execute the IMB 2017 and HPCC 1.5.0 benchmarks on our HPC to be sure that all is configured correctly.

we have 32 compute nodes, each one with 16 cores and 32GB of memory. each node have a qlogic infiniband card with one port at 40Gb/s.

the OS used is RHEL 6.5 with IBM Platform HPC 4.2.

Ofed used is the : IntelIB-OFED.RHEL6-x86_64.3.5.2.34

GCC : gcc version 4.4.7

I managed to compile IMB and HPCC with both IBM platform MPI (PMPI) and OpenMPI 2.0.1 (OMPI)

  1. IMB
    When executing IMB benchmark with both PMPI and OMPI on Infiniband links I get at most

    #---------------------------------------------------
    # Benchmarking PingPong 
    # #processes = 2 
    #---------------------------------------------------
           #bytes #repetitions      t[usec]   Mbytes/sec
                0         1000         1.51         0.00
                1         1000         1.51         0.63
               ...         ...           ...         ...
          2097152           20       675.20      2962.09
          4194304           10      1320.45      3029.26
    

    3029 MB/s of throughput, I expected more, something near 4000 Mb/s is this result correct?

  2. HPCC
    I used this site to generate the test profile.
    When executing the benchmark, using less than 25 nodes, the test goes without problem, I didn't wait for the test to complete, but my problem is when I start the benchmark on all nodes, after 2 to 5 seconds I get this error message :

    compute014.6359Exhausted 1048576 MQ irecv request descriptors, which usually indicates a user program error or insufficient request descriptors (PSM_MQ_RECVREQS_MAX=1048576)
    

    and the benchmark exits and is killed, it's not the same node, each time it's another node. any idea?
    This are the commands used to start the hpcc benchmark:

    OMPI :

    mpirun -np 512 --display-allocation --mca btl self,sm --mca mtl psm --hostfile hosts32 /shared/build/hpcc-150-blas-ompi-201/hpcc hpccinf.txt
    

    PMPI:

    mpirun -np 512 -PSM -hostfile hosts32 /shared/build/hpcc-150-blas-pmpi/hpcc hpccinf.txt
    

If you need more information, let me know.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.