Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-easyconfigs-pr-builder/2940/
Test PASSed.

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
Linux SLES 11_SP2, Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz, Python 2.6.8
See https://gist.github.com/4db08734020338b2b5bb for a full test report.

@rapostolo: suggestions for a small GROMACS benchmark I can use to verify that this is indeed an MPI-enabled build (i.e. not using the MPI-via-threading fallback)? Maybe something we can run as a part of the build process (i.e., in test_step), so something that ships with GROMACS itself?

Or is there another easy way to check whether 'true' MPI-mode is indeed enabled (a config file after running CMake, binary names, etc.)?

If you run
$ mdrun -version
you will see in the output a line:
"MPI library: none"
if it's without mpi support. Other possibilities are "MPI" (normal MPI support) or "thread-MPI" (internal threaded implementation).

@rapostolo You told me you're looking into building GROMACs statically on Cray, where are you hitting issues? The 'support' tools are built dynamically and mdrun statically with our build.

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
Linux SLES 11_SP2, Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz, Python 2.6.9
See https://gist.github.com/328afaee45a5787deec7 for a full test report.

retested on top of easybuilders/easybuild-easyblocks#513

When running mdrun_mpi -version on the Cray login node where GROMACS was built, I'm running into this though:

$ /project/g89/cscs_demo/software/GROMACS/4.6.7-CrayGNU-5.1.29-mpi/bin/mdrun_mpi 
-bash: /project/g89/cscs_demo/software/GROMACS/4.6.7-CrayGNU-5.1.29-mpi/bin/mdrun_mpi: No such file or directory

$ readelf -a /project/g89/cscs_demo/software/GROMACS/4.6.7-CrayGNU-5.1.29-mpi/bin/mdrun_mpi  | grep Req
      [Requesting program interpreter: /lib/ld64.so.1]

$ ls -l /lib/lib64.so.1
ls: cannot access /lib/lib64.so.1: No such file or directory

@rapostolo: have you seen this before? Is this the Cray compiler driver linking in things that are only available on the workernodes? @pforai: any idea?

Haven't seen that. In general on Cray you need to use their wrapper compilers, e.g. pass it as -DCMAKE_C_COMPILER=cc and -DCMAKE_CXX_COMPILER=CC. You might need also to set CRAYPE_LINK_TYPE=dynamic during compilation.

@rapostolo: we are using the Cray compiler wrappers already; we have learned that setting $CRAYPE_LINK_TYPE is something we also need to do, for some builds (but for GROMACS it seems to be fine)

This is a different issue though, I'm not sure what's going on here. Somehow, /lib/lib64.so.1 is being linked in, but it's not available on the login nodes. This may be the Cray way of doing things though, i.e. assuming that mdrun_mpi is never meant to be used directly on the login nodes, and only via aprun.

I'm going to try rebuilding GROMACS after running export CRAYPE_LINK_TYPE=dynamic, to see if that makes a difference w.r.t. the mdrun_mpi issue.

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
Linux SLES 11_SP2, Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz, Python 2.6.9
See https://gist.github.com/1dc1734c19e500577496 for a full test report.

On 13/05/15 10:38, Kenneth Hoste wrote:

@rapostolo https://github.com/rapostolo: we are using the Cray
compiler wrappers already; we have learned that setting
|$CRAYPE_LINK_TYPE| is something we also need to do, for some builds
(but for GROMACS it seems to be fine)

This is a different issue though, I'm not sure what's going on here.
Somehow, |/lib/lib64.so.1| is being linked in, but it's not available
on the login nodes. This may be the Cray way of doing things though,
i.e. assuming that |mdrun_mpi| is never meant to be used directly on
the login nodes, and only via |aprun|.

—
Reply to this email directly or view it on GitHub
#1590 (comment).

On some systems you can't run mpi linked codes directly on the login
node, only on the compute nodes via aprun.

@rapostolo: yes, I'm aware of that, but then you run into other problems, not 'No such file or directory'

I've just rebuilt GROMACS after setting export CRAYPE_LINK_TYPE=dynamic, and now mdrun_mpi does work:

on login node:

kehoste@daint104:~> mdrun_mpi -version
[Wed May 13 11:20:41 2015] [unknown] Fatal error in MPI_Init: Other MPI error, error stack:
MPIR_Init_thread(547): 
MPID_Init(203).......: channel initialization failed
MPID_Init(584).......:  PMI2 init failed: 1 
Aborted

on worker node, via aprun: https://gist.github.com/gppezzi/0b2480de866c986f64e9

so @pforai: we need to get the export CRAYPE_LINK_TYPE=dynamic included in the Cray toolchain support...

defining $CRAYPE_LINK_TYPE as dynamic tackled in easybuilders/easybuild-framework#1283, retesting this...

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
Linux SLES 11_SP2, Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz, Python 2.6.9
See https://gist.github.com/b33f6b6034d766240dcc for a full test report.

Works like a charm on top of easybuilders/easybuild-framework#1283, so going in.

Thanks for the feedback @pforai and @rapostolo!