Skip to content

Override impi defaults for mpicc etc.#493

Merged
boegel merged 20 commits intoeasybuilders:developfrom
ocaisa:change_impi_default_compiler
Oct 30, 2014
Merged

Override impi defaults for mpicc etc.#493
boegel merged 20 commits intoeasybuilders:developfrom
ocaisa:change_impi_default_compiler

Conversation

@ocaisa
Copy link
Copy Markdown
Member

@ocaisa ocaisa commented Oct 24, 2014

Added a switch so that one can override the defaults of the impi installation. Defaults are to wrap the system compilers with mpicc, mpicxx, mpif90...

The switch overrides this behaviour by setting environment variables.

These environment variables also affect the behaviour of mpigcc and mpigxx so I've added aliases to give back the expected behaviour that they wrap the GNU compilers.

installation. Defaults are to wrap the system compilers with mpicc,
mpicxx, mpif90, ... The switch overrides this behaviour by setting some
environment varaibles. These environment varaibles also affect the
behaviour of mpigcc and mpigxx so I've added aliases to give back the
expected behaviour that they wrap the GNU compilers.
@hpcugentbot
Copy link
Copy Markdown

Automatic reply from Jenkins: Can I test this?

compiler -> compilers
@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 24, 2014

Jenkins: ok to test

Comment thread easybuild/easyblocks/i/impi.py Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indent the comments too please

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and maybe change to # $FC isn't defined by EasyBuild framework, so use $F90 instead

@hpcugentbot
Copy link
Copy Markdown

Test FAILed.

Comment thread easybuild/easyblocks/i/impi.py Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about using a for-loop here:

for var in ['CC', 'CXX', 'F77', 'F90']:
    txt += self.moduleGenerator.set_environment('I_MPI_%s' % var, os.getenv[var])

@hpcugentbot
Copy link
Copy Markdown

Test FAILed.

@hpcugentbot
Copy link
Copy Markdown

Test FAILed.

Updated based on comments
@hpcugentbot
Copy link
Copy Markdown

Test FAILed.

@ocaisa
Copy link
Copy Markdown
Member Author

ocaisa commented Oct 24, 2014

No mpigfortran with Intel MPI so this alias has not been set

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

Comment thread easybuild/easyblocks/i/impi.py Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should use set_mpi_wrappers_compiler now...

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 24, 2014

@geimer: any thoughts on this before we merge it in? Looks good to me: optional, disabled by default, ...

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@geimer
Copy link
Copy Markdown
Contributor

geimer commented Oct 24, 2014

@boegel: The approach looks OK to me.

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 24, 2014

This is working as intended now for impi installations that are using a non-dummy toolchain:

$ eb impi-4.1.1.036-iccifort-2013.5.192.eb --try-amend=set_mpi_wrappers_compiler=True -f
== temporary log file in case of crash /tmp/easybuild-a304cd/easybuild-ewblzT.log
== resolving dependencies ...
== processing EasyBuild easyconfig /tmp/easybuild-a304cd/tweaked_easyconfigs/impi-4.1.1.036-iccifort-2013.5.192.eb
== building and installing impi/4.1.1.036-iccifort-2013.5.192...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== packaging...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/easybuild/easybuild-impi-4.1.1.036-20141024.111248.log
== Build succeeded for 1 out of 1
== temporary log file /tmp/easybuild-a304cd/easybuild-ewblzT.log has been removed.
== temporary directory /tmp/easybuild-a304cd has been removed.

$ module show impi/4.1.1.036-iccifort-2013.5.192 
-------------------------------------------------------------------
/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/modules/all/impi/4.1.1.036-iccifort-2013.5.192:

module-whatis    Description: The Intel(R) MPI Library for Linux* OS is a multi-fabric message
 passing library based on ANL MPICH2 and OSU MVAPICH2. The Intel MPI Library for
 Linux OS implements the Message Passing Interface, version 2 (MPI-2) specification. - Homepage: http://software.intel.com/en-us/intel-mpi-library/ 
conflict     impi 
module       load iccifort/2013.5.192 
prepend-path     CPATH /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/include64 
prepend-path     LD_LIBRARY_PATH /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/lib64 
prepend-path     LIBRARY_PATH /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/lib64 
prepend-path     PATH /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/bin64 
setenv       EBROOTIMPI /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192 
setenv       EBVERSIONIMPI 4.1.1.036 
setenv       EBDEVELIMPI /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/easybuild/impi-4.1.1.036-iccifort-2013.5.192-easybuild-devel 
prepend-path     INTEL_LICENSE_FILE /user/home/gent/vsc400/vsc40023/licenses/intel/license.lic 
setenv       I_MPI_ROOT /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192 
setenv       I_MPI_CC icc 
setenv       I_MPI_CXX icpc 
setenv       I_MPI_F77 ifort 
setenv       I_MPI_F90 ifort 
setenv       I_MPI_FC ifort 
set-alias    mpigcc mpigcc -cc=gcc 
set-alias    mpigxx mpigxx -cc=g++ 
-------------------------------------------------------------------

$ module load impi/4.1.1.036-iccifort-2013.5.192
$ alias mpigcc
alias mpigcc='mpigcc -cc=gcc'

$ mpicc -show
icc -I/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/intel64/include -L/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.1.036-iccifort-2013.5.192/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/4.1 -lmpigf -lmpi -lmpigi -ldl -lrt -lpthread

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 24, 2014

However, there's a problem for impi that's being installed with a dummy toolchain:

$ eb impi-4.1.2.040.eb --try-amend=set_mpi_wrappers_compiler=True -f
$ module show impi/4.1.2.040 2>&1 | grep I_MPI
setenv       I_MPI_ROOT /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/impi/4.1.2.040 
setenv       I_MPI_CC None 
setenv       I_MPI_CXX None 
setenv       I_MPI_F77 None 
setenv       I_MPI_F90 None 
setenv       I_MPI_FC None 

$CC and co are not set for a dummy toolchain, so we need to check whether $CC is set rather than ending up using None. I'll issue a PR for this to your branch @ocaisa.

Comment thread easybuild/easyblocks/i/impi.py Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ocaisa: shouldn't we apply the same trick to mpiicc, mpiicpc, etc.? What if Intel MPI is installed on top of GCC? Setting $I_MPI_CC to gcc would cause mpiicc to produce unexpected behaviour.

If we fix it for the GCC-specific wrappers, we should also do so for the Intel specific wrappers?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@boegel: IMHO, modifying the normal behaviour of mpiicc sounds kinda overzealous.

Note that this issue relates nicely to case #100 and it is in effect a tunable implementation of the style of the very last proposition there; the bigger picture here is, that sites need control to configure those variables, in a centralized way.

btw.
The very best group of people to ask about desired behaviour of such settings are those involved with benchmarking (thinking of Jube now, but not exclusively) and mpi testing, across multiple architectures...
So, I'd suggest to invite, via an RFC email, experts on or off the mailing list to validate the prototyped solution.

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 26, 2014

@ocaisa: PR for fixing the issue with the None values is open at ocaisa#2

@fgeorgatos
Copy link
Copy Markdown
Contributor

and one more remark: using aliases within modulefiles is a tricky proposition, because they do not get inherited properly on children shells (I don't recall if this is mainly a bash itch or affects more). I used to use them in the past until I got bitten a few times and then learned to find other workarounds (such as adding in the $PATH a binary that actually does the wrapping).

check compiler env vars before using them
@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

Comment thread easybuild/easyblocks/i/impi.py Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a separate parameter for defining aliases for the 'GCC-specific' wrappers, e.g. set_mpi_wrapper_aliases_gcc

Added individual triggers for the environment variables and the aliases, as well as a catch-all for everything
@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

Added a debug error statement
@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@fgeorgatos
Copy link
Copy Markdown
Contributor

Visual check: PASSed :)

@hpcugentbot
Copy link
Copy Markdown

Test PASSed.

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 30, 2014

Going in, thanks a lot for tackling this @ocaisa, and for being so patient with the revisions!

boegel added a commit that referenced this pull request Oct 30, 2014
@boegel boegel merged commit acb04ff into easybuilders:develop Oct 30, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants