Skip to content

include '-ldl' in $LIBBLAS*/* & co when linking with Intel MKL 11.x or newer#1761

Merged
boegel merged 6 commits intoeasybuilders:developfrom
boegel:imkl_dl
May 12, 2016
Merged

include '-ldl' in $LIBBLAS*/* & co when linking with Intel MKL 11.x or newer#1761
boegel merged 6 commits intoeasybuilders:developfrom
boegel:imkl_dl

Conversation

@boegel
Copy link
Copy Markdown
Member

@boegel boegel commented May 12, 2016

This change results in -lm -ldl being added to $LIBBLAS, $LIBLAPACK, etc. for Intel MKL 11.0 and newer, following the discussion in easybuilders/easybuild-easyconfigs#2900, and as instructed by the Intel MKL Link Advisor.

This is required in particular for building HPL with PGI, cfr. easybuilders/easybuild-easyconfigs#3046.

@boegel boegel added this to the v2.8.0 milestone May 12, 2016
Comment thread easybuild/toolchains/linalg/intelmkl.py Outdated
# but it only seems to be strictly required for certain compilers
mkl_version = self.get_software_version(self.BLAS_MODULE_NAME)[0]
if LooseVersion(mkl_version) >= LooseVersion('11'):
#if LooseVersion(mkl_version) >= LooseVersion('11') and self.COMPILER_FAMILY in [TC_CONSTANT_PGI]:
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little bit reluctant to do this for any compiler, but since this is what is advised by the Intel MKL Link Advisor, it should be fine I think.

Thoughts, @bartoldeman, @ocaisa, @damianam, @wpoely86?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's not needed, the linker will discard them?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but my biggest concern is: what if they're not around? I'm not sure if that's realistic though, they're probably always there...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The math library and the dynamic linking library? If these are not around, we will have bigger concerns that this. Sure, glibc-devel is needed but that's a requirement anyway.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is glibc-devel a requirement already (just to be clear)?

Copy link
Copy Markdown
Member

@wpoely86 wpoely86 May 12, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ rpm -qR gcc-5.3.1-6.fc23.x86_64
/bin/sh
/bin/sh
/bin/sh
/sbin/install-info
/sbin/install-info
binutils >= 2.24
cpp = 5.3.1-6.fc23
glibc-devel >= 2.2.90-12
...

Like for example libpthread.so is also in there.

@bartoldeman
Copy link
Copy Markdown
Contributor

Conservatively I think it's only really needed for PGI + static linking only, and then only -ldl (not -lm).

Progressively the link advisor also suggests -lpthread (though perhaps not everywhere?). Either way I don't have a strong opinion on this, whatever works works...

@boegel
Copy link
Copy Markdown
Member Author

boegel commented May 12, 2016

I did see this pop up, when compiling something with the Intel compilers that was using $LIBBLAS in a funny place (quite early in the link line):

ifort: warning #10315: specifying -lm before files may supercede the Intel(R) math library and affect performance

That makes me want to reconsider either doing this for all compilers or only slapping on -ldl.

@bartoldeman why would -lm not be required?

@bartoldeman
Copy link
Copy Markdown
Contributor

lm and lpthread are not required for PGI because they are already included by pgcc:

pgcc -v hello.c
...
/software/CentOS-6/eb/software/Toolchain/GCCcore/4.9.3/binutils/2.25/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /sb/software/CentOS-6/eb/software/Core/PGI/16.3-GCC-4.9.3-2.25/linux86-64/16.3/lib/trace_init.o /sb/software/CentOS-6/eb/software/Core/GCCcore/4.9.3/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.9.3/crtbegin.o /sb/software/CentOS-6/eb/software/Core/PGI/16.3-GCC-4.9.3-2.25/linux86-64/16.3/lib/initmp.o --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /sb/software/CentOS-6/eb/software/Core/PGI/16.3-GCC-4.9.3-2.25/linux86-64/16.3/lib/pgi.ld -L/sb/software/CentOS-6/eb/software/Core/PGI/16.3-GCC-4.9.3-2.25/linux86-64/16.3/lib -L/usr/lib64 -L/sb/software/CentOS-6/eb/software/Core/GCCcore/4.9.3/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.9.3 /tmp/pgccspCb24NhjivM.o -rpath /sb/software/CentOS-6/eb/software/Core/PGI/16.3-GCC-4.9.3-2.25/linux86-64/16.3/lib -lpgmp -lnuma -lpthread -lnspgc -lpgc -lm -lgcc -lc -lgcc /sb/software/CentOS-6/eb/software/Core/GCCcore/4.9.3/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.9.3/crtend.o /usr/lib64/crtn.o

@bartoldeman
Copy link
Copy Markdown
Contributor

Similarly, icc includes -lm and -ldl by default.

@boegel
Copy link
Copy Markdown
Member Author

boegel commented May 12, 2016

So, is PGI not including -ldl a bug?

Anyway, it seems like w can switch this to only including -ldl for PGI, and maybe always (not just with Intel MKL), similar to how we consistently include -lm for GCC (which may only be needed for older versions of GCC): https://github.com/hpcugent/easybuild-framework/blob/master/easybuild/toolchains/compiler/gcc.py#L89

@bartoldeman
Copy link
Copy Markdown
Contributor

No it's not a bug, not including -ldl. Many programs compile without using -ldl.

It's just static MKL that needs it.

It will probably not hurt to add -ldl everywhere (as icc does it by default for some reason) but from a minimalistic point of view it is not necessary. I'm tending to favour the most minimalistic change (only use -ldl for statically linking MKL).

@boegel
Copy link
Copy Markdown
Member Author

boegel commented May 12, 2016

@bartoldeman OK, thanks for the feedback, I changed things to only slap on -ldl.

@boegel
Copy link
Copy Markdown
Member Author

boegel commented May 12, 2016

@boegel boegel changed the title include '-lm -ldl' in $LIBBLAS*/* & co when linking with Intel MKL 11.x or newer include '-ldl' in $LIBBLAS*/* & co when linking with Intel MKL 11.x or newer May 12, 2016
@boegel
Copy link
Copy Markdown
Member Author

boegel commented May 12, 2016

Good to go now, thanks for the feedback @bartoldeman and @wpoely86!

@boegel boegel merged commit 8177da1 into easybuilders:develop May 12, 2016
@boegel boegel deleted the imkl_dl branch May 12, 2016 20:58
@bartoldeman
Copy link
Copy Markdown
Contributor

Thanks, I can confirm it works.

@boegel
Copy link
Copy Markdown
Member Author

boegel commented May 14, 2016

I had to make the inclusion of -ldl conditional on using PGI after all, since it breaking backwards compatibility, cfr. #1764

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants