Skip to content

Enhance amber.py with support for OpenBLAS and better Intel MPI support + fix support for only installing AmberTools#1305

Merged
boegel merged 7 commits intoeasybuilders:developfrom
akesandgren:enhance-amber-for-better-openblas+mpi-support
Jan 13, 2018
Merged

Enhance amber.py with support for OpenBLAS and better Intel MPI support + fix support for only installing AmberTools#1305
boegel merged 7 commits intoeasybuilders:developfrom
akesandgren:enhance-amber-for-better-openblas+mpi-support

Conversation

@akesandgren
Copy link
Copy Markdown
Contributor

Amber differs between -mpi and -intelmpi

Also make NAB openmp capable.

Comment thread easybuild/easyblocks/a/amber.py Outdated

if self.with_mpi:
build_targets.append(('-mpi', 'test.parallel'))
build_targets.append(('%s' % self.mpi_option, 'test.parallel'))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for the '%s' % bit, just use self.mpi_option:

build_targets.append((self.mpi_option, 'test.parallel'))


# The NAB compiles need openmp flag
if self.toolchain.options.get('openmp', None):
env.setvar('CUSTOMBUILDFLAGS', self.toolchain.get_flag('openmp'))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, shouldn't we pass the full $CFLAGS here? If OpenMP is enabled via toolchainopts, it will include -fopenmp.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amber doesn't quite use EB's CFLAGS at all, yet. So at the moment adding just the openmp flag is what should be done for consistency until that is dealt with.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, isn't this an opportunity to do so? What would be the correct way to let Amber honor the compiler flags put in place by EasyBuild? Wouldn't it be defining $CUSTOMBUILDFLAGS equal to $CFLAGS?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing that requires more patching of makefiles that i haven't had time to dive into. The amber build is somewhat convoluted... There are places where CUSTOMBUILDFLAGS isn't used for instance.
Getting this right is not trivial.

Comment thread easybuild/easyblocks/a/amber.py Outdated
if mklroot:
env.setvar('MKL_HOME', mklroot)
elif openblasroot:
env.setvar('GOTO', os.environ['LIBLAPACK'])
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os.getenv('LIBLAPACK', 'LIBLAPACK_NOT_SET') may make more sense

In theory, you could list OpenBLAS as a dependency and build with an MPI toolchain like gompi, which would make the current code crash hard because $LIBLAPACK is not defined.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should we complain in an else that no LAPACK library was found?

Building without it is probably going to end in tears performance-wise?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't do "-lopenblas -lgfortran" (which LIBLAPACK is with foss) since the openblas could have been built by another compiler, like intel or pgi and in that case -lgfortran is wrong.
And doing a list of all the possible combos is just wrong.
But technically speaking, openblas does, or at least used to , not depend on fortran libs, so LIBLAPACK being -lopenblas -lgfortran is wrong i think. And one could then do just -lopenblas here.

So, at the moment, we really have to rely on LIBLAPACK giving us the correct set of libraries. (And that applies to all other easyblocks/easyconfigs really). But complaining if LIBLAPACK isn't set is a good thing to do.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akesandgren Wait, os.getenv('LIBLAPACK') and os.environ['LIBLAPACK'] are both going to give you the value for $LIBLAPACK...

Or are you just pointing out an issue in case OpenBLAS is combined with non-GCC compilers?

If so, please open an issue in the framework repository to track that.

That's orthogonal to this though?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is OpenBLAS built with non-GCC toolchains and probably that GCC-toolchain should define LIBLAPACK as just -lopenblas in this case, but i have too take a second look at how openblas is being built at the moment. It used to be the case that openblas was fortran-free. This might have changed with later versions and must be verified.

So orthogonal to the matter at hand yes.

@boegel boegel added this to the next release milestone Dec 12, 2017
Removing useless "'%s' %" construct.
Comment thread easybuild/easyblocks/a/amber.py Outdated
env.setvar('MKL_HOME', mklroot)
elif openblasroot:
lapack = os.getenv('LIBLAPACK', 'LIBLAPACK_NOT_SET')
if lapack == 'LIBLAPACK_NOT_SET':
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you'll check it immediately like this, this should be:

lapack = os.getenv('LIBLAPACK')
if lapack is None:
    raise ...
else:
    ...

Comment thread easybuild/easyblocks/a/amber.py Outdated
elif openblasroot:
lapack = os.getenv('LIBLAPACK')
if lapack is None:
raise EasyBuildError("LIBLAPACK (from OpenBLAS) not found in environement. Not building with a lapack capable toolchain?")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akesandgren This line is too long (> 120 chars). I would just drop the 2nd sentence, since a situation where OpenBLAS is there but $LIBLAPACK is not defined is quite strange...

…pi-support

don't use -static for AmberTools + fix sanity check
@boegel boegel changed the title Enhance amber.py with support for OpenBLAS and better Intel MPI support. Enhance amber.py with support for OpenBLAS and better Intel MPI support + fix support for only installing AmberTools Jan 12, 2018
@boegel
Copy link
Copy Markdown
Member

boegel commented Jan 13, 2018

Tested with Amber-14-intel-2016a-AmberTools-15-patchlevel-13-13.eb (the only one I can test) + AmberTools (see easybuilders/easybuild-easyconfigs#5632), looks good, so going in. Thanks @akesandgren!

@boegel boegel merged commit 6da5180 into easybuilders:develop Jan 13, 2018
@akesandgren akesandgren deleted the enhance-amber-for-better-openblas+mpi-support branch February 21, 2018 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants