Enhance amber.py with support for OpenBLAS and better Intel MPI support + fix support for only installing AmberTools#1305
Conversation
Also make NAB openmp capable.
|
|
||
| if self.with_mpi: | ||
| build_targets.append(('-mpi', 'test.parallel')) | ||
| build_targets.append(('%s' % self.mpi_option, 'test.parallel')) |
There was a problem hiding this comment.
no need for the '%s' % bit, just use self.mpi_option:
build_targets.append((self.mpi_option, 'test.parallel'))|
|
||
| # The NAB compiles need openmp flag | ||
| if self.toolchain.options.get('openmp', None): | ||
| env.setvar('CUSTOMBUILDFLAGS', self.toolchain.get_flag('openmp')) |
There was a problem hiding this comment.
Hmm, shouldn't we pass the full $CFLAGS here? If OpenMP is enabled via toolchainopts, it will include -fopenmp.
There was a problem hiding this comment.
Amber doesn't quite use EB's CFLAGS at all, yet. So at the moment adding just the openmp flag is what should be done for consistency until that is dealt with.
There was a problem hiding this comment.
Well, isn't this an opportunity to do so? What would be the correct way to let Amber honor the compiler flags put in place by EasyBuild? Wouldn't it be defining $CUSTOMBUILDFLAGS equal to $CFLAGS?
There was a problem hiding this comment.
Fixing that requires more patching of makefiles that i haven't had time to dive into. The amber build is somewhat convoluted... There are places where CUSTOMBUILDFLAGS isn't used for instance.
Getting this right is not trivial.
| if mklroot: | ||
| env.setvar('MKL_HOME', mklroot) | ||
| elif openblasroot: | ||
| env.setvar('GOTO', os.environ['LIBLAPACK']) |
There was a problem hiding this comment.
os.getenv('LIBLAPACK', 'LIBLAPACK_NOT_SET') may make more sense
In theory, you could list OpenBLAS as a dependency and build with an MPI toolchain like gompi, which would make the current code crash hard because $LIBLAPACK is not defined.
There was a problem hiding this comment.
Also, should we complain in an else that no LAPACK library was found?
Building without it is probably going to end in tears performance-wise?
There was a problem hiding this comment.
Can't do "-lopenblas -lgfortran" (which LIBLAPACK is with foss) since the openblas could have been built by another compiler, like intel or pgi and in that case -lgfortran is wrong.
And doing a list of all the possible combos is just wrong.
But technically speaking, openblas does, or at least used to , not depend on fortran libs, so LIBLAPACK being -lopenblas -lgfortran is wrong i think. And one could then do just -lopenblas here.
So, at the moment, we really have to rely on LIBLAPACK giving us the correct set of libraries. (And that applies to all other easyblocks/easyconfigs really). But complaining if LIBLAPACK isn't set is a good thing to do.
There was a problem hiding this comment.
@akesandgren Wait, os.getenv('LIBLAPACK') and os.environ['LIBLAPACK'] are both going to give you the value for $LIBLAPACK...
Or are you just pointing out an issue in case OpenBLAS is combined with non-GCC compilers?
If so, please open an issue in the framework repository to track that.
That's orthogonal to this though?
There was a problem hiding this comment.
The issue is OpenBLAS built with non-GCC toolchains and probably that GCC-toolchain should define LIBLAPACK as just -lopenblas in this case, but i have too take a second look at how openblas is being built at the moment. It used to be the case that openblas was fortran-free. This might have changed with later versions and must be verified.
So orthogonal to the matter at hand yes.
Removing useless "'%s' %" construct.
| env.setvar('MKL_HOME', mklroot) | ||
| elif openblasroot: | ||
| lapack = os.getenv('LIBLAPACK', 'LIBLAPACK_NOT_SET') | ||
| if lapack == 'LIBLAPACK_NOT_SET': |
There was a problem hiding this comment.
If you'll check it immediately like this, this should be:
lapack = os.getenv('LIBLAPACK')
if lapack is None:
raise ...
else:
...| elif openblasroot: | ||
| lapack = os.getenv('LIBLAPACK') | ||
| if lapack is None: | ||
| raise EasyBuildError("LIBLAPACK (from OpenBLAS) not found in environement. Not building with a lapack capable toolchain?") |
There was a problem hiding this comment.
@akesandgren This line is too long (> 120 chars). I would just drop the 2nd sentence, since a situation where OpenBLAS is there but $LIBLAPACK is not defined is quite strange...
…pi-support don't use -static for AmberTools + fix sanity check
|
Tested with |
Amber differs between -mpi and -intelmpi
Also make NAB openmp capable.