Skip to content

Use CPATH/LD_LIBRARY_PATH for CMake's find_path/find_library functions.#1165

Merged
boegel merged 2 commits intoeasybuilders:developfrom
akesandgren:more-include_lib-search-paths-for-cmakemake
Jul 5, 2018
Merged

Use CPATH/LD_LIBRARY_PATH for CMake's find_path/find_library functions.#1165
boegel merged 2 commits intoeasybuilders:developfrom
akesandgren:more-include_lib-search-paths-for-cmakemake

Conversation

@akesandgren
Copy link
Copy Markdown
Contributor

Using only self.toolchain.get_variable("CPPFLAGS", list) when setting
CMAKE_INCLUDE_PATH results in cmake incapable of finding things coming
from (build)dependencies modules.

One example is GROMACS which uses find_path/library to find hwloc and
either fails completely or finds any hwloc installed from the OS. When
using foss it should pick up the path to hwloc from the corresponding
module.

See issue #1164

Using only self.toolchain.get_variable("CPPFLAGS", list) when setting
CMAKE_INCLUDE_PATH results in cmake incapable of finding things coming
from (build)dependencies modules.

One example is GROMACS which uses find_path/library to find hwloc and
either fails completely or finds any hwloc installed from the OS. When
using foss it should pick up the path to hwloc from the corresponding
module.
wpoely86
wpoely86 previously approved these changes Apr 13, 2017
@boegel boegel modified the milestone: 3.2.0 May 2, 2017
@boegel
Copy link
Copy Markdown
Member

boegel commented May 2, 2017

lgtm

I'll try and test this thoroughly to see if this raises any problems, but it's indeed the right thing to do.

I hope to include this in the upcoming EasyBuild v3.2.0...

@boegel
Copy link
Copy Markdown
Member

boegel commented May 2, 2017

Testing showed that this change somehow breaks the installation of ANTs-2.1.0rc3-goolf-1.5.14.eb (and possibly others).

I'm not quite sure how though, since the output of the cmake command in the configure step looks identical.

The build fails with errors like this (with EasyBuild configured to use /prefix):

cp: cannot create regular file ‘/prefix/software/ANTs/2.1.0rc3-goolf-1.5.14/ITKv4/Modules/Remote/MGHIO/.git/objects/pack/pack-f3f9b3791c291e460f99b75a7eb3f4f4e1e02508.idx’: Permission denied
cp: cannot create regular file ‘/prefix/software/ANTs/2.1.0rc3-goolf-1.5.14/ITKv4/Modules/Remote/MGHIO/.git/objects/pack/pack-f3f9b3791c291e460f99b75a7eb3f4f4e1e02508.pack’: Permission denied
cp: cannot create regular file ‘/prefix/software/ANTs/2.1.0rc3-goolf-1.5.14/ITKv4/.git/objects/pack/pack-4364526d8e63450d626d092bd29932e7a2f390af.pack’: Permission denied
cp: cannot create regular file ‘/prefix/software/ANTs/2.1.0rc3-goolf-1.5.14/ITKv4/.git/objects/pack/pack-4364526d8e63450d626d092bd29932e7a2f390af.idx’: Permission denied

This problem does not occur without this patch to the CMakeMake easyblock...

@boegel boegel modified the milestones: 3.3.0, 3.2.0 May 2, 2017
@akesandgren
Copy link
Copy Markdown
Contributor Author

Why is something running around copying stuff beneath .git??

And that change can't cause permission changes in itself...

@wpoely86
Copy link
Copy Markdown
Member

wpoely86 commented May 2, 2017

@boegel please upload the cmake logs.

@boegel boegel modified the milestones: 3.3.0, 3.x Jun 22, 2017
@akesandgren
Copy link
Copy Markdown
Contributor Author

akesandgren commented Aug 17, 2017

@boegel Is this still a problem?

@akesandgren
Copy link
Copy Markdown
Contributor Author

ANTs should be scrapped.
The latest version is 2.1.0 which still doesn't understand GCC>4

@akesandgren
Copy link
Copy Markdown
Contributor Author

@boegel How about this one? is the ant stuff still giving you problems?

cpaths = os.getenv('CPATH', '').split(os.pathsep)
lpaths = os.getenv('LD_LIBRARY_PATH', '').split(os.pathsep)
include_paths = os.pathsep.join(tc_ipaths + cpaths)
library_paths = os.pathsep.join(tc_lpaths + lpaths)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we avoid duplicates here? You can use nub (which you can import from vsc.utils.missing):

include_paths = os.pathsep.join(nub(tc_ipaths + cpaths))
library_paths = os.pathsep.join(nub(tc_lpaths + lpaths))

@akesandgren
Copy link
Copy Markdown
Contributor Author

Any more comments on this?
It is still needed to correctly build GROMACS with CUDA.

Without it it fails to find NVML at least.

@akesandgren
Copy link
Copy Markdown
Contributor Author

@migueldiascosta Adding you too since you seem to be very active today :-)

@migueldiascosta
Copy link
Copy Markdown
Member

@akesandgren no good deed goes unpunished? :)

this looks sensible to me, will test it on some CMakeMake easyconfigs

@akesandgren
Copy link
Copy Markdown
Contributor Author

As you might guess I'd like to get this into 3.6.2 too.

We've been using it for quite some time now without and problems.

@boegel boegel removed this from the 3.x milestone Jul 5, 2018
@boegel boegel added this to the 3.6.2 milestone Jul 5, 2018
@boegel boegel added the change label Jul 5, 2018
@boegel
Copy link
Copy Markdown
Member

boegel commented Jul 5, 2018

Sorry for letting this sit for so long @akesandgren...

I think this is good to go, it shouldn't cause any problems, and if it does, it likely was a problem already that we just weren't aware of yet.

We'll see if anything serious pops up during the upcoming regression test for EasyBuild v3.6.2...

@boegel boegel merged commit cfdc645 into easybuilders:develop Jul 5, 2018
@boegel boegel changed the title Use CPATH/LD_LIBRARY_PATH for Cmakes find_path/find_library functions. Use CPATH/LD_LIBRARY_PATH for CMake's find_path/find_library functions. Jul 5, 2018
@akesandgren akesandgren deleted the more-include_lib-search-paths-for-cmakemake branch July 5, 2018 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants