Skip to content

enhance OpenFOAM easyblock to add symlinks for libraries to ensure 'mpi' versions have preference over 'dummy' versions#2196

Merged
boegel merged 3 commits intoeasybuilders:developfrom
rug-cit-hpc:openfoam_symlinks
Oct 13, 2020
Merged

enhance OpenFOAM easyblock to add symlinks for libraries to ensure 'mpi' versions have preference over 'dummy' versions#2196
boegel merged 3 commits intoeasybuilders:developfrom
rug-cit-hpc:openfoam_symlinks

Conversation

@bedroge
Copy link
Copy Markdown
Contributor

@bedroge bedroge commented Oct 5, 2020

Some OpenFOAM tools, e.g. snappyHexMesh have MPI issues when RPATH is being used, because platforms/linux64GccDPInt32Opt/lib/mpi is not added to RPATH, while platforms/linux64GccDPInt32Opt/lib and platforms/linux64GccDPInt32Opt/lib/dummy are. This will make some tools pick up, for instance, libPstream.so from lib/dummy, while they should use the one from lib/mpi. The result is that they will not run with the -parallel flag, i.e. that they don't support MPI.

Spack seems to solve this issue by making symlinks in lib to all libraries in lib/mpi. Since lib takes precedence over dummy in the RPATH, the affected tools will then pick up the correct libraries (with MPI support).
This PR adds similar functionality to the install_step.

I've (only) tested this with OpenFOAM/8-foss-2020a, and it seems to work well for that version.

Also see: EESSI/software-layer#24
and: #1193

@boegel boegel added the bug fix label Oct 5, 2020
@boegel boegel added this to the next release (4.3.1) milestone Oct 5, 2020
@migueldiascosta
Copy link
Copy Markdown
Member

migueldiascosta commented Oct 11, 2020

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-2.2.x-intel-2019a.eb
  • SUCCESS OpenFOAM-2.3.1-intel-2019b.eb
  • SUCCESS OpenFOAM-2.4.0-intel-2019a.eb
  • SUCCESS OpenFOAM-3.0.1-intel-2018a.eb
  • SUCCESS OpenFOAM-5.0-20180108-foss-2018a.eb
  • SUCCESS OpenFOAM-5.0-20180108-intel-2018a.eb
  • SUCCESS OpenFOAM-6-foss-2018b.eb
  • SUCCESS OpenFOAM-6-foss-2019b.eb
  • FAIL (build issue) OpenFOAM-6-intel-2018a.eb (partial log available at https://gist.github.com/46999bf17e9c0c59a4df4ca2d9bfda86)
  • SUCCESS OpenFOAM-7-foss-2019b-20200508.eb
  • SUCCESS OpenFOAM-7-foss-2019b.eb
  • SUCCESS OpenFOAM-8-foss-2020a.eb
  • SUCCESS OpenFOAM-v1606+-foss-2018b.eb
  • SUCCESS OpenFOAM-v1612+-foss-2018b.eb
  • SUCCESS OpenFOAM-v1806-foss-2018b.eb
  • SUCCESS OpenFOAM-v1812-foss-2018b.eb
  • SUCCESS OpenFOAM-v1906-foss-2019b.eb
  • SUCCESS OpenFOAM-v1912-foss-2019b.eb
  • SUCCESS OpenFOAM-v1912-intel-2019b.eb
  • SUCCESS OpenFOAM-v2006-foss-2020a.eb

Build succeeded for 19 out of 20 (20 easyconfigs in total)
sms - Linux centos linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor (zen), Python 2.7.5
See https://gist.github.com/424028a7d88879b2d636abc98b72311e for a full test report.

@bedroge
Copy link
Copy Markdown
Contributor Author

bedroge commented Oct 12, 2020 

/opt/apps/centos7/naples/software/OpenFOAM/6-intel-2018a/OpenFOAM-6/platforms/linux64IccDPInt32Opt/applications/utilities/mesh/manipulation/checkMesh/checkTools.o:
file not recognized: File truncated
make[3]: *** [/opt/apps/centos7/naples/software/OpenFOAM/6-intel-2018a/OpenFOAM-6/platforms/linux64IccDPInt32Opt/bin/checkMesh] Error 1

Hmm, weird error... any idea what happened here?

@boegel boegel changed the title Fix MPI issues for OpenFOAM when RPATH is used enhance OpenFOAM easyblock to add symlinks for libraries to ensure 'mpi' versions have preference over 'dummy' versions Oct 12, 2020
@migueldiascosta
Copy link
Copy Markdown
Member

migueldiascosta commented Oct 13, 2020

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-6-intel-2018a.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
sms - Linux centos linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor (zen), Python 2.7.5
See https://gist.github.com/d37cc198bbe2e39f88f33b4c169ffb7d for a full test report.

@migueldiascosta
Copy link
Copy Markdown
Member

migueldiascosta commented Oct 13, 2020

hm, the failed test was probably some IO error... (?)

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 13, 2020

I'm also testing this extensively, incl. with some OpenFOAM-Extend easyconfigs, test report coming up soon-ish...

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 13, 2020

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-2.2.2-intel-2017a.eb
  • SUCCESS OpenFOAM-2.3.1-intel-2017a.eb
  • SUCCESS OpenFOAM-2.4.0-intel-2017a.eb
  • SUCCESS OpenFOAM-3.0.0-foss-2016a.eb
  • SUCCESS OpenFOAM-4.0-foss-2016b.eb
  • SUCCESS OpenFOAM-4.1-foss-2016b.eb
  • SUCCESS OpenFOAM-5.0-20180108-foss-2018a.eb
  • SUCCESS OpenFOAM-5.0-foss-2017b.eb
  • SUCCESS OpenFOAM-6-foss-2018b.eb
  • SUCCESS OpenFOAM-6-foss-2019b.eb
  • SUCCESS OpenFOAM-7-foss-2019b-20200508.eb
  • SUCCESS OpenFOAM-7-foss-2019b.eb
  • SUCCESS OpenFOAM-8-foss-2020a.eb
  • SUCCESS OpenFOAM-v1606+-foss-2018b.eb
  • SUCCESS OpenFOAM-v1612+-foss-2018b.eb
  • SUCCESS OpenFOAM-v1712-foss-2017b.eb
  • SUCCESS OpenFOAM-v1806-foss-2018b.eb
  • SUCCESS OpenFOAM-v1812-foss-2018b.eb
  • SUCCESS OpenFOAM-v1906-foss-2019b.eb
  • SUCCESS OpenFOAM-v1912-foss-2019b.eb
  • SUCCESS OpenFOAM-v2006-foss-2020a.eb
  • SUCCESS Mako-1.0.7-intel-2018a-Python-2.7.14.eb
  • SUCCESS Mesa-17.3.6-intel-2018a.eb
  • SUCCESS libGLU-9.0.0-intel-2018a.eb
  • SUCCESS Qt5-5.10.1-intel-2018a.eb
  • SUCCESS Qt-4.8.7-intel-2018a.eb
  • SUCCESS CGAL-4.11.1-intel-2018a-Python-2.7.14.eb
  • SUCCESS ParaView-5.4.1-intel-2018a-mpi.eb
  • SUCCESS OpenFOAM-6-intel-2018a.eb

Build succeeded for 29 out of 29 (22 easyconfigs in total)
node2411.golett.os - Linux centos linux 7.8.2003, x86_64, Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (haswell), Python 2.7.5
See https://gist.github.com/daad74714f67da89575718acb47bc10f for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 13, 2020

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-Extend-4.1-20200408-foss-2019b-Python-2.7.16.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3404.kirlia.os - Linux centos linux 7.8.2003, x86_64, Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz (cascadelake), Python 2.7.5
See https://gist.github.com/462b633eba282f7d59ceb22c31a7b2f5 for a full test report.

boegel
boegel approved these changes Oct 13, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Tested on a wide variety of OpenFOAM and OpenFOAM-Extend easyconfigs, and also confirmed that it fixes the reported problem when OpenFOAM is installed with "eb --rpath"

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 13, 2020

I'm working on an enhanced sanity check to catch the problem being fixed here, but I'll open a separate PR for that.

Thanks a lot @bedroge!

@boegel boegel merged commit f98e37d into easybuilders:develop Oct 13, 2020
@bedroge bedroge deleted the openfoam_symlinks branch October 13, 2020 21:58
@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 13, 2020

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-Extend-3.1-gimkl-2.11.5.eb
  • SUCCESS OpenFOAM-Extend-3.2-gimkl-2.11.5.eb
  • SUCCESS OpenFOAM-Extend-4.1-20200408-foss-2019b-Python-2.7.16.eb

Build succeeded for 3 out of 3 (3 easyconfigs in total)
node2411.golett.os - Linux centos linux 7.8.2003, x86_64, Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (haswell), Python 2.7.5
See https://gist.github.com/0a3907765cc73066811ee4f5ab3dbc1a for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Oct 14, 2020

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-Extend-4.0-intel-2019b-Python-2.7.16.eb
  • SUCCESS OpenFOAM-Extend-4.1-20191120-intel-2019b-Python-2.7.16.eb
  • SUCCESS OpenFOAM-Extend-4.1-20200408-foss-2019b-Python-2.7.16.eb

Build succeeded for 3 out of 3 (3 easyconfigs in total)
generoso - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/75866e3ec84b46dfe6d429866acf66bf for a full test report.

@bedroge bedroge mentioned this pull request Oct 14, 2020
Closed
@boegel boegel mentioned this pull request Oct 16, 2020
Merged
@boegel boegel mentioned this pull request Dec 23, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

bug fix

Projects

None yet

Milestone

4.3.1

Development

Successfully merging this pull request may close these issues.

3 participants

@bedroge @migueldiascosta @boegel