{toolchain} foss 2016.07 · Pull Request #3517 · easybuilders/easybuild-easyconfigs

ghost · 2016-09-02T13:26:07Z

An update to foss which includes using GCC 6.1.0. All the easyconfigs included are relevant dependencies.

foss 2016.07 includes:

GCC 6.1.0
Open MPI ~~1.10.2~~ 1.10.3
OpenBLAS 0.2.18 (with LAPACK 3.6.1)
FFTW 3.3.4
ScaLAPACK 2.0.2 (with OpenBLAS 0.2.18)

EDIT:
Also contains HPL 2.2 as a test for the toolchain.

ghost · 2016-09-02T14:11:31Z

934 self.assertEqual(ec[key], dumped_ec[key])

935AssertionError: [{'external_module_metadata': {}, 'full_mod_name': 'OpenBLAS/0.2.18-gompi-2016.07-LAPACK-3.6.1', 'toolchain': {'version': '2016.07', 'name': 'gompi'}, 'name': 'OpenBLAS', 'dummy': False, 'short_mod_name': 'OpenBLAS/0.2.18-gompi-2016.07-LAPACK-3.6.1', 'external_module': False, 'versionsuffix': '-LAPACK-3.6.1', 'version': '0.2.18', 'toolchain_inherited': False, 'hidden': False, 'build_only': False}] != [{'external_module_metadata': {}, 'full_mod_name': 'OpenBLAS/0.2.18-gompi-2016.07-LAPACK-3.6.1', 'toolchain': {'version': '2016.07', 'name': 'gompi'}, 'name': 'OpenBLAS', 'dummy': False, 'short_mod_name': 'OpenBLAS/0.2.18-gompi-2016.07-LAPACK-3.6.1', 'external_module': False, 'versionsuffix': '-LAPACK-3.6.1', 'version': '0.2.18', 'toolchain_inherited': True, 'hidden': False, 'build_only': False}]

The difference between them seems to be 'toolchain_inherited', what does this indicate? Is it something to do with OpenBLAS being built with gompi 2016.07 which is included in the commit instead of already being in the repo?

jhein32 · 2016-09-02T14:23:21Z

OpenMPI 1.10.4, bugfix release is out since yesterday. If one includes a new version, should one not move this forward?

ghost · 2016-09-02T14:37:11Z

Yes, that would be good to do in due time but I have been using this toolchain for a few weeks to do builds and since the release of the GCC 6 fixes, this toolchain should be buildable using just easyconfigs (as I had to handbuild GCC 6 into our EasyBuild tree to get it to work before). It's more so for compatibility with what I have already built so far.

boegel · 2016-09-02T15:31:00Z

+
+versionsuffix = "-%s-%s%s" % (blaslib, blasver, blassuff)
+
+dependencies = [(blaslib, blasver, blassuff, ('gompi', '2016.07'))]


@AFCJamie the error in the tests is because you're hardcoding the toolchain to use for this dep to the same toolchain used in this easyconfig, so please change this to:

dependencies = [(blaslib, blasver, blassuff)]

@boegel Ohh, okay, no worries - I'll change that now and push.

boegel · 2016-09-02T15:32:25Z

@jhein32 I agree with @AFCJamie on this one, if this is a well-tested combination of toolchain components, it's good to go in as-is

A toolchain bump to 2016.09 which includes OpenMPI 1.10.4 can be looked into later if deemed useful.

boegel · 2016-09-02T17:18:48Z

@@ -0,0 +1,455 @@
+# All mpfr patches as of 2016-08-04, from same pull request for GCC 6.2.0


@AFCJamie why are you duplicating this patch? There's no need to since this only patches MPFR itself, nothign in GCC, just use the existing patch for mpfr-3.1.4?

@boegel I've removed it now, I'm not sure why I thought it'd be specific to the GCC version.

boegel · 2016-09-02T18:29:38Z

Test report by @boegel
SUCCESS
Build succeeded for 15 out of 15 (15 easyconfigs in this PR)
node2404.golett.os - Linux centos linux 7.2.1511, Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, Python 2.7.5
See https://gist.github.com/c4fd01f77c0104ef45ec0868e963bafb for a full test report.

boegel · 2016-09-02T18:31:21Z

Test report by @boegel
SUCCESS
Build succeeded for 15 out of 15 (15 easyconfigs in this PR)
node2023.delcatty.os - Linux SL 6.8, Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz, Python 2.6.6
See https://gist.github.com/7564113d5ab659fdeeb976ff35ccf1f8 for a full test report.

boegel · 2016-09-02T18:46:51Z

@AFCJamie next to the comments above, please also add an easyconfig for HPL v2.2 as a test case for using this toolchain

Changed Open MPI version to 1.10.3, removed duplicate patch, removed mirror link and fixed some style issues.

ghost · 2016-09-03T10:29:46Z

@boegel Working on HPL now.

ghost · 2016-09-03T10:47:27Z

@boegel @jhein32 If there's going to be a toolchain bump to 2016.09 in the near future, could it be worth jumping to Open MPI 2.0.0? Or is it worth waiting a couple more months before using the next major version?

boegel · 2016-09-03T11:25:25Z

This looks good to me now, thanks for your efforts @AFCJamie!
I'll submit a new set of tests reports, if those turn out OK it's good to merge.

W.r.t. OpenMPI 2.x: it's definitely worth considering already (for 2016.09), but it would be nice to test a new version of the foss toolchain that includes the latest OpenMPI 2.x by rebuilding for example the existing foss-2016a easyconfigs, and seeing what issues arise...

In any case, jumping to OpenMPI 2.x or not is a decision we'll need to make for the 'common' 2017a version of foss (early Jan'17). Similarly for GCC 6.x (as opposed to 5.x we have in 2016b now), but that's likely more of a can of worms than OpenMPI, especially w.r.t. C++ support.

boegel · 2016-09-03T13:00:34Z

Test report by @boegel
SUCCESS
Build succeeded for 17 out of 17 (17 easyconfigs in this PR)
node2032.delcatty.os - Linux SL 6.8, Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz, Python 2.6.6
See https://gist.github.com/e6d2a137340a135a3cf9b5e53d4a9b4f for a full test report.

boegel · 2016-09-03T13:03:36Z

Test report by @boegel
SUCCESS
Build succeeded for 17 out of 17 (17 easyconfigs in this PR)
node2559.golett.os - Linux centos linux 7.2.1511, Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, Python 2.7.5
See https://gist.github.com/22aba39351308945b1ea562b4931ce88 for a full test report.

ghost · 2016-09-03T13:50:45Z

@boegel No worries! Thanks for the guidance.

Okay, that sounds like a fair plan of action. With regard to the GCC aspect, is it not feasible to have 2 'common' foss toolchains but the only difference is one uses GCC 6 and the other GCC 5? (e.g. foss 2017a-GCC-6.x.x and foss-2017a-GCC-5.x.x)

boegel · 2016-09-03T15:35:34Z

@AFCJamie The whole point with the common toolchains is that we have a reasonably tested stable toolchain composition that we can rely on for a while (~6 months, sometimes longer).

So, we should pick either GCC 5.x or 6.x for foss/2017a, based on technical arguments.

That doesn't mean there can't be other foss definitions following the <year>.<month> versioning scheme with either GCC version, of course everyone can dedice which toolchain they go with, but sticking to a single 'top-level' toolchain for a while has significant benefits...

boegel · 2016-09-03T16:18:17Z

lgtm

boegel · 2016-09-03T16:18:27Z

Going in, thanks @AFCJamie!

ghost · 2016-09-05T08:37:55Z

@boegel Okay, that makes sense. Thanks for merging!

jhein32 · 2016-09-05T09:13:45Z

Hi James and Kenneth

Right now, I have no opinion on OpenMPI 2.0.x v OpenMPI 1.10.x. This is a major version bump. What I was keen on, what the x. The OpenMPI 1.10.2, used here, is two bugfix releases old with 1.10.3 being out for a long time. 1.10.x is still maintained and the latest bugfix release only came out 4 days ago. I am a bit opposed to having fossils in newly included stuff.

Regarding 2.0.x, there is now also a 2.0.1, came out last week. For newly added toolchains I would see whether one could have 2.0.1 instead of 2.0.0 now.

In general I think EB needs to get foss toolchains out with gcc 6.x and openmpi 2.0.x out, to give it exposure asap. That way, we have will a better idea on what to put in foss/2017a. Having said this, I noted activity regarding gcc 6 and openmpi 2 in the git repository and (so far) choose to invest my time in other places. In that view, for foss, the year.month naming is perhaps not the best. One has the version bumps (minor version) in the current stuff and issue of major version progression.

In that view I really like the 2016a, 2016b, 2017a, … concept. EB seal of approval: good reliable choices that should build almost any package, free from experimental stuff.

Okay that was a bit an opinion piece.

Best wishes
Joachim

On 03 Sep 2016, at 12:47, James Birch <[email protected]mailto:[email protected]> wrote:

@boegelhttps://github.com/boegel @jhein32 https://github.com/jhein32 If there's going to be a toolchain bump to 2016.09 in the near future, could it be worth jumping to Open MPI 2.0.0? Or is it worth waiting a couple more months before using the next major version?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//pull/3517#issuecomment-244539795, or mute the threadhttps://github.com/notifications/unsubscribe-auth/APK9dCfCgrBme3Q4pmr9PkBhLk1FosQqks5qmVA_gaJpZM4JztGF.

boegel · 2016-09-05T09:24:04Z

@jhein32 just to be clear, we did bump OpenMPI to 1.10.3 here, which aligns with the foss version being used, and avoids downgrading OpenMPI compared to previous foss versions

I agree that defining a foss version that combines GCC 6.x and OpenMPI 2.x should be looked into, since that may make the whole discussion mute; if that combo works out well, it becomes a strong candidate for foss/2017a

jhein32 · 2016-09-05T09:31:52Z

Apologies, the 1.10.2 -> 1.10.3 version bump bypassed me.

Appmaint added 2 commits September 2, 2016 14:15

foss 2016.07 toolchain (uses GCC 6.1.0)

f86546c

missed numactl off list

a24c191

boegel added this to the v2.9.0 milestone Sep 2, 2016

boegel reviewed Sep 2, 2016
View reviewed changes

Removed hard-coded dep

1144223

boegel reviewed Sep 2, 2016
View reviewed changes

Appmaint added 2 commits September 3, 2016 11:15

Made requested changes

a8bc587

Changed Open MPI version to 1.10.3, removed duplicate patch, removed mirror link and fixed some style issues.

Forgot to change string substitution args

dfbfc0c

HPL 2.2 test for foss 2016.07

31c24cd

boegel merged commit 0429279 into easybuilders:develop Sep 3, 2016

ghost deleted the foss2016.07 branch September 5, 2016 08:36


		versionsuffix = "-%s-%s%s" % (blaslib, blasver, blassuff)

		dependencies = [(blaslib, blasver, blassuff, ('gompi', '2016.07'))]

		@@ -0,0 +1,455 @@
		# All mpfr patches as of 2016-08-04, from same pull request for GCC 6.2.0

Conversation

ghost commented Sep 2, 2016 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Sep 2, 2016

Uh oh!

jhein32 commented Sep 2, 2016

Uh oh!

ghost commented Sep 2, 2016 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegel Sep 2, 2016

Choose a reason for hiding this comment

Uh oh!

ghost Sep 2, 2016

Choose a reason for hiding this comment

Uh oh!

boegel commented Sep 2, 2016

Uh oh!

boegel Sep 2, 2016

Choose a reason for hiding this comment

Uh oh!

ghost Sep 3, 2016 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

boegel commented Sep 2, 2016

Uh oh!

boegel commented Sep 2, 2016

Uh oh!

boegel commented Sep 2, 2016

Uh oh!

ghost commented Sep 3, 2016 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Sep 3, 2016

Uh oh!

boegel commented Sep 3, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegel commented Sep 3, 2016

Uh oh!

boegel commented Sep 3, 2016

Uh oh!

ghost commented Sep 3, 2016 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegel commented Sep 3, 2016

Uh oh!

boegel commented Sep 3, 2016

Uh oh!

boegel commented Sep 3, 2016

Uh oh!

ghost commented Sep 5, 2016

Uh oh!

jhein32 commented Sep 5, 2016

Uh oh!

boegel commented Sep 5, 2016

Uh oh!

jhein32 commented Sep 5, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ghost commented Sep 2, 2016 •

edited by ghost

Loading

ghost commented Sep 2, 2016 •

edited by ghost

Loading

ghost Sep 3, 2016 •

edited by ghost

Loading

ghost commented Sep 3, 2016 •

edited by ghost

Loading

boegel commented Sep 3, 2016 •

edited

Loading

ghost commented Sep 3, 2016 •

edited by ghost

Loading