Skip to content

reset keep toolchain component class 'constants' every time#1294

Merged
boegel merged 10 commits intoeasybuilders:developfrom
boegel:keep_class_constants_constant
Jul 7, 2015
Merged

reset keep toolchain component class 'constants' every time#1294
boegel merged 10 commits intoeasybuilders:developfrom
boegel:keep_class_constants_constant

Conversation

@boegel
Copy link
Copy Markdown
Member

@boegel boegel commented Jun 5, 2015

This fixes a problem that often occurs when testing large easyconfig PRs that contain easyconfigs with a large variety on toolchains using --from-pr, for example easybuilders/easybuild-easyconfigs#1337 (comment) .

If old and new version of Intel compilers are used in the same session, things break because constants aren't exactly constant.

This fixes the problem; I took care of checking the other toolchain components too.

@wpoely86, @pforai, @stdweird: please review?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmmm, this is not inheritance-safe. need to think about this one

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe

if self._init_LIB_MULTITHREAD is None:
    self._init_LIB_MULTITHREAD = self.LIB_MULTITHREAD[:]
else:
    self.LIB_MULTITHREAD = self._init_LIB_MULTITHREAD[:]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and add self._init_... = None to the __init__? (or use a dict to store all such non-constant class constants)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On another note, why do we manually add libiomp5? Using -fopenmp should take care of this?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should, we had a very good reason at some point for this, and I'm not keen on breaking stuff that works just to remove a handful of characters ;)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are libraries you can pass via LDFLAGS, doesn't always work with compiler options. there's probably some crappy code out there that wants library names.

ideally, EB should figure out what libraries -fopenmp links in and use those instead of hardcoding.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stdweird: good point on inheritance

How about simply this in __init__.py:

self._init_LIB_MULTITHREAD = self.LIB_MULTITHREAD[:]

and this here:

self.LIB_MULTITHREAD = self._init_LIB_MULTITHREAD[:]

That should work, no?

Toolchain instances are recreated for different versions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then maybe also make 2 private methods _copy_some_class_constants and _restore_some_class_constants and call/extend those.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't generalise this, it should be considered an exception, not encouraged.

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1732/
EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1732/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1733/
EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1733/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Jun 5, 2015

@stdweird: fixed the inheritance issue you pointed out, please rereview?

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1734/
EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1734/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, please hide this in the __init__, people will start to manipultae this

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1737/
EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1737/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1738/
EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1738/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1739/
EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1739/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Jun 5, 2015

@stdweird: central solution implemented, unit test enhanced to make sure things work as expected when multiple toolchains and toolchain versions are in play

Comment thread easybuild/toolchains/linalg/intelmkl.py Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will remove, thx

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1740/
EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1740/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

Comment thread easybuild/tools/toolchain/toolchain.py Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make a new method

def _init_class_constants(self, names=None):
  self.add_class_constants_to_restore(names or [])
  self._copy_class_constants()  
  self._restore_class_constants()

because this looks so strange otherwise

@stdweird
Copy link
Copy Markdown
Contributor

stdweird commented Jun 6, 2015

although the code looks good, it just feel weird looking at it. but my alternative with the __init__ isn't much better 😄
so feel free to modify or merge

@hpcugentbot
Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1743/
EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1743/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Jun 15, 2015

The last test report for easybuilders/easybuild-easyconfigs#1337, available at easybuilders/easybuild-easyconfigs#1337 (comment), still shows problems, so I need to look at it more closely, and see why this change isn't fixing the issues there...

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Jul 7, 2015

retested with easybuilders/easybuild-easyconfigs#1337, works as intended and fixes the problem it intends to fix (previous failing test was due to issues with Intel license server, not due to EB itself)

Going in, thanks for the review @stdweird!

boegel added a commit that referenced this pull request Jul 7, 2015
reset keep toolchain component class 'constants' every time
@boegel boegel merged commit f1841ab into easybuilders:develop Jul 7, 2015
@boegel boegel deleted the keep_class_constants_constant branch July 7, 2015 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants