Skip to content

add generic ModuleRC easyblock#1503

Merged
ocaisa merged 9 commits intoeasybuilders:developfrom
boegel:modulerc
Sep 13, 2018
Merged

add generic ModuleRC easyblock#1503
ocaisa merged 9 commits intoeasybuilders:developfrom
boegel:modulerc

Conversation

@boegel
Copy link
Copy Markdown
Member

@boegel boegel commented Sep 7, 2018

This is a new generic easyblock that generates a .modulerc file to define a module version alias rather than installing an actual module file (cfr. suggestion by @mboisson in easybuilders/easybuild-easyconfigs#5203 (comment)).

This can be used to for example install a Java/1.8 wrapper (see easybuilders/easybuild-easyconfigs#6712) for another Java module, without causing trouble with for example Lmod's module collection concept...

edit: now requires easybuilders/easybuild-framework#2571 and easybuilders/easybuild-framework#2575

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Sep 7, 2018

I should mention that this doesn't require any changes to EasyBuild framework, strictly speaking...

The aliased module is seen by EasyBuild thanks to the fallback to module show when checking if a module already exists, so that's covered.

The hardcoding of the contents of .modulerc could be handled better though, by enhancing the module_generator module in framework to support something like alias_version in both Tcl & Lua format (which requires Lmod 7.8)...

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Sep 7, 2018

The tests only fail with EnvironmentModulesC 3.2.10, looks like there was a weird bug with module-version there, see https://sourceforge.net/p/modules/mailman/message/31144974 + https://sourceforge.net/p/modules/mailman/message/33399425...

@xdelaruelle Any ideas what's going on here? In this case, there's only a singe module-version statement in the .modulerc file, like this:

#%Module
module-version Java/1.8.0_181 1.8

Then how can it trigger a Duplicate version symbol error?!

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Sep 7, 2018

OK, the problem was probably that there was an actual module file foo/1.3.2 together with a module alias for foo defined via module-version in .modulerc that was also using 1.3.2 as version, which results in a clash...

That should be fixed now by user different versions for the ModulesAlias test.

@easybuilders easybuilders deleted a comment from boegelbot Sep 8, 2018
@xdelaruelle
Copy link
Copy Markdown

@boegel It seems that .modulerc files are parsed twice on version 3.2 which leads to these Duplicate version symbol messages. The symbolic version is however correctly set, so I believe this message could be ignored on your test cases.

@boegel
Copy link
Copy Markdown
Member Author

boegel commented Sep 8, 2018

@xdelaruelle The problem was that there was both a foo/1.2.3.4 module file, and an alias for foo/1.2.3.4.5 named foo/1.2.3.4. After fixing that, all is well.

@easybuilders easybuilders deleted a comment from boegelbot Sep 8, 2018
@boegel boegel changed the title add generic ModulesAlias easyblock add generic ModulesRC easyblock Sep 11, 2018
from easybuild.tools.build_log import EasyBuildError
from easybuild.tools.config import build_option
from easybuild.tools.filetools import write_file
from easybuild.tools.module_generator import ModuleGeneratorTcl
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'easybuild.tools.module_generator.ModuleGeneratorTcl' imported but unused

@easybuilders easybuilders deleted a comment from boegelbot Sep 11, 2018
@easybuilders easybuilders deleted a comment from boegelbot Sep 11, 2018
@boegel boegel requested a review from ocaisa September 11, 2018 19:16
if self.name != deps[0]['name']:
raise EasyBuildError("Name does not match dependency name: %s vs %s", self.name, deps[0]['name'])

# version to prefix of version to alias to
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awful lot of tos there...not sure what you mean, is it

# ensure version to alias to is a prefix match for the version of the dependency

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly that, and you're right, the comment that was there didn't make any sense, it should have been something like # version is prefix of version to alias to, but yours is better :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly that, and you're right, the comment that was there didn't make any sense, it should have been something like # version is prefix of version to alias to, but yours is better :)

modfile_path = self.module_generator.get_module_filepath(fake=fake)
modulerc = os.path.join(os.path.dirname(modfile_path), '.modulerc')

if os.path.exists(modulerc) and not build_option('force'):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implies that you can only have one entry in the .modulerc per software package. Probably ok, but worth noting the limitation

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, that's a rather strict limitation for now, which we can review later when the need for it pops up?

A whole bunch of questions come into play to relax this:

  • What do we do with an existing .modulerc that was not created by EasyBuild?
  • How do we allow both appending to an existing .modulerc and overwriting the existing one (from the eb command line, I don't think specifying this in the easyconfig makes much sense)?
  • Should we detect potentially conflicting lines in an existing .modulerc?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, that's a rather strict limitation for now, which we can review later when the need for it pops up?

A whole bunch of questions come into play to relax this:

  • What do we do with an existing .modulerc that was not created by EasyBuild?
  • How do we allow both appending to an existing .modulerc and overwriting the existing one (from the eb command line, I don't think specifying this in the easyconfig makes much sense)?
  • Should we detect potentially conflicting lines in an existing .modulerc?

Comment thread easybuild/easyblocks/generic/modulerc.py
Copy link
Copy Markdown
Member

@ocaisa ocaisa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, going in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants