Skip to content

update easyblocks for toy extensions to make sure that asynchronous installation command is run in correct working directory#4471

Merged
branfosj merged 2 commits intoeasybuilders:5.0.xfrom
boegel:run_async_test_fix_workdir
Mar 2, 2024
Merged

update easyblocks for toy extensions to make sure that asynchronous installation command is run in correct working directory#4471
branfosj merged 2 commits intoeasybuilders:5.0.xfrom
boegel:run_async_test_fix_workdir

Conversation

@boegel
Copy link
Copy Markdown
Member

@boegel boegel commented Mar 1, 2024 •
edited
Loading

Fixes flaky test failures in test_toy_exts_parallel, which are blocking #4469 (and also #4453), because the installation command for the extension is being run from the wrong working directory.

Apparently all threads are affected by an os.chdir call, which is global state for the whole process, so we can't rely on it.
It seems like for newer Python versions this is less of a problem, I've only seen failing tests with Python 3.6-3.8.

Details 
======================================================================
ERROR: test_toy_exts_parallel (test.framework.toy_build.ToyBuildTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/test/framework/toy_build.py", line 1920, in test_toy_exts_parallel
    stdout, stderr = self.run_test_toy_build_with_output(ec_file=test_ec, extra_args=args, raise_error=True)
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/test/framework/toy_build.py", line 239, in run_test_toy_build_with_output
    self._test_toy_build(*args, **kwargs)
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/test/framework/toy_build.py", line 195, in _test_toy_build
    raise myerr
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/test/framework/toy_build.py", line 191, in _test_toy_build
    raise_error=raise_error, testing=testing, raise_systemexit=raise_systemexit)
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/test/framework/utilities.py", line 341, in eb_main
    raise myerr
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/test/framework/utilities.py", line 314, in eb_main
    main(args=main_args, logfile=logfile, do_build=do_build, testing=testing, modtool=modtool)
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/easybuild/main.py", line 728, in main
    hooks, do_build)
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/easybuild/main.py", line 565, in process_eb_args
    exit_on_failure=exit_on_failure)
  File "/tmp/runner/cf2839720f6465e4fb88190e2f581e088d205f7d/lib/python3.6/site-packages/easybuild/main.py", line 175, in build_and_install_software
    raise EasyBuildError(test_msg)
easybuild.tools.build_log.EasyBuildError: 'Installation of test.eb failed: "shell command \'gcc ...\' failed in extensions step for test.eb"'

edit: https://stackoverflow.com/questions/13438910/if-i-chdir-within-a-thread-will-that-affect-the-cwd-of-the-parent-program/73867506#73867506 is a good source of information on this (hat tip @zao)

@boegel
update easyblocks for toy extensions to make sure that asynchronous i…
ff4ff49 
…nstallation command is run in correct working directory
@boegel boegel added this to the 5.0 milestone Mar 1, 2024
@boegel boegel mentioned this pull request Mar 1, 2024
Merged
@boegel boegel mentioned this pull request Mar 2, 2024
Merged
@branfosj branfosj merged commit 128505f into easybuilders:5.0.x Mar 2, 2024
@boegel boegel deleted the run_async_test_fix_workdir branch March 2, 2024 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ocaisa ocaisa ocaisa approved these changes

Assignees

No one assigned

Labels

bug fix EasyBuild-5.0 EasyBuild 5.0 tests

Projects

None yet

Milestone

5.0.0

Development

Successfully merging this pull request may close these issues.

3 participants

@boegel @ocaisa @branfosj