Skip to content

add CI workflow to run unit tests with Python 2 (again)#4333

Merged
branfosj merged 13 commits intoeasybuilders:developfrom
boegel:tests_python2
Sep 14, 2023
Merged

add CI workflow to run unit tests with Python 2 (again)#4333
branfosj merged 13 commits intoeasybuilders:developfrom
boegel:tests_python2

Conversation

@boegel
Copy link
Copy Markdown
Member

@boegel boegel commented Aug 16, 2023

We stopped testing with Python 2 in #4267 because official support for using Python 2 was dropped in both GitHub Actions and actions/setup-python.

That has led to silently introducing bugs that make EasyBuild crash hard when running on top of Python 2 however, see for example #4328.

While support for running EasyBuild on top of Python 2 is deprecated (since EasyBuild 4.7.0 - Jan'23, cfr. #4136), we should do what we can to maintain compatibility with Python 2 until EasyBuild 5.0 is released.

boegel added 9 commits August 16, 2023 09:36
@boegel
run unit tests with python2 via CentOS 7.9 container
8eeae32
@boegel
strip out GC3Pie from requirements.txt before installing Python packa…
cf81cec 
…ges when testing with Python 2
@boegel
run command via easybuild user and login shell when running test suit…
c335cc9 
…e with python2
@boegel
prepend updates to $PATH and $PYTHONPATH in commands being run via `s…
c941493 
…udo -u easybuild bash -l -c ...`
@boegel
tweak eval_quoted_string helper function in test_quote_py_str to be c…
a042318 
…ompatible with both (old versions of) Python 2.7 & 3.x
@boegel
silence deprecation warning for using Python 2 in workflow to run tes…
b93eb7d 
…ts with Python 2.7 in CentOS 7.9 container
@boegel
fix code style issues in helper function in test_quote_py_str
1e8dee8
@boegel
configure Git for easybuild user in workflow for running test suite w…
905d919 
…ith Python 2
@boegel
fix :avail pattern in test_debug_lmod
6c026b3
@boegel boegel added this to the next release (4.8.1?) milestone Aug 16, 2023
@boegel boegel mentioned this pull request Aug 16, 2023
Merged
@boegel
Copy link
Copy Markdown
Member Author

boegel commented Aug 17, 2023

Down to 4 failing tests, probably due to side-effects caused by running the test suite via sudo bash -l -c?

  • test_regtest in test/framework/toy_build:
Details 
2023-08-17T08:42:56.8061932Z ERROR: test_regtest (test.framework.toy_build.ToyBuildTest)
2023-08-17T08:42:56.8062530Z Test use of --regtest.
2023-08-17T08:42:56.8063056Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8063389Z Traceback (most recent call last):
2023-08-17T08:42:56.8063959Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 2112, in test_regtest
2023-08-17T08:42:56.8064586Z     self.test_toy_build(extra_args=['--regtest', '--sequential'], verify=False)
2023-08-17T08:42:56.8065101Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 202, in test_toy_build
2023-08-17T08:42:56.8065582Z     self.assertEqual(read_file(os.path.join(toy_install_path, 'README')), "TOY\n")
2023-08-17T08:42:56.8066105Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/easybuild/tools/filetools.py", line 219, in read_file
2023-08-17T08:42:56.8066465Z     raise EasyBuildError("Failed to read %s: %s", path, err)
2023-08-17T08:42:56.8067350Z EasyBuildError: "Failed to read /tmp/eb-TDA5dZ/eb-Qr5M24/eb-6TItku/tmp6YmlCM/software/toy/0.0/README: [Errno 2] No such file or directory: '/tmp/eb-TDA5dZ/eb-Qr5M24/eb-6TItku/tmp6YmlCM/software/toy/0.0/README'"
  • test_toy_lock_cleanup_signals in test/framework/toy_build
Details 
2023-08-17T08:42:56.8068065Z ERROR: test_toy_lock_cleanup_signals (test.framework.toy_build.ToyBuildTest)
2023-08-17T08:42:56.8068404Z Test cleanup of locks after EasyBuild session gets a cancellation signal.
2023-08-17T08:42:56.8068784Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8069206Z Traceback (most recent call last):
2023-08-17T08:42:56.8069714Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 3648, in test_toy_lock_cleanup_signals
2023-08-17T08:42:56.8070081Z     raise_error=True, testing=False, raise_systemexit=True)
2023-08-17T08:42:56.8070586Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/easybuild/base/testing.py", line 170, in assertErrorRegex
2023-08-17T08:42:56.8070908Z     call(*args, **kwargs)
2023-08-17T08:42:56.8071365Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 194, in test_toy_build
2023-08-17T08:42:56.8071651Z     raise myerr
2023-08-17T08:42:56.8072254Z EasyBuildError: "Undefined build option: 'silent'. Make sure you have set up the EasyBuild configuration using set_up_configuration() (from easybuild.tools.options) in case you're not using EasyBuild via the 'eb' CLI."
  • test_download_instructions in test/framework/easyblock:
Details 
2023-08-17T08:42:56.8073001Z FAIL: test_download_instructions (test.framework.easyblock.EasyBlockTest)
2023-08-17T08:42:56.8073320Z Test use of download_instructions easyconfig parameter.
2023-08-17T08:42:56.8073693Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8073959Z Traceback (most recent call last):
2023-08-17T08:42:56.8074452Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/easyblock.py", line 1532, in test_download_instructions
2023-08-17T08:42:56.8074861Z     self.assertErrorRegex(EasyBuildError, error_pattern, eb.fetch_step)
2023-08-17T08:42:56.8075396Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/easybuild/base/testing.py", line 178, in assertErrorRegex
2023-08-17T08:42:56.8076270Z     self.assertTrue(regex.search(msg), "Pattern '%s' is found in '%s'" % (regex.pattern, msg))
2023-08-17T08:42:56.8077886Z AssertionError: None is not true : Pattern '^Couldn't find file software_with_missing_sources-0.0.tar.gz anywhere, and downloading it didn't work either' is found in 'Failed to create directory /tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/sandbox/sources/s/software_with_missing_sources: [Errno 13] Permission denied: '/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/sandbox/sources/s''
  • test_reproducibility in test/framework/toy_build:
Details 
2023-08-17T08:42:56.8079302Z FAIL: test_reproducibility (test.framework.toy_build.ToyBuildTest)
2023-08-17T08:42:56.8090104Z Test toy build produces expected reproducibility files
2023-08-17T08:42:56.8090574Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8090844Z Traceback (most recent call last):
2023-08-17T08:42:56.8091349Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 2145, in test_reproducibility
2023-08-17T08:42:56.8091895Z     self.test_toy_build(extra_args=['--minimal-toolchains', '--easyblock=EB_toytoy', '--hooks=%s' % hooks_file])
2023-08-17T08:42:56.8092459Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 197, in test_toy_build
2023-08-17T08:42:56.8093060Z     self.check_toy(self.test_installpath, outtxt, name=name, versionsuffix=versionsuffix, error=myerr)
2023-08-17T08:42:56.8093615Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 125, in check_toy
2023-08-17T08:42:56.8094114Z     self.assertTrue(success.search(outtxt), "COMPLETED message found in '%s'%s" % (outtxt, error_msg))
2023-08-17T08:42:56.8094650Z AssertionError: None is not true : COMPLETED message found in '
...
2023-08-17T08:42:56.9565365Z == 2023-08-17 08:17:51,527 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:126 in __init__): Build of /tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/easyconfigs/test_ecs/t/toy/toy-0.0.eb failed (err: "build failed (first 300 chars): Can't determine patch level for patch /tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/sandbox/sources/toy/toy-0.0_fix-silly-typo-in-printf-statement.patch from directory /tmp/eb-TDA5dZ/eb-Qr5M24/eb-xoEv4h/tmpjWHDt8/toy/0.0/system-system") (at easybuild/main.py:175 in build_and_install_software)
2023-08-17T08:42:56.9565450Z '

@boegel boegel mentioned this pull request Aug 30, 2023
Merged
@easybuilders easybuilders deleted a comment from boegelbot Sep 13, 2023
@boegel boegel marked this pull request as ready for review September 13, 2023 10:37
@boegel
Copy link
Copy Markdown
Member Author

boegel commented Sep 13, 2023 •
edited
Loading

As suggested by @branfosj, I've skipped the 4 tests that are broken when running on top of Python 2 for now in c3fc220, so we can get this merged rather than spending more time to figure out why the tests are only broken with Python 2.

Very likely the problem with those tests is specific to using Python 2 in a niche feature of EasyBuild (like --regtest), which we don't care much about since support for using Python 2 has been deprecated since EasyBuild v4.7.0 (Jan'23) and the upcoming EasyBuild 5.x (ETA: by end of 2023, hopefully) will no longer support Python 2...

@easybuilders easybuilders deleted a comment from boegelbot Sep 13, 2023
@boegel boegel requested a review from branfosj September 13, 2023 22:24
@branfosj branfosj merged commit bbf0c71 into easybuilders:develop Sep 14, 2023
@boegel boegel deleted the tests_python2 branch September 14, 2023 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@branfosj branfosj branfosj approved these changes

Assignees

No one assigned

Labels

bug fix tests

Projects

None yet

Milestone

4.8.2

Development

Successfully merging this pull request may close these issues.

2 participants

@boegel @branfosj