Skip to content

use all available cores for running Perl test suite#2637

Merged
bartoldeman merged 2 commits intoeasybuilders:developfrom
boegel:20211215134546_new_pr_perl
Dec 18, 2021
Merged

use all available cores for running Perl test suite#2637
bartoldeman merged 2 commits intoeasybuilders:developfrom
boegel:20211215134546_new_pr_perl

Conversation

@boegel
Copy link
Copy Markdown
Member

@boegel boegel commented Dec 15, 2021 •
edited
Loading

(created using eb --new-pr)

Running the Perl test suite with make test is notoriously slow, but apparently there's support for running the tests in parallel for quite a while now, see https://perldoc.perl.org/perlhack#Parallel-tests

With make test for Perl-5.34.0-GCCcore-11.2.0.eb on an Intel Haswell system (so running tests sequentially):

All tests successful.
Elapsed: 1012 sec
u=6.98  s=3.14  cu=638.68  cs=47.13  scripts=2549  tests=1198571

Using 10 cores (TEST_JOBS=10 PERL_TEST_HARNESS_ASAP=1 make -j 10 test_harness):

All tests successful.
Files=2677, Tests=1198650, 286 wallclock secs (69.32 usr  6.57 sys + 634.33 cusr 50.98 csys = 761.20 CPU)
Result: PASS

So roughly 4x faster.

@boegel boegel added this to the release after 4.5.1 milestone Dec 15, 2021
@boegel
Copy link
Copy Markdown
Member Author

boegel commented Dec 15, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS Perl-5.30.0-GCCcore-8.3.0.eb
  • SUCCESS Perl-5.30.2-GCCcore-9.3.0-minimal.eb
  • SUCCESS Perl-5.30.2-GCCcore-9.3.0.eb
  • SUCCESS Perl-5.32.0-GCCcore-10.2.0-minimal.eb
  • SUCCESS Perl-5.32.0-GCCcore-10.2.0.eb
  • SUCCESS Perl-5.32.1-GCCcore-10.3.0-minimal.eb
  • SUCCESS Perl-5.32.1-GCCcore-10.3.0.eb
  • SUCCESS Perl-5.34.0-GCCcore-11.2.0.eb

Build succeeded for 8 out of 8 (8 easyconfigs in total)
node2618.swalot.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (haswell), Python 3.6.8
See https://gist.github.com/068f7faa11c70071ae68b359ff7927a4 for a full test report.

@boegel boegel requested a review from bartoldeman December 15, 2021 17:26
bartoldeman
bartoldeman previously approved these changes Dec 18, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@bartoldeman bartoldeman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

bartoldeman
bartoldeman approved these changes Dec 18, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@bartoldeman bartoldeman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (fixed small confict)

@bartoldeman bartoldeman merged commit 9418f17 into easybuilders:develop Dec 18, 2021
@boegel boegel deleted the 20211215134546_new_pr_perl branch December 18, 2021 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bartoldeman bartoldeman bartoldeman approved these changes

Assignees

No one assigned

Labels

enhancement performance

Projects

None yet

Milestone

4.5.2

Development

Successfully merging this pull request may close these issues.

2 participants

@boegel @bartoldeman