Skip to content

update error detection for PyTorch tests#3033

Merged
boegel merged 1 commit intoeasybuilders:developfrom
Flamefire:20231115135716_new_pr_pytorch
Nov 26, 2023
Merged

update error detection for PyTorch tests#3033
boegel merged 1 commit intoeasybuilders:developfrom
Flamefire:20231115135716_new_pr_pytorch

Conversation

@Flamefire
Copy link
Copy Markdown
Contributor

@Flamefire Flamefire commented Nov 15, 2023 •
edited
Loading

(created using eb --new-pr)

Fixes multiple issues:

  • Detection of individual tests fails on recent pytest(?) which outputs a time after "FAILED"
  • Misdetection of a test summary printed above the final list of failed tests as a failure, leading to e.g. dynamo/test_dynamic_shapes 1/1 (5452 passed, 135 skipped)
  • Counting tests doesn't take PyTest output into account leading to 7 test failures, 0 test errors (out of 0)

@boegel boegel added the bug fix label Nov 22, 2023
@boegel boegel added this to the next release (4.9.0?) milestone Nov 22, 2023
@boegel
Copy link
Copy Markdown
Member

boegel commented Nov 22, 2023 •
edited
Loading

@boegelbot please test @ jsc-zen2
CORE_CNT=16
EB_ARGS="PyTorch-1.10.0-foss-2021a.eb PyTorch-2.0.1-foss-2022a.eb"

@boegelbot
Copy link
Copy Markdown

boegelbot commented Nov 22, 2023

@boegel: Request for testing this PR well received on jsczen2l1.int.jsc-zen2.easybuild-test.cluster

PR test command 'EB_PR=3033 EB_ARGS="PyTorch-1.10.0-foss-2021a.eb PyTorch-2.0.1-foss-2022a.eb" EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --mem-per-cpu=4000M --job-name test_PR_3033 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen2.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 3779

Test results coming soon (I hope)...

Details

- notification for comment with ID 1822950727 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegel
Copy link
Copy Markdown
Member

boegel commented Nov 22, 2023

@boegelbot please test @ generoso
CORE_CNT=16
EB_ARGS="PyTorch-1.13.1-foss-2022b.eb"

@boegelbot
Copy link
Copy Markdown

boegelbot commented Nov 22, 2023

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=3033 EB_ARGS="PyTorch-1.13.1-foss-2022b.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --job-name test_PR_3033 --ntasks="16" ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 12226

Test results coming soon (I hope)...

Details

- notification for comment with ID 1822963892 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Copy Markdown

boegelbot commented Nov 22, 2023

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS PyTorch-1.13.1-foss-2022b.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
cnx2 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/dfb6044ff9524a7cb4f245b69d7bc568 for a full test report.

@boegelbot
Copy link
Copy Markdown

boegelbot commented Nov 22, 2023

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS PyTorch-1.10.0-foss-2021a.eb
  • SUCCESS PyTorch-2.0.1-foss-2022a.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen2c1.int.jsc-zen2.easybuild-test.cluster - Linux Rocky Linux 8.5, x86_64, AMD EPYC 7742 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/boegelbot/8a8e75fa3d0cbc6be71463711c4ceea1 for a full test report.

boegel
boegel approved these changes Nov 26, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel merged commit eda2199 into easybuilders:develop Nov 26, 2023
@Flamefire Flamefire deleted the 20231115135716_new_pr_pytorch branch November 27, 2023 10:36
@Flamefire Flamefire mentioned this pull request Dec 19, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

bug fix

Projects

None yet

Milestone

4.9.0

Development

Successfully merging this pull request may close these issues.

3 participants

@Flamefire @boegel @boegelbot