Skip to content

make excluded tests for PyTorch 1.12.1 consistent + also exclude flaky test_optim rather than adding patch to try and fix it#17737

Merged
Micket merged 3 commits intoeasybuilders:developfrom
branfosj:revert
Apr 18, 2023
Merged

make excluded tests for PyTorch 1.12.1 consistent + also exclude flaky test_optim rather than adding patch to try and fix it#17737
Micket merged 3 commits intoeasybuilders:developfrom
branfosj:revert

Conversation

@branfosj
Copy link
Copy Markdown
Member

@branfosj branfosj commented Apr 15, 2023 •
edited
Loading

Based on the failures in #17732 and #17733 I think we should revert this. It looks to be a case of fixing one flaky test and making other tests less reliable. :(

We'll also add test_optim to excluded tests, which it has been for 1.9.0 to 1.12.0.

I've also made sure all 1.12.1 easyconfigs skip the tests metnioned in #17615.

@branfosj branfosj added this to the next release (4.7.2) milestone Apr 15, 2023
This was referenced Apr 15, 2023
Closed
Closed
Closed
Merged
Merged
Merged
@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 15, 2023 •
edited
Loading

cc @VRehnberg

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 16, 2023

@branfosj Should we just add the flaky test to expected_tests?

@VRehnberg
Copy link
Copy Markdown
Contributor

VRehnberg commented Apr 17, 2023

@branfosj Should we just add the flaky test to expected_tests?

Yes

@branfosj
Copy link
Copy Markdown
Member Author

branfosj commented Apr 17, 2023

I've disabled test_optim. On checking, I found this in 1.9.0, 1.10.0, 1.11.0, and 1.12.0:

        # Fails on A10s: https://github.com/pytorch/pytorch/issues/63079
        'test_optim',

@branfosj branfosj changed the title revert use predefined data in test option for PyTorch 1.12.1 revert adding patch and skip test_optim; and add excluded tests from #17615 consistently to PyTorch 1.12.1 Apr 17, 2023
@boegel boegel changed the title revert adding patch and skip test_optim; and add excluded tests from #17615 consistently to PyTorch 1.12.1 make excluded tests for PyTorch 1.12.1 consistent + also exclude flaky test_optim rather than adding patch to try and fix it Apr 17, 2023
@boegel boegel added the bug fix label Apr 17, 2023
@Micket
Copy link
Copy Markdown
Contributor

Micket commented Apr 17, 2023

Test report by @Micket
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
vera-c1 - Linux Rocky Linux 8.6, x86_64, Intel Xeon Processor (Skylake), Python 3.6.8
See https://gist.github.com/Micket/a6dd81044ccad52a5de41fb208ac4dab for a full test report.

@Micket
Copy link
Copy Markdown
Contributor

Micket commented Apr 17, 2023

Test report by @Micket
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
alvis10-02 - Linux Rocky Linux 8.6, x86_64, Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz, 4 x NVIDIA NVIDIA A40, 520.61.05, Python 3.6.8
See https://gist.github.com/Micket/c621516854450ca9824ed2fba6f38f26 for a full test report.

Micket
Micket approved these changes Apr 18, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@Micket Micket left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Micket
Copy link
Copy Markdown
Contributor

Micket commented Apr 18, 2023

I think spot testing one of each type is good enough here since we are just adding a few identical ignores.

@Micket
Copy link
Copy Markdown
Contributor

Micket commented Apr 18, 2023

Going in, thanks @branfosj!

@Micket Micket merged commit d76289b into easybuilders:develop Apr 18, 2023
@branfosj branfosj deleted the revert branch April 18, 2023 09:06
@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 19, 2023

Test report by @boegel
FAILED
Build succeeded for 1 out of 3 (3 easyconfigs in total)
node3159.skitty.os - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/boegel/24e103b9822f3fe68fc63f59f2479047 for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 19, 2023

Test report by @boegel
FAILED
Build succeeded for 1 out of 3 (3 easyconfigs in total)
node3903.accelgor.os - Linux RHEL 8.6, x86_64, AMD EPYC 7413 24-Core Processor (zen3), 1 x NVIDIA NVIDIA A100-SXM4-80GB, 530.30.02, Python 3.6.8
See https://gist.github.com/boegel/c9ed02ae23c967a4111bd25df96fe1b2 for a full test report.

@Flamefire Flamefire mentioned this pull request Jun 8, 2023
Merged
Flamefire added a commit to Flamefire/easybuild-easyconfigs that referenced this pull request Jun 8, 2023
@Flamefire
Remove unused PyTorch-1.12.1_use-predefined-data-in-test-optim.patch
4cedf28 
The usage of this patch was removed in easybuilders#17737 so this patch is no longer
required to be kept.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Micket Micket Micket approved these changes

Assignees

No one assigned

Labels

bug fix change

Projects

None yet

Milestone

4.7.2

Development

Successfully merging this pull request may close these issues.

4 participants

@branfosj @boegel @VRehnberg @Micket