Skip to content

disable assembly tests for googlebenchmark#24423

Merged
Crivella merged 1 commit intoeasybuilders:developfrom
Thyre:20251031091454_new_pr_googlebenchmark191
Oct 31, 2025
Merged

disable assembly tests for googlebenchmark#24423
Crivella merged 1 commit intoeasybuilders:developfrom
Thyre:20251031091454_new_pr_googlebenchmark191

Conversation

@Thyre
Copy link
Copy Markdown
Collaborator

@Thyre Thyre commented Oct 31, 2025 •
edited
Loading

(created using eb --new-pr)

Avoids error messages like

Click to open 
83/85 Test #83: run_donotoptimize_assembly_test_CHECK ......***Failed    0.01 sec
donotoptimize_assembly_test.cc:50:12: error: CHECK: expected string not found in input
 // CHECK: movl %eax, -{{[0-9]+}}(%[[REG:[a-z]+]]
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:6:22: note: scanning from here
 movq ExternInt(%rip), %rax
                     ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:9:3: note: possible intended match here
 vmovd %xmm0, -8(%rsp)
  ^
donotoptimize_assembly_test.cc:77:12: error: CHECK: expected string not found in input
 // CHECK: movl %eax, -{{[0-9]+}}(%[[REG:[a-z]+]])
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:24:22: note: scanning from here
 movq ExternInt(%rip), %rax
                     ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:27:3: note: possible intended match here
 vmovd %xmm0, -8(%rsp)
  ^
donotoptimize_assembly_test.cc:87:12: error: CHECK: expected string not found in input
 // CHECK: movl $42, ExtraLargeObj+64(%rip)
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:31:38: note: scanning from here
test_with_extra_large_lvalue_with_op:
                                     ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:32:2: note: possible intended match here
 movq ExtraLargeObj(%rip), %rax
 ^
donotoptimize_assembly_test.cc:95:12: error: CHECK: expected string not found in input
 // CHECK: movl $42, BigArray+64(%rip)
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:36:28: note: scanning from here
test_with_big_array_with_op:
                           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:38:2: note: possible intended match here
 movl $42, 64(%rax)
 ^
donotoptimize_assembly_test.cc:121:12: error: CHECK: expected string not found in input
 // CHECK: movl %eax, -{{[0-9]+}}(%[[REG:[a-z]+]])
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:52:22: note: scanning from here
 movq ExternInt(%rip), %rax
                     ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:55:3: note: possible intended match here
 vmovd %xmm0, -8(%rsp)
  ^
donotoptimize_assembly_test.cc:174:12: error: CHECK: expected string not found in input
 // CHECK: leaq [[DEST]], %rax
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:88:20: note: scanning from here
 movl $42, -4(%rsp)
                   ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:88:20: note: with "DEST" equal to "-4(%rsp)"
 movl $42, -4(%rsp)
                   ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:91:20: note: possible intended match here
test_pointer_const_lvalue:
                   ^
donotoptimize_assembly_test.cc:184:12: error: CHECK: expected string not found in input
 // CHECK: leaq [[DEST]], %rax
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:93:20: note: scanning from here
 movl $42, -4(%rsp)
                   ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:93:20: note: with "DEST" equal to "-4(%rsp)"
 movl $42, -4(%rsp)
                   ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:96:14: note: possible intended match here
test_pointer_lvalue:
             ^
donotoptimize_assembly_test.cc:195:12: error: CHECK: expected string not found in input
 // CHECK: leaq [[DEST]], %rax
           ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:98:20: note: scanning from here
 movl $42, -4(%rsp)
                   ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:98:20: note: with "DEST" equal to "-4(%rsp)"
 movl $42, -4(%rsp)
                   ^
/data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s:102:42: note: possible intended match here
 jmp _ZN9benchmark8internal17InitializeStreamsEv@PLT
                                         ^

Input file: /data/EasyBuild-develop/build/googlebenchmark/1.9.4/GCCcore-14.2.0/easybuild_obj/test/donotoptimize_assembly_test.s
Check file: donotoptimize_assembly_test.cc

-dump-input=help explains the following input dump.

Input was:
<<<<<<
             1: test_with_rvalue: 
             2:  movl $42, %eax 
             3:  ret 
             4:  
             5: test_with_large_rvalue: 
             6:  movq ExternInt(%rip), %rax 
check:50'0                           X~~~~~~ error: no match found
             7:  vmovd (%rax), %xmm0 
check:50'0      ~~~~~~~~~~~~~~~~~~~~~
             8:  vpshufd $0xe0, %xmm0, %xmm1 
check:50'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
             9:  vmovd %xmm0, -8(%rsp) 
check:50'0      ~~~~~~~~~~~~~~~~~~~~~~~
check:50'1        ?                     possible intended match
            10:  vmovq %xmm1, -16(%rsp) 
check:50'0      ~~~~~~~~~~~~~~~~~~~~~~~~
            11:  ret 
check:50'0      ~~~~~
            12:  
check:50'0      ~
            13: test_with_non_trivial_rvalue: 
check:50'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            14:  movq ExternInt(%rip), %rax 
             .
             .
             .
            19: test_with_lvalue: 
            20:  movl $101, %eax 
            21:  ret 
            22:  
            23: test_with_large_lvalue: 
            24:  movq ExternInt(%rip), %rax 
check:77'0                           X~~~~~~ error: no match found
            25:  vmovd (%rax), %xmm0 
check:77'0      ~~~~~~~~~~~~~~~~~~~~~
            26:  vpshufd $0xe0, %xmm0, %xmm1 
check:77'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            27:  vmovd %xmm0, -8(%rsp) 
check:77'0      ~~~~~~~~~~~~~~~~~~~~~~~
check:77'1        ?                     possible intended match
            28:  vmovq %xmm1, -16(%rsp) 
check:77'0      ~~~~~~~~~~~~~~~~~~~~~~~~
            29:  ret 
check:77'0      ~~~~~
            30:  
check:77'0      ~
            31: test_with_extra_large_lvalue_with_op: 
check:77'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:87'0                                           X error: no match found
            32:  movq ExtraLargeObj(%rip), %rax 
check:87'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:87'1       ?                               possible intended match
            33:  movl $42, 64(%rax) 
check:87'0      ~~~~~~~~~~~~~~~~~~~~
            34:  ret 
check:87'0      ~~~~~
            35:  
check:87'0      ~
            36: test_with_big_array_with_op: 
check:87'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:95'0                                 X~ error: no match found
            37:  movq BigArray(%rip), %rax 
check:95'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~
            38:  movl $42, 64(%rax) 
check:95'0      ~~~~~~~~~~~~~~~~~~~~
check:95'1       ?                   possible intended match
            39:  ret 
check:95'0      ~~~~~
            40:  
check:95'0      ~
            41: test_with_non_trivial_lvalue: 
check:95'0      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            42:  movq ExternInt(%rip), %rax 
            43:  movl (%rax), %eax 
            44:  movl %eax, -4(%rsp) 
            45:  ret 
            46:  
            47: test_with_const_lvalue: 
            48:  movl $123, %eax 
            49:  ret 
            50:  
            51: test_with_large_const_lvalue: 
            52:  movq ExternInt(%rip), %rax 
check:121'0                          X~~~~~~ error: no match found
            53:  vmovd (%rax), %xmm0 
check:121'0     ~~~~~~~~~~~~~~~~~~~~~
            54:  vpshufd $0xe0, %xmm0, %xmm1 
check:121'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            55:  vmovd %xmm0, -8(%rsp) 
check:121'0     ~~~~~~~~~~~~~~~~~~~~~~~
check:121'1       ?                     possible intended match
            56:  vmovq %xmm1, -16(%rsp) 
check:121'0     ~~~~~~~~~~~~~~~~~~~~~~~~
            57:  ret 
check:121'0     ~~~~~
            58:  
check:121'0     ~
            59: test_with_const_extra_large_obj: 
check:121'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            60:  ret 
             .
             .
             .
            83:  incl %eax 
            84:  ret 
            85:  
            86: test_pointer_rvalue: 
            87:  leaq -4(%rsp), %rax 
            88:  movl $42, -4(%rsp) 
check:174'0                        X error: no match found
check:174'1                          with "DEST" equal to "-4(%rsp)"
            89:  ret 
check:174'0     ~~~~~
            90:  
check:174'0     ~
            91: test_pointer_const_lvalue: 
check:174'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~
check:174'2                        ?        possible intended match
            92:  leaq -4(%rsp), %rax 
            93:  movl $42, -4(%rsp) 
check:184'0                        X error: no match found
check:184'1                          with "DEST" equal to "-4(%rsp)"
            94:  ret 
check:184'0     ~~~~~
            95:  
check:184'0     ~
            96: test_pointer_lvalue: 
check:184'0     ~~~~~~~~~~~~~~~~~~~~
check:184'2                  ?        possible intended match
            97:  leaq -4(%rsp), %rax 
            98:  movl $42, -4(%rsp) 
check:195'0                        X error: no match found
check:195'1                          with "DEST" equal to "-4(%rsp)"
            99:  ret 
check:195'0     ~~~~~
           100:  
check:195'0     ~
           101: GLOBAL__sub_I_donotoptimize_assembly_test.cc: 
check:195'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           102:  jmp _ZN9benchmark8internal17InitializeStreamsEv@PLT 
check:195'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:195'2                                              ?            possible intended match
           103:  
check:195'0     ~
           104: _ZL18ConstExtraLargeObj: 
check:195'0     ~~~~~~~~~~~~~~~~~~~~~~~~~
           105:  
check:195'0     ~
           106: _ZL13ConstBigArray: 
check:195'0     ~~~~~~~~~~~~~~~~~~~~
>>>>>>

in the test suite. This change was already made for 2025b. So this PR just backports this to older toolchains as well.
As mentioned in google/benchmark#1326 (comment), these tests are highly OS and compiler dependent, therefore can easily fail if the configuration is just a little bit off.

Generally, the tests are only enabled when FileCheck is found. Since we do not provide that as a dependency, it is pulled in from the system. So we'd better disable this generally to be consistent...

@Thyre Thyre added bug fix 2023b 2024a issues & PRs related to 2024a common toolchains 2025a issues & PRs related to 2025a common toolchains labels Oct 31, 2025
@Thyre Thyre added this to the next release (5.2.0?) milestone Oct 31, 2025
@Thyre Thyre removed the change label Oct 31, 2025
@Thyre
Copy link
Copy Markdown
Collaborator Author

Thyre commented Oct 31, 2025

Test report by @Thyre
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
Linux - Linux Arch Linux UNKNOWN, x86_64, AMD Ryzen 7 7800X3D 8-Core Processor (zen4), 1 x AMD Navi 48 [Radeon RX 9070/9070 XT/9070 GRE] (device id: 0x7550, gfx: gfx1201, driver: Linuxversion6.17.5-arch1-1(linux@archlinux)(gcc(GCC)15.2.120250813,GNUld(GNUBinutils)2.45.0)#1SMPPREEMPT_DYNAMICThu,23Oct202518:49:03+0000), 1 x AMD Raphael (device id: 0x164e, gfx: gfx1036, driver: Linuxversion6.17.5-arch1-1(linux@archlinux)(gcc(GCC)15.2.120250813,GNUld(GNUBinutils)2.45.0)#1SMPPREEMPT_DYNAMICThu,23Oct202518:49:03+0000), Python 3.13.7
See https://gist.github.com/Thyre/f5e35eb2211050a6a7babc2889d0ae27 for a full test report.

@Thyre
Copy link
Copy Markdown
Collaborator Author

Thyre commented Oct 31, 2025

@boegelbot please test @ jsc-zen3
EB_ARGS="--installpath=/tmp/$USER/ecpr-24423"

@Thyre Thyre mentioned this pull request Oct 31, 2025
Merged
@boegelbot
Copy link
Copy Markdown
Collaborator

boegelbot commented Oct 31, 2025

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=24423 EB_ARGS="--installpath=/tmp/$USER/ecpr-24423" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_24423 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 8604

Test results coming soon (I hope)...

Details

- notification for comment with ID 3471797898 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Copy Markdown
Collaborator

boegelbot commented Oct 31, 2025

Test report by @boegelbot
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/f27dbc9c1df05a4af3d6696ee27d92c1 for a full test report.

@Crivella
Copy link
Copy Markdown
Contributor

Crivella commented Oct 31, 2025

Test report by @Crivella
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.13
See https://gist.github.com/Crivella/93b5e32fba92b0f59a7dfe8838e41a81 for a full test report.

Crivella
Crivella approved these changes Oct 31, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@Crivella Crivella left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Crivella
Copy link
Copy Markdown
Contributor

Crivella commented Oct 31, 2025

Going in, thanks @Thyre!

@Crivella Crivella merged commit b181e65 into easybuilders:develop Oct 31, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Crivella Crivella Crivella approved these changes

Assignees

No one assigned

Labels

2023b 2024a issues & PRs related to 2024a common toolchains 2025a issues & PRs related to 2025a common toolchains bug fix

Projects

None yet

Milestone

5.2.0

Development

Successfully merging this pull request may close these issues.

3 participants

@Thyre @boegelbot @Crivella