Skip to content

control number of parallel processes used by Ninja and add fix for aarch64 in easyconfig for Qt6 6.7.2#22588

Merged
boegel merged 1 commit intoeasybuilders:developfrom
SebastianAchilles:20250317112302_new_pr_Qt6672
May 21, 2025
Merged

control number of parallel processes used by Ninja and add fix for aarch64 in easyconfig for Qt6 6.7.2#22588
boegel merged 1 commit intoeasybuilders:developfrom
SebastianAchilles:20250317112302_new_pr_Qt6672

Conversation

@SebastianAchilles
Copy link
Copy Markdown
Member

@SebastianAchilles SebastianAchilles commented Mar 17, 2025

(created using eb --new-pr)

@SebastianAchilles SebastianAchilles added the aarch64 Related to Arm 64-bit (aarch64) label Mar 17, 2025
@SebastianAchilles
Copy link
Copy Markdown
Member Author

SebastianAchilles commented Mar 18, 2025

@boegelbot please test @ jsc-zen3

@boegelbot
Copy link
Copy Markdown
Collaborator

boegelbot commented Mar 18, 2025

@SebastianAchilles: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=22588 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_22588 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5997

Test results coming soon (I hope)...

Details

- notification for comment with ID 2734432109 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Copy Markdown
Collaborator

boegelbot commented Mar 18, 2025

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/dc0cafc9ebdbce256d80631b938f1d88 for a full test report.

@Thyre
Copy link
Copy Markdown
Collaborator

Thyre commented Mar 19, 2025 •
edited
Loading

It looks like some parts of the build process respect the set parallelism via NINJAFLAGS, some don't. I still see ~16-20 processes being spawned to compile or generate stuff sometimes (e.g. with moc). However, it is still an improvement compared to not setting NINJAFLAGS at all.

image

Let's see if my build passes.

@Thyre
Copy link
Copy Markdown
Collaborator

Thyre commented Mar 19, 2025 •
edited
Loading

Test report by @Thyre
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
ZAM054 - Linux Zorin OS 17, x86_64, 12th Gen Intel(R) Core(TM) i7-1260P, 1 x NVIDIA NVIDIA GeForce MX550, 565.57.01, Python 3.10.12
See https://gist.github.com/Thyre/c9d5150135e8b1a14db584471f0ea727 for a full test report.

The build failed with:

FAILED: qtconnectivity/src/nfc/CMakeFiles/Nfc.dir/cmake_pch.hxx.gch 
/tmp/eb-7nzxt5jc/tmpzzt7m21k/rpath_wrappers/gxx_wrapper/g++ -DNfc_EXPORTS -DQT_ASCII_CAST_WARNINGS -DQT_BUILDING_QT -DQT_BUILD_NFC_LIB -DQT_CORE_LIB -DQT_DEPRECATED_WARNINGS -DQT_DISABLE_DEPRECATED_UP_TO=0x050000 -DQT_EXPLICIT_QFILE_CONSTRUCTION_FROM_PATH -DQT_MOC_COMPAT -DQT_NO_AS_CONST -DQT_NO_AS_CONST=1 -DQT_NO_CAST_TO_ASCII -DQT_NO_CONTEXTLESS_CONNECT -DQT_NO_DEBUG -DQT_NO_EXCEPTIONS -DQT_NO_FOREACH -DQT_NO_FOREACH=1 -DQT_NO_JAVA_STYLE_ITERATORS -DQT_NO_NARROWING_CONVERSIONS_IN_CONNECT -DQT_NO_QEXCHANGE -DQT_USE_QSTRINGBUILDER -DQT_WARN_DEPRECATED_UP_TO=0x070000 -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtconnectivity/src/nfc/Nfc_autogen/include -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtbase/include -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtbase/include/QtNfc -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/qt-everywhere-src-6.7.2/qtconnectivity/src/nfc -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtconnectivity/src/nfc -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtbase/include/QtNfc/6.7.2 -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtbase/include/QtNfc/6.7.2/QtNfc -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtbase/include/QtCore -I/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/qt-everywhere-src-6.7.2/qtbase/mkspecs/linux-g++ -O2 -fno-tree-vectorize -march=native -fno-math-errno -DNDEBUG -O2 -std=c++17 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -Wall -Wextra -fno-exceptions -fPIC -Wsuggest-override -Winvalid-pch -x c++-header -include /opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtconnectivity/src/nfc/CMakeFiles/Nfc.dir/cmake_pch.hxx -MD -MT qtconnectivity/src/nfc/CMakeFiles/Nfc.dir/cmake_pch.hxx.gch -MF qtconnectivity/src/nfc/CMakeFiles/Nfc.dir/cmake_pch.hxx.gch.d -o qtconnectivity/src/nfc/CMakeFiles/Nfc.dir/cmake_pch.hxx.gch -c /opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtconnectivity/src/nfc/CMakeFiles/Nfc.dir/cmake_pch.hxx.cxx
In file included from /opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtbase/include/QtCore/qxpfunctional.h:1,
                 from /opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtbase/include/QtCore/QtCore:316,
                 from /opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/easybuild_obj/qtconnectivity/src/nfc/CMakeFiles/Nfc.dir/cmake_pch.hxx:5,
                 from <command-line>:
/opt/EasyBuild/apps/build/Qt6/6.7.2/GCCcore-13.3.0/qt-everywhere-src-6.7.2/qtbase/src/corelib/global/qxpfunctional.h:191:1: internal compiler error: in relocate_ptrs, at ggc-common.cc:382
  191 | } // namespace qxp
      | ^
0x72db6d relocate_ptrs
        ../../gcc/ggc-common.cc:382
0xbf7cab gt_pch_p_11symtab_node(void*, void*, void (*)(void*, void*, void*), void*)
        /opt/EasyBuild/apps/build/GCCcore/13.3.0/system-system/gcc-13.3.0/stage3_obj/gcc/gtype-desc.cc:9670
0xb8edf1 gt_pch_save(_IO_FILE*)
        ../../gcc/ggc-common.cc:641
0x9c7465 c_common_write_pch()
        ../../gcc/c-family/c-pch.cc:175
0x810a25 c_parse_final_cleanups()
        ../../gcc/cp/decl2.cc:4949
0x9c70a0 c_common_parse_file()
        ../../gcc/c-family/c-opts.cc:1266
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.

This doesn't look like an issue introduced with this PR...
Seems like I triggered an internal assertion in GCC: https://github.com/gcc-mirror/gcc/blob/2123635378d32d884c8ebfa5d01c8006b24ff54f/gcc/ggc-common.cc#L382C1-L382C23

@Thyre
Copy link
Copy Markdown
Collaborator

Thyre commented Mar 19, 2025

For some reason, entering the environment generated by EB, and running the same ninja command worked fine. Don't know why I ran into this issue...

@Crivella
Copy link
Copy Markdown
Contributor

Crivella commented May 19, 2025 •
edited
Loading

Test report by @Crivella
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.12
See https://gist.github.com/Crivella/aeb82138f33241d12bda1bbb112517ae for a full test report.

I would say this is working concerning the honoring of the max-parallel now

Without the PR changes

Screenshot from 2025-05-19 16-25-34

Including the PR changes

Screenshot from 2025-05-19 16-55-26

@Thyre
Copy link
Copy Markdown
Collaborator

Thyre commented May 19, 2025

Test report by @Thyre
SUCCESS
Build succeeded for 39 out of 39 (1 easyconfigs in total)
datenlager - Linux Ubuntu 24.04, x86_64, AMD Ryzen 7 3700X 8-Core Processor, Python 3.12.3
See https://gist.github.com/Thyre/dd5c42bc8d92d79b8d52863b9ed4cfd1 for a full test report.

@boegel boegel added this to the next release (5.1.0) milestone May 21, 2025
@boegel boegel changed the title control number of parallel processes used by Ninja and add fix for aarch64 in Qt6-6.7.2 control number of parallel processes used by Ninja and add fix for aarch64 in easyconfig for Qt6 6.7.2 May 21, 2025
boegel
boegel approved these changes May 21, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel
Copy link
Copy Markdown
Member

boegel commented May 21, 2025

Going in, thanks @SebastianAchilles!

@boegel boegel merged commit d050fa8 into easybuilders:develop May 21, 2025
10 checks passed
@boegel boegel added bug fix and removed change labels May 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

aarch64 Related to Arm 64-bit (aarch64) bug fix

Projects

None yet

Milestone

5.1.0

Development

Successfully merging this pull request may close these issues.

5 participants

@SebastianAchilles @boegelbot @Thyre @Crivella @boegel