use `DCPU_BASELINE=DETECT` for OpenCV when default optarch compiler option is used by smoors · Pull Request #2954 · easybuilders/easybuild-easyblocks

smoors · 2023-06-24T15:26:28Z

(created using eb --new-pr)

sets -DCPU_BASELINE=DETECT also if

GCC is used and optarch is set to march=native (default for GCC)
intel is used and optarch is set to xHost (default for intel)

avoids build failure for example with GCC and
--optarch='Intel:march=core-avx2;GCC:march=native'

boegel · 2023-07-05T14:35:11Z

+            if (
+                not optarch
+                or (get_software_root('GCC') and (optarch == 'march=native' or optarch.get('GCC') == 'march=native'))
+                or (get_software_root('intel-compilers') and (optarch == 'xHost' or optarch.get('Intel') == 'xHost'))


Ideally we don't hardcode march=native and xHost here, since those values may change.

We should check whether we can extract the default from framework (and if not maybe we need to introduce a constant in framework so the default can be easily obtained without hardcoding)

And note that a manually set optarch may well be something more than march=native/xHost so a simple == doesn't suffice

@boegel @akesandgren done in 7310de7

migueldiascosta · 2023-07-18T23:04:29Z

+                or (isinstance(optarch, str) and optimal_arch_option in optarch)
+                or (isinstance(optarch, dict) and (
+                    (get_software_root('GCC') and optimal_arch_option in optarch.get('GCC', ''))
+                    or (get_software_root('intel-compilers') and optimal_arch_option in optarch.get('Intel', ''))


for intel < 2021a we would need to look for iccifort instead of intel-compilers?

done in eb0ee64

smoors · 2023-07-19T14:20:48Z

Test report by @smoors

Overview of tested easyconfigs (in order)

SUCCESS OpenCV-4.6.0-foss-2022a-contrib.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node301.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/smoors/ac803c8f88a0203138902442ad5dd4e5 for a full test report.

migueldiascosta · 2023-07-20T04:11:42Z

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

SUCCESS OpenCV-4.2.0-foss-2019b-Python-3.7.4.eb
SUCCESS OpenCV-4.2.0-foss-2020a-Python-3.8.2-contrib.eb
SUCCESS OpenCV-4.2.0-foss-2020a-Python-3.8.2.eb
SUCCESS OpenCV-4.5.1-foss-2020b-contrib.eb
SUCCESS OpenCV-4.5.3-foss-2021a-contrib.eb
SUCCESS OpenCV-4.5.5-foss-2021b-contrib.eb

Build succeeded for 6 out of 6 (6 easyconfigs in total)
sms - Linux CentOS Linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor, Python 3.6.8
See https://gist.github.com/migueldiascosta/a5f47dc26a901be29ee59c8e47f486a8 for a full test report.

migueldiascosta · 2023-07-20T09:31:26Z

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

SUCCESS OpenCV-3.3.0-intel-2017b-Python-2.7.14.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
sms - Linux CentOS Linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor, Python 3.6.8
See https://gist.github.com/migueldiascosta/32b52d01997d10a77188eb93c97f4f40 for a full test report.

boegel · 2023-08-17T09:53:48Z

Test report by @boegel

Overview of tested easyconfigs (in order)

FAIL (build issue) OpenCV-4.6.0-foss-2022a-contrib.eb (partial log available at https://gist.github.com/boegel/1f546b88ece4f262fbcb1f8ce3ecfd9a)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
node3568.doduo.os - Linux RHEL 8.6, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/boegel/fa275d7d035bd8b760e137e67d9f6bd1 for a full test report.

boegel · 2023-08-17T09:55:57Z

@smoors Hmm, I was hoping this would work with EasyBuild is configured with $EASYBUILD_OPTARCH set to "Intel:march=core-avx2", but it doesn't?

smoors · 2023-08-17T14:32:17Z

Test report by @smoors

Overview of tested easyconfigs (in order)

SUCCESS OpenCV-4.6.0-foss-2022a-contrib.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node364.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/smoors/5c6d35c3ef87428212dd77a4fdb72807 for a full test report.

smoors · 2023-08-17T14:38:18Z

@smoors Hmm, I was hoping this would work with EasyBuild is configured with $EASYBUILD_OPTARCH set to "Intel:march=core-avx2", but it doesn't?

that case was missing indeed, fixed in 25a66f0

boegel · 2023-08-30T13:48:40Z

Test report by @boegel

Overview of tested easyconfigs (in order)

SUCCESS OpenCV-4.6.0-foss-2022a-contrib.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3537.doduo.os - Linux RHEL 8.6, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/boegel/a4e0c77764dff04a5b6270657e38277f for a full test report.

boegel · 2023-08-30T14:42:11Z

Test report by @boegel

Overview of tested easyconfigs (in order)

SUCCESS OpenCV-3.4.1-intel-2018a-Python-3.6.4.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3102.skitty.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 2.7.5
See https://gist.github.com/boegel/3473dd29b8e7f07bd68a28f7211fc7d6 for a full test report.

improve cpu-arch optimization settings for OpenCV

323ec7a

smoors added the enhancement label Jun 24, 2023

Samuel Moors added 2 commits June 24, 2023 17:43

also for intel compilers and xHost

e0d703b

simplify

fcdf922

boegel added this to the release after 4.7.3 milestone Jul 5, 2023

boegel requested changes Jul 5, 2023

View reviewed changes

Samuel Moors added 2 commits July 18, 2023 21:54

use COMPILER_OPTIMAL_ARCHITECTURE_OPTION

7310de7

remove unused imports

ee27c12

migueldiascosta reviewed Jul 18, 2023

View reviewed changes

also check iccifort; refactor for readability

eb0ee64

branfosj mentioned this pull request Jul 25, 2023

{vis}[foss/2022b] OpenCV v4.8.0 w/ contrib easybuilders/easybuild-easyconfigs#18381

Merged

2 tasks

account for missing case

25a66f0

boegel approved these changes Aug 30, 2023

View reviewed changes

boegel merged commit 6bcef45 into easybuilders:develop Aug 30, 2023

boegel changed the title ~~improve cpu-arch optimization settings for OpenCV~~ use DCPU_BASELINE=DETECT for OpenCV when default optarch compiler option is used Sep 10, 2023

Conversation

smoors commented Jun 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegel Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

akesandgren Jul 6, 2023

Choose a reason for hiding this comment

Uh oh!

smoors Jul 18, 2023

Choose a reason for hiding this comment

Uh oh!

migueldiascosta Jul 18, 2023

Choose a reason for hiding this comment

Uh oh!

smoors Jul 19, 2023

Choose a reason for hiding this comment

Uh oh!

smoors commented Jul 19, 2023

Overview of tested easyconfigs (in order)

Uh oh!

migueldiascosta commented Jul 20, 2023

Overview of tested easyconfigs (in order)

Uh oh!

migueldiascosta commented Jul 20, 2023

Overview of tested easyconfigs (in order)

Uh oh!

boegel commented Aug 17, 2023

Overview of tested easyconfigs (in order)

Uh oh!

boegel commented Aug 17, 2023

Uh oh!

smoors commented Aug 17, 2023

Overview of tested easyconfigs (in order)

Uh oh!

smoors commented Aug 17, 2023

Uh oh!

boegel commented Aug 30, 2023

Overview of tested easyconfigs (in order)

Uh oh!

boegel commented Aug 30, 2023

Overview of tested easyconfigs (in order)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

smoors commented Jun 24, 2023 •

edited

Loading