{ai,tools}[foss/2023a] WhisperX v3.3.1, pyannote.audio v3.3.2 w/ CUDA 12.1.1 by pavelToman · Pull Request #22259 · easybuilders/easybuild-easyconfigs

pavelToman · 2025-02-04T17:42:38Z

(created using eb --new-pr)
resolves vscentrum/vsc-software-stack#482
require:

unset $BUILD_VERSION set by torchvision easyblock in preinstallopts for torchaudio in easyconfigs for PyTorch-bundle 2.1.2 #22258

…e.audio-3.3.2-foss-2023a-CUDA-12.1.1.eb

pavelToman · 2025-02-04T17:45:00Z

@boegelbot please test @ jsc-zen3-a100

boegelbot · 2025-02-04T17:50:08Z

@pavelToman: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=22259 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_22259 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 5663

Test results coming soon (I hope)...

Details

- notification for comment with ID 2634657369 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

pavelToman · 2025-02-04T18:03:44Z

Test report by @pavelToman
SUCCESS
Build succeeded for 3 out of 3 (2 easyconfigs in total)
node3305.joltik.os - Linux RHEL 9.4, x86_64, Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz, 1 x NVIDIA Tesla V100-SXM2-32GB, 555.42.06, Python 3.9.18
See https://gist.github.com/pavelToman/62be1d8b394809a3ce8e46a2bca1967e for a full test report.

pavelToman · 2025-02-04T18:05:53Z

Test report by @pavelToman
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
node4012.donphan.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz, 1 x NVIDIA NVIDIA A2, 545.23.08, Python 3.6.8
See https://gist.github.com/pavelToman/afbaff99157222cb06ca90095974d4d3 for a full test report.

boegelbot · 2025-02-04T19:28:02Z

Test report by @boegelbot
FAILED
Build succeeded for 1 out of 3 (2 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.21
See https://gist.github.com/boegelbot/e6514284c68dec5c39eca0fe35ee71a2 for a full test report.

pavelToman · 2025-02-05T09:48:27Z

@boegelbot please test @ jsc-zen3-a100

boegelbot · 2025-02-05T09:50:09Z

@pavelToman: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=22259 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_22259 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 5664

Test results coming soon (I hope)...

Details

- notification for comment with ID 2636238748 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2025-02-05T09:56:08Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 2 (2 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.21
See https://gist.github.com/boegelbot/51cbe238de3b8eb0838257d9d31f9cb9 for a full test report.

pavelToman · 2025-02-05T10:16:10Z

Test report by @boegelbot FAILED Build succeeded for 0 out of 2 (2 easyconfigs in total) jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.21 See https://gist.github.com/boegelbot/51cbe238de3b8eb0838257d9d31f9cb9 for a full test report.

== 2025-02-05 09:56:02,102 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/easybuild-framework/easybuild/base/exceptions.py:126 in __init__): Sanity check failed: extensions sanity check failed for 1 extensions: soundfile
failing sanity check for 'soundfile' extension: command "python -c "import soundfile"" failed; output:
Traceback (most recent call last):
  File "/project/def-maintainers/boegelbot/rocky9/zen3/software/pyannote.audio/3.3.2-foss-2023a-CUDA-12.1.1/lib/python3.11/site-packages/soundfile.py", line 161, in <module>
    import _soundfile_data  # ImportError if this doesn't exist
    ^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_soundfile_data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/def-maintainers/boegelbot/rocky9/zen3/software/pyannote.audio/3.3.2-foss-2023a-CUDA-12.1.1/lib/python3.11/site-packages/soundfile.py", line 170, in <module>
    raise OSError('sndfile library not found using ctypes.util.find_library')
OSError: sndfile library not found using ctypes.util.find_library

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/project/def-maintainers/boegelbot/rocky9/zen3/software/pyannote.audio/3.3.2-foss-2023a-CUDA-12.1.1/lib/python3.11/site-packages/soundfile.py", line 192, in <module>
    _snd = _ffi.dlopen(_explicit_libname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: cannot load library 'libsndfile.so': libsndfile.so: cannot open shared object file: No such file or directory,  (at easybuild/easybuild-framework/easybuild/framework/easyblock.py:3678 in _sanity_check_step)

pavelToman · 2025-02-05T10:52:03Z

Test report by @pavelToman
SUCCESS
Build succeeded for 3 out of 3 (2 easyconfigs in total)
node3908.accelgor.os - Linux RHEL 9.4, x86_64, AMD EPYC 7413 24-Core Processor, 1 x NVIDIA NVIDIA A100-SXM4-80GB, 555.42.06, Python 3.9.18
See https://gist.github.com/pavelToman/641c828e0f9b387ddf41f432d0d58928 for a full test report.

pavelToman · 2025-02-05T11:58:20Z

@boegelbot please test @ jsc-zen3-a100

boegelbot · 2025-02-05T12:00:08Z

@pavelToman: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=22259 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_22259 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 5665

Test results coming soon (I hope)...

Details

- notification for comment with ID 2636530692 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2025-02-05T12:07:08Z

Test report by @boegelbot
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.21
See https://gist.github.com/boegelbot/caa1f3c59aa96c8e0e52f4a2298dde39 for a full test report.

pavelToman · 2025-02-05T12:13:56Z

Test report by @pavelToman
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3908.accelgor.os - Linux RHEL 9.4, x86_64, AMD EPYC 7413 24-Core Processor, 1 x NVIDIA NVIDIA A100-SXM4-80GB, 555.42.06, Python 3.9.18
See https://gist.github.com/pavelToman/9958cc873384f02e0bdd99a98d782587 for a full test report.

pavelToman · 2025-02-05T12:58:02Z

Test report by @pavelToman
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
node4011.donphan.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz, 1 x NVIDIA NVIDIA A2, 545.23.08, Python 3.6.8
See https://gist.github.com/pavelToman/04a965579a7241a095a989f6926e19b3 for a full test report.

pavelToman · 2025-02-05T12:59:09Z

Test report by @pavelToman
SUCCESS
Build succeeded for 4 out of 4 (2 easyconfigs in total)
node3305.joltik.os - Linux RHEL 9.4, x86_64, Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz, 1 x NVIDIA Tesla V100-SXM2-32GB, 555.42.06, Python 3.9.18
See https://gist.github.com/pavelToman/9661aafad87531475860b4ac694d903a for a full test report.

boegel · 2025-02-13T17:58:37Z

@boegelbot please test @ jsc-zen3-a100

boegelbot · 2025-02-13T18:00:11Z

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=22259 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_22259 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 5719

Test results coming soon (I hope)...

Details

- notification for comment with ID 2657353235 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegel · 2025-02-13T19:26:16Z

@pavelToman I'm now seeing this, so maybe we need to re-instate that preinstallopts for pyannote-audio after all?

pyannote-audio 3.3.2 has requirement torchaudio>=2.2.0, but you have torchaudio 2.1.2

We better make sure they don't have a good reason to claim they require torchaudio 2.2.0+ though...

boegelbot · 2025-02-13T21:21:13Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 2 (2 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.21
See https://gist.github.com/boegelbot/2f2e930aacd1fccc216123c2e3827765 for a full test report.

pavelToman · 2025-02-14T10:48:49Z

@pavelToman I'm now seeing this, so maybe we need to re-instate that preinstallopts for pyannote-audio after all?
pyannote-audio 3.3.2 has requirement torchaudio>=2.2.0, but you have torchaudio 2.1.2
We better make sure they don't have a good reason to claim they require torchaudio 2.2.0+ though...

From this commit it seems only change is:

- from torchaudio.backend.common import AudioMetaData
+ from torchaudio import AudioMetaData

And the from torchaudio import AudioMetaData is working either with torchaudio-2.1.2, so it should be ok?

pavelToman · 2025-02-14T10:55:30Z

@boegelbot please test @ jsc-zen3-a100

boegelbot · 2025-02-14T11:00:10Z

@pavelToman: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=22259 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_22259 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 5723

Test results coming soon (I hope)...

Details

- notification for comment with ID 2658979725 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2025-02-14T11:07:13Z

Test report by @boegelbot
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.21
See https://gist.github.com/boegelbot/40b775f40bb5622a276c4548debaf23d for a full test report.

boegel · 2025-02-19T17:59:16Z

Test report by @boegel
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3905.accelgor.os - Linux RHEL 9.4, x86_64, AMD EPYC 7413 24-Core Processor (zen3), 1 x NVIDIA NVIDIA A100-SXM4-80GB, 555.42.06, Python 3.9.18
See https://gist.github.com/boegel/4e220c79ffaf7b2dc376c5c2725f6304 for a full test report.

boegel

lgtm

boegel · 2025-02-19T18:23:56Z

Going in, thanks @pavelToman!

adding easyconfigs: WhisperX-3.3.1-foss-2023a-CUDA-12.1.1.eb, pyannot…

a8cf9d6

…e.audio-3.3.2-foss-2023a-CUDA-12.1.1.eb

pavelToman added the new label Feb 4, 2025

pavelToman mentioned this pull request Feb 4, 2025

WhisperX vscentrum/vsc-software-stack#482

Closed

2 tasks

pyannote - add java as builddep

ef6bb44

update deps for soundfile

fec212a

pavelToman closed this Feb 5, 2025

pavelToman reopened this Feb 5, 2025

boegel requested changes Feb 12, 2025

View reviewed changes

Comment thread easybuild/easyconfigs/w/WhisperX/WhisperX-3.3.1-foss-2023a-CUDA-12.1.1.eb Outdated

pavelToman added 2 commits February 12, 2025 14:01

Update WhisperX-3.3.1-foss-2023a-CUDA-12.1.1.eb

590a95c

Update pyannote.audio-3.3.2-foss-2023a-CUDA-12.1.1.eb

34783b3

boegel mentioned this pull request Feb 13, 2025

pyannote.audio vscentrum/vsc-software-stack#509

Closed

pyannote.audio - unpin torchaudio version

743ffc5

boegel approved these changes Feb 19, 2025

View reviewed changes

boegel added this to the release after 4.9.4 milestone Feb 19, 2025

boegel merged commit bd7b20a into easybuilders:develop Feb 19, 2025

This was referenced Feb 28, 2025

{ai,lang}[foss/2023a] Scikit-HEP v2024.8.1, whisper-ctranslate2 v0.5.2 w/ CUDA 12.1.1 #22394

Open

{tools}[foss/2023a] pyannote v2023.12 w/ CUDA 12.1.1 #22404

Closed

boegel modified the milestones: release after 4.9.4, 5.0.0 Mar 18, 2025

Conversation

pavelToman commented Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pavelToman commented Feb 4, 2025

Uh oh!

boegelbot commented Feb 4, 2025

Uh oh!

pavelToman commented Feb 4, 2025

Uh oh!

pavelToman commented Feb 4, 2025

Uh oh!

boegelbot commented Feb 4, 2025

Uh oh!

pavelToman commented Feb 5, 2025

Uh oh!

boegelbot commented Feb 5, 2025

Uh oh!

boegelbot commented Feb 5, 2025

Uh oh!

pavelToman commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pavelToman commented Feb 5, 2025

Uh oh!

pavelToman commented Feb 5, 2025

Uh oh!

boegelbot commented Feb 5, 2025

Uh oh!

boegelbot commented Feb 5, 2025

Uh oh!

pavelToman commented Feb 5, 2025

Uh oh!

pavelToman commented Feb 5, 2025

Uh oh!

pavelToman commented Feb 5, 2025

Uh oh!

Uh oh!

boegel commented Feb 13, 2025

Uh oh!

boegelbot commented Feb 13, 2025

Uh oh!

boegel commented Feb 13, 2025

Uh oh!

boegelbot commented Feb 13, 2025

Uh oh!

pavelToman commented Feb 14, 2025

Uh oh!

pavelToman commented Feb 14, 2025

Uh oh!

boegelbot commented Feb 14, 2025

Uh oh!

boegelbot commented Feb 14, 2025

Uh oh!

boegel commented Feb 19, 2025

Uh oh!

boegel left a comment

Choose a reason for hiding this comment

Uh oh!

boegel commented Feb 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pavelToman commented Feb 4, 2025 •

edited

Loading

pavelToman commented Feb 5, 2025 •

edited

Loading