Skip to content

DPL Analysis: use Input/OutputSpec metadata to detect special tables instead of origins#15213

Open
aalkin wants to merge 1 commit intoAliceO2Group:devfrom
aalkin:metadata-instead-of-origin
Open

DPL Analysis: use Input/OutputSpec metadata to detect special tables instead of origins#15213
aalkin wants to merge 1 commit intoAliceO2Group:devfrom
aalkin:metadata-instead-of-origin

Conversation

@aalkin
Copy link
Copy Markdown
Member

@aalkin aalkin commented Mar 25, 2026

  • Populate OutputSpec for spawned or index table with corresponding metadata, similar to the one present in InputSpec for the same table
  • Use the presence of these metadata entries to form the inputs list for the aod-spawner and index-builder

Previously, special origins DYN/IDX were used to identify spawned and index tables. With embedding in mind, we will need to distinguish between the spawned and index tables coming from different file sources, meaning we can no longer rely on origins. This change moves the logic to metadata inspection, that permits us to free the origins for their original purpose - determining the source of data.

@aalkin aalkin requested a review from a team as a code owner March 25, 2026 11:51
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 25, 2026

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

@alibuild alibuild mentioned this pull request Mar 25, 2026
Merged
@aalkin aalkin force-pushed the metadata-instead-of-origin branch from fa96f06 to a27eaf1 Compare March 25, 2026 11:54
@alibuild
Copy link
Copy Markdown
Collaborator

alibuild commented Mar 31, 2026 •
edited
Loading

Error while checking build/O2/fullCI_slc9 for a27eaf1 at 2026-03-31 16:59:

## sw/BUILD/O2-full-system-test-latest/log
command /sw/slc9_x86-64/O2/slc9_x86-64-slc9_x86-64-local4/prodtests/full-system-test/dpl-workflow.sh had nonzero exit code 1
[9071:internal-dpl-ccdb-backend]: [16:58:45][ERROR] Failed to open file alien:///alice/data/CCDB/CTP/Calib/OrbitReset/01/35179/b07c53e0-b4c0-11ec-b66d-90ce809b250c?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:58:51][ERROR] Failed to open file alien:///alice/data/CCDB/EMC/Config/RecoParam/02/14620/d791f4c0-3ffb-11ed-a67e-2a010e0a0b16?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:58:53][ERROR] Failed to open file alien:///alice/data/CCDB/GLO/Config/GRPECS/04/02718/cf968f3f-0210-11ed-8000-200114580202?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:58:56][ERROR] Failed to open file alien:///alice/data/CCDB/ITS/Config/ClustererParam/06/44918/b0c41285-f1ae-11ec-b9c0-2a010e0a0b16?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:58:58][ERROR] Failed to open file alien:///alice/data/CCDB/MFT/Config/ClustererParam/06/44388/8c661b10-f480-11ec-841e-2a010e0a0b16?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:00][ERROR] Failed to open file alien:///alice/data/CCDB/MFT/Calib/ClusterDictionary/13/20237/c9a0bd70-9fe3-11ec-975c-200114580202?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:03][ERROR] Failed to open file alien:///alice/data/CCDB/PHS/Config/RecoParams/09/61096/abd15e1e-0613-11ed-8002-0aa202a21b9a?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:06][ERROR] Failed to open file alien:///alice/data/CCDB/CTP/Config/TriggerOffsets/05/51872/907486a2-ecaa-11ec-bea3-2a010e0a0b16?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:08][ERROR] Failed to open file alien:///alice/data/CCDB/CPV/Calib/Pedestals/06/01515/aa7d7bad-caea-11ec-8000-0aa202a21b9a?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:10][ERROR] Failed to open file alien:///alice/data/CCDB/CPV/Calib/BadChannelMap/12/29504/aa7c4363-caea-11ec-8000-0aa202a21b9a?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:12][ERROR] Failed to open file alien:///alice/data/CCDB/FDD/Config/LookupTable/07/18689/1b819df6-c186-11ed-a9a7-200114580204?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:19][ERROR] Failed to open file alien:///alice/data/CCDB/ZDC/Config/Module/12/06629/706c8778-60e8-11ed-9dc3-2a010e0a0b16?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:23][ERROR] Failed to open file alien:///alice/data/CCDB/TPC/Config/FEEPad/07/32310/3e279f1b-5e92-11ed-9dc3-200114580202?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:25][ERROR] Failed to open file alien:///alice/data/CCDB/TPC/Calib/PadGainFull/10/00989/b929213c-a46e-11ec-982e-0aa2043e1b9a?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:30][ERROR] Failed to open file alien:///alice/data/CCDB/TPC/Calib/LaserTracks/14/60057/dd26d771-0929-11ed-8000-2a010e0a0b16?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:33][ERROR] Failed to open file alien:///alice/data/CCDB/ZDC/Calib/TDCCorr/01/30762/0d5c2790-cca6-11ec-b790-200114580d00?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:37][ERROR] Failed to open file alien:///alice/data/CCDB/PHS/BadMap/Occ/03/18733/73f8583d-b1d5-11ec-a6f9-0aa202a21b9a?filetype=raw
[9071:internal-dpl-ccdb-backend]: [16:59:37][ERROR] File PHS/BadMap/Occ could not be retrieved. No more hosts to try.
[9071:internal-dpl-ccdb-backend]: [16:59:37][FATAL] Unable to find CCDB object PHS/BadMap/Occ/1550600800023
[9071:internal-dpl-ccdb-backend]: [16:59:37][ERROR] Exception while running: Fatal error. Rethrowing.
[9071:internal-dpl-ccdb-backend]: [16:59:37][FATAL] Unhandled o2::framework::runtime_error reached the top of main of o2-itsmft-stf-decoder-workflow, device shutting down. Reason: Fatal error
[ERROR] Workflow crashed - PID 9071 (internal-dpl-ccdb-backend) did not exit correctly however it's not clear why. Exit code forced to 128.
[ERROR] Workflow crashed - PID 9091 (gpu-reconstruction) was killed abnormally with Killed and exited code was set to 137.
[ERROR]  - Device internal-dpl-ccdb-backend: pid 9071 (exit 128)
[INFO]    - First error: [16:59:37][FATAL] Unable to find CCDB object PHS/BadMap/Occ/1550600800023
[ERROR]  - Device gpu-reconstruction: pid 9091 (exit 137)
[ERROR] SEVERE: Device internal-dpl-ccdb-backend (9071) had at least one message above severity 7: Unable to find CCDB object PHS/BadMap/Occ/1550600800023


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
grep: error-log.txt: binary file matches
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/fa843402c25da9af52b48c5b9bf4f6c541df66cd/slc9_x86-64/o2checkcode/1.0-local25/etc/modulefiles
++ cat
--

Full log here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@aalkin @alibuild