Skip to content

[WIP] Final migration to configuration version 1: structured and validated configs#3198

Draft
deruyter92 wants to merge 87 commits intomainfrom
feat/structured_configs
Draft

[WIP] Final migration to configuration version 1: structured and validated configs#3198
deruyter92 wants to merge 87 commits intomainfrom
feat/structured_configs

Conversation

@deruyter92
Copy link
Copy Markdown
Collaborator

This PR aims to implement the features proposed in #3193, and will be the final PR for merging all changes from the feature branch, implemented in the smaller PRs #3190, #3191, #3194, etc..

[A more elaborate description will added]

@deruyter92 deruyter92 added this to the Structured configs milestone Feb 4, 2026
@deruyter92 deruyter92 force-pushed the feat/structured_configs branch 5 times, most recently from b5f2fdc to fdfec16 Compare February 4, 2026 11:04
@deruyter92 deruyter92 force-pushed the feat/structured_configs branch 2 times, most recently from 786653b to 46a0e56 Compare February 18, 2026 07:31
deruyter92 and others added 21 commits February 18, 2026 09:19
- reading breaks for corrupted yaml
- added tests for prefered behavior that is currently not implemented (yaml safeloading and validating config keys)
This mixin provides methods for:
    - Loading configurations from dictionaries or YAML files
    - Validating configuration data against pydantic models
    - Converting configurations to dictionaries
    - Pretty printing configuration data

add imports from utils
- Introduced new configuration classes for inference, logging, model, pose, project, runner, and training settings.
- Refactored data loading mechanisms to utilize new configuration structures.
- Moved the multithreading and compilation options in inference configuration to the config module.
- Typed configuration for logging.
- Updated dataset loaders to accept model configurations directly or via file paths.
(The fields are kept identical to old multianimal project configs for now)

move ProjectConfig to deeplabcut/core/config
The return value should be the dictionary, not the instantiated transforms
@deruyter92 deruyter92 force-pushed the feat/structured_configs branch from 760c7ae to 00e2b77 Compare February 18, 2026 12:37
deruyter92 and others added 27 commits February 18, 2026 13:38
…fig/dict/None-type handling.

C3-II Additional refactoring of configurations 
Part 3-II of migration to typed configs, see issue #3193
…ith validate_assignment=True)

This is necessary to prevent a bug:
The isinstance(data, cls): return data shortcut in MigrationMixin.migrate_then_validate (a model_validator(mode="wrap")). Pydantic re-enters this validator during validate_assignment, passing the current instance as data. The shortcut returned it unchanged, silently discarding every field update. This broke all assignment — not just validation, but even plain cfg.count = 42 was a no-op.

The problem is that model_validator(mode="wrap") wraps the entire validation pipeline, including validate_assignment flow. Migration should only run during construction, not on every field assignment.
The clean solution is to switch from mode="wrap" to mode="before" — it transforms raw input data before validation and doesn't participate in validate_assignment at all.
Part 4 of migration to typed configs, see issue #3193
Part 5 of migration to typed configs, see issue [#3193](#3193)

Removes the intermediary DictConfig format and works with fully typed configuration dataclasses instead (which have dict access funcitonality).
@deruyter92 deruyter92 added the WORK IN PROGRESS! developers are currently working on this feature... stay tuned. label Feb 28, 2026
@C-Achard C-Achard added the lint required Please run pre-commit hooks to ensure your formatting is up-to-date label Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lint required Please run pre-commit hooks to ensure your formatting is up-to-date WORK IN PROGRESS! developers are currently working on this feature... stay tuned.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants