Skip to content

[AI-FSSDK] [FSSDK-12368] Local Holdouts - Cleanup flag base setup and add includedRules and rule-level lookup#501

Open
Mat001 wants to merge 1 commit intomasterfrom
ai/mat001/FSSDK-12368
Open

[AI-FSSDK] [FSSDK-12368] Local Holdouts - Cleanup flag base setup and add includedRules and rule-level lookup#501
Mat001 wants to merge 1 commit intomasterfrom
ai/mat001/FSSDK-12368

Conversation

@Mat001
Copy link
Copy Markdown
Contributor

@Mat001 Mat001 commented Apr 10, 2026

Summary

Implements Local Holdouts support for the Python SDK, replacing legacy flag-level holdout targeting (includedFlags/excludedFlags) with rule-level targeting (includedRules).

Related Ticket: FSSDK-12368

Changes

Data Model

  • Added included_rules field to Holdout model (optional array of rule IDs)
  • Added is_global property (True when included_rules is None)
  • Removed legacy fields: included_flags, excluded_flags

Holdout Configuration

  • Updated HoldoutConfig from flag-level to rule-level mapping
  • Implemented get_global_holdouts() - returns holdouts with included_rules == None
  • Implemented get_holdouts_for_rule(rule_id) - returns holdouts targeting specific rule
  • Added validation for rule holdouts map integrity

Decision Flow

  • Global holdouts evaluated at flag level (before forced decisions)
  • Local holdouts evaluated per-rule (after forced decisions, before audience/traffic)
  • If user hits local holdout, rule evaluation is skipped
  • Returns correct decision metadata: source="holdout", experiment_id, ruleKey

Edge Cases

  • Missing included_rules field defaults to None (global holdout)
  • Empty array [] vs None handled correctly (empty = local with no rules, None = global)
  • Invalid rule IDs skipped gracefully (no errors)
  • Cross-flag targeting supported (one holdout can target rules from multiple flags)

Testing

New Tests (11 comprehensive tests in tests/test_local_holdouts.py)

  • Global holdout evaluation
  • Local single-rule targeting
  • Local multi-rule targeting (same flag and cross-flag)
  • Precedence (global before local)
  • Edge cases (missing field, empty array, invalid rule IDs)

Updated Tests (60 existing tests)

  • Migrated from included_flags/excluded_flags to included_rules
  • Updated to use is_global property
  • Verified backward compatibility

Quality Metrics

  • ✅ Tests: 71/71 passing (100%)
  • ✅ Critical Issues: 0
  • ✅ Warnings: 0
  • ✅ Comprehensive edge case coverage

Files Modified

  • optimizely/entities.py - Holdout model updates
  • optimizely/project_config.py - Mapping and retrieval logic
  • optimizely/decision_service.py - Decision flow integration
  • optimizely/helpers/types.py - Type definitions
  • tests/test_local_holdouts.py - New comprehensive test suite
  • tests/test_decision_service_holdout.py - API updates
  • tests/test_config.py - Model updates

Test Plan

  • Unit tests pass (71/71)
  • Global holdouts work correctly
  • Local holdouts work correctly (single rule, multiple rules)
  • Edge cases handled (missing field, empty array, invalid IDs)
  • Backward compatibility verified (old datafiles work)
  • Cross-flag targeting works
  • Precedence correct (global → forced → local → rule)

🤖 Generated with Claude Code

Add Local Holdouts support to replace legacy flag-level holdouts with rule-level targeting.

Changes:
- Add includedRules field to Holdout model (replaces includedFlags/excludedFlags)
- Add isGlobal property for global vs local holdout detection
- Update HoldoutConfig mapping from flag-level to rule-level
- Implement get_global_holdouts() and get_holdouts_for_rule() methods
- Integrate local holdout evaluation in decision flow (per-rule, before audience/traffic)
- Handle edge cases (missing field, empty array, invalid rule IDs, cross-flag targeting)
- Add comprehensive unit tests for local holdouts
- Update existing tests to use new API

Quality Metrics:
- Tests: 71/71 passing (100%)
- Critical Issues: 0
- Warnings: 0

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant