feat: Add version tracking to FeatureView#6101
feat: Add version tracking to FeatureView#6101franciscojavierarceo wants to merge 28 commits intomasterfrom
Conversation
…emandFeatureView Every `feast apply` now creates a version snapshot. Users can pin a feature view to a specific historical version declaratively via `version="v2"`. By default, the latest version is always served. - New proto: FeatureViewVersion.proto with version record/history - Added `version` field to FeatureViewSpec, StreamFeatureViewSpec, OnDemandFeatureViewSpec and version metadata to their Meta messages - New version_utils module for parsing/normalizing version strings - Version-aware apply_feature_view in both SQL and file registries - New `list_feature_view_versions` API on FeatureStore and registries - CLI: `feast feature-views versions <name>` subcommand - Updated all 14 templates with explicit `version="latest"` - Unit tests (28) and integration tests (7) for versioning Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Fix current_version_number=0 being silently dropped during proto deserialization in FeatureView, OnDemandFeatureView (proto3 int32 default 0 is falsy in Python); use spec.version to disambiguate - Add current_version_number restoration in StreamFeatureView.from_proto (was missing entirely) - Use timezone-aware UTC datetime in SqlRegistry.list_feature_view_versions for consistency with the rest of the codebase - Add test for v0 proto roundtrip Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Add Versioning section to feature-view.md concept page covering automatic snapshots, version pinning, version string formats, CLI usage, and Python SDK API - Add `feast feature-views versions` command to CLI reference Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Fix current_version_number roundtrip bug: version="latest" (always truthy) caused None to become 0 after proto roundtrip; now check that spec.version is not "latest" before treating 0 as intentional - Use write_engine (not read_engine) for pre/post apply reads in SqlRegistry to avoid read replica lag causing missed version snapshots - Remove redundant version check in StreamFeatureView.__eq__ (parent FeatureView.__eq__ already checks it) - Add else clause to StreamFeatureView.from_proto for consistency - Add test for latest/None roundtrip preservation Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ntly - delete_feature_view now also deletes version history records, preventing IntegrityError when re-creating a previously deleted FV - _get_next_version_number uses write_engine instead of read_engine to avoid stale version numbers with read replicas Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Add step-by-step walkthrough showing how versions auto-increment on changes and skip on identical re-applies - Add CLI example showing the apply/change/apply cycle - Clarify that pinning ignores constructor params and uses the snapshot - Explain how to return to auto-incrementing after a pin/revert Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
🐛 1 issue in files not directly in the diff
🐛 BatchFeatureView missing version parameter — incomplete transformation (sdk/python/feast/batch_feature_view.py:80-102)
FeatureView, StreamFeatureView, and OnDemandFeatureView all accept a version parameter in their constructors, but BatchFeatureView.__init__ (sdk/python/feast/batch_feature_view.py:80-102) does not. Furthermore, BatchFeatureView.__init__ does not pass version to super().__init__() at sdk/python/feast/batch_feature_view.py:144-158. This means users cannot construct a BatchFeatureView with a pinned version (e.g., BatchFeatureView(..., version="v2") raises TypeError), and any BatchFeatureView created directly always defaults to version="latest". The proto round-trip path works because FeatureView.from_proto sets feature_view.version after construction, but the constructor API is inconsistent with the other feature view types.
View 19 additional findings in Devin Review.
Raises FeatureViewPinConflict when a user pins to an older version while also modifying the feature view definition (schema, source, etc.). Fixes FeatureView.__copy__() to include description and owner fields, which was causing false positive conflict detection. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
🐛 1 issue in files not directly in the diff
🐛 BatchFeatureView missing version parameter — incomplete transformation (sdk/python/feast/batch_feature_view.py:80-102)
FeatureView, StreamFeatureView, and OnDemandFeatureView all accept a version parameter in their constructors, but BatchFeatureView.__init__ (sdk/python/feast/batch_feature_view.py:80-102) does not. Furthermore, BatchFeatureView.__init__ does not pass version to super().__init__() at sdk/python/feast/batch_feature_view.py:144-158. This means users cannot construct a BatchFeatureView with a pinned version (e.g., BatchFeatureView(..., version="v2") raises TypeError), and any BatchFeatureView created directly always defaults to version="latest". The proto round-trip path works because FeatureView.from_proto sets feature_view.version after construction, but the constructor API is inconsistent with the other feature view types.
View 23 additional findings in Devin Review.
- Add version parameter to BatchFeatureView constructor for consistency with FeatureView, StreamFeatureView, and OnDemandFeatureView - Clean up version history records in file registry delete_feature_view to prevent orphaned records on re-creation - Fix current_version_number proto roundtrip: preserve 0 when version="latest" (after first apply) instead of incorrectly returning None Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Clarify that versioning provides definition management and rollback, not concurrent multi-version serving. Document recommended approaches (separate projects or distinct FV names) for A/B testing scenarios. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Extends feature view versioning with support for reading features from specific
versions at query time using the syntax: "driver_stats@v2:trips_today"
Core changes:
- Add _parse_feature_ref() to parse version-qualified feature references
- Update all feature reference parsing to use _parse_feature_ref()
- Add get_feature_view_by_version() to BaseRegistry and all implementations
- Add FeatureViewProjection.version_tag for multi-version query support
- Add version-aware _table_id() in SQLite online store (v0→unversioned, v1+→_v{N})
- Add VersionedOnlineReadNotSupported error for unsupported stores
Features:
- "driver_stats:trips" = "driver_stats@latest:trips" (backward compatible)
- "driver_stats@v2:trips" reads from v2 snapshot using _v2 table suffix
- Multiple versions in same query: ["driver@v1:trips", "driver@v2:daily"]
- Version parameter added to all decorator functions for consistency
Backward compatibility:
- Unversioned table serves as v0, only v1+ get _v{N} suffix
- All existing queries work unchanged
- SQLite-only for now, other stores raise clear error
Documentation:
- Updated feature-view.md with @Version syntax examples
- Updated feature-retrieval.md reference format
- Added version examples to how-to guides
Tests: 47 unit + 11 integration tests pass, no regressions
Co-Authored-By: Claude Sonnet 4 <[email protected]>
- Fix type inference issues in get_feature_view_by_version() - Use distinct variable names for different proto types - Ensure proper type annotations for BaseFeatureView subclasses
|
I haven't been able to go through the whole thing yet, but iiuc the expectation is that online stores would be expected to handle this by essentially treating fv1@v1 and fv1@v2 as two different feature views, right? (sqlite example infers table name from the version). Just wondering, maybe that's to high a bat for online store implementations. for example, if you have fv1@v1 and fv1@v2 both containing 100 features that differ by a single feature only, the online store would have to keep redundant copies of the other 99 features or do some complicated logic to diff values and deduplicate. wdyt about concentrating on introducing versioning on feature level instead? tbh, that makes more intuitive sense to me. there's a feature called |
I spent some time reasoning about feature-level versioning with Claude. My initial reaction was that it's too large of a change and it only works today in a broken sense. By "in a broken sense" I mean that today we don't really version feature views or features, if someone changes the feature view or the feature, we just overwrite it and lose history. Moreover, we essentially force the materialization to be out of sync. Today, that just ignores the behavior silently. I like how we've implemented it here (i.e., creating a new table and storing the history of the metadata but allowing for callers to specify exact versions and defaulting to the existing behavior) because it is far more explicit about the challenges of materialization consistency issues when you change feature versions. So, I don't recommend we do feature-level versioning as I worry it makes materialization very unreliable. We can, in fact, declare feature view level versioning because transformations are a collection of features mapped one-to-one with a table. |
Implement optional feature view version metadata in API responses to address the issue where internal @v2 version syntax was leaking into client responses. Co-Authored-By: Claude Sonnet 4 <[email protected]>
Add missing include_feature_view_version_metadata parameter to: - EmbeddedGoOnlineFeaturesService.get_online_features() - FooProvider.get_online_features() and get_online_features_async() - FooProvider.retrieve_online_documents() and retrieve_online_documents_v2() This resolves CI failures where provider implementations were not updated with the new parameter from the abstract Provider interface. Co-Authored-By: Claude Sonnet 4 <[email protected]>
Add missing include_feature_view_version_metadata parameter to: - SqliteOnlineStore.retrieve_online_documents/v2 - FaissOnlineStore.retrieve_online_documents - QdrantOnlineStore.retrieve_online_documents - MilvusOnlineStore.retrieve_online_documents_v2 - RemoteOnlineStore.retrieve_online_documents/v2 - PostgresOnlineStore.retrieve_online_documents/v2 - ElasticsearchOnlineStore.retrieve_online_documents/v2 Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Fix _schema_or_udf_changed overrides to accept BaseFeatureView parameter type, satisfying Liskov substitution principle. Each subclass narrows via isinstance check in the method body. - Use getattr for __code__ access on UDF in StreamFeatureView to handle MethodType correctly. - Change _update_metadata_fields proto parameter from Message to Any since the method accesses .spec and .meta attributes. - Guard BaseFeatureView attribute access (online, offline, etc.) with hasattr checks in _update_metadata_fields. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Fix version-qualified features dropped with full_feature_names=True: use _parse_feature_ref to build clean requested_result_row_names - Fix retrieve_online_documents breaking with @vn refs: use _parse_feature_ref instead of split(":") for FV name extraction - Fix metadata-only updates not committed: add self.commit() after _update_metadata_fields in file registry - Fix ODFV transforms broken by version-qualified refs: use _parse_feature_ref in _augment_response_with_on_demand_transforms - Fix _update_metadata_fields not updating spec.version: add version field update so pinned-to-latest transitions persist - Fix _resolve_feature_counts inflating FV count: strip @vn from feature view names in metrics - Fix version snapshots storing stale current_version_number: set version number before serializing snapshot in both file and SQL registries Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Set spec.project on snapshot protos in SqlRegistry before serializing, so version snapshots include the correct project field - Fix _check_versioned_read_support to check projection.version_tag instead of current_version_number, so explicitly version-qualified reads (@v0) are correctly rejected on non-SQLite stores Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Versioning was always active on every `feast apply`. This adds an `enable_feature_view_versioning` boolean (default False) to RegistryConfig so version history, version pins, and version-qualified refs are only available when explicitly enabled. Existing behaviour is fully preserved when the flag is set to true. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
| def _table_id(project: str, table: FeatureView) -> str: | ||
| return f"{project}_{table.name}" | ||
| name = table.name | ||
| version = getattr(table, "current_version_number", None) | ||
| if version is not None and version > 0: | ||
| name = f"{table.name}_v{version}" | ||
| return f"{project}_{name}" |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
- Fix SQLite _table_id to prefer projection.version_tag over current_version_number so @v2 refs read from the correct table - Detect feature name collisions for multi-version queries with full_feature_names=True (e.g. fv@v1:feat vs fv@v2:feat) - Remove unused include_feature_view_version_metadata parameter from retrieve_online_documents (v1) across all providers and online stores - Remove redundant name check from _schema_or_udf_changed since callers always match by name first Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…queries When version-qualified refs (e.g. fv@v1:feat, fv@v2:feat) are used, include the version tag in full_feature_names output so multi-version queries produce distinct columns (fv@v1__feat vs fv@v2__feat). Also fix proto roundtrip test to match -1 sentinel behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…d declaration Auto-increment path (version="latest") now retries up to 3 times on IntegrityError in the SQL registry when concurrent applies race on the same version number. Explicit version path (version="v<N>") now creates the version if it doesn't exist (forward declaration) instead of raising FeatureViewVersionNotFound, with ConcurrentVersionConflict on race. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Fail fast at apply time and retrieval time when a feature service references a versioned FV (current_version_number > 0) and enable_online_feature_view_versioning is off. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Rename enable_feature_view_versioning -> enable_online_feature_view_versioning to clarify that it controls online reads, not version history tracking. Fix mypy type narrowing issues with current_version_number (int | None) and variable redefinition in feature_store.py. Remove -1 sentinel from proto serialization in favor of treating proto default 0 without spec.version as None. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
When enable_online_feature_view_versioning is on and a FeatureService references a versioned FV, set version_tag on the projection so that online reads resolve to the correct versioned table. Previously the FeatureService path never set version_tag, causing reads from the wrong (unversioned) online store table. Changes: - _get_features(): version-qualify feature refs for FeatureService projections - _get_feature_views_to_use(): capture current_version_number before with_projection() discards it, then set version_tag on the projection - feature_store.py: fix mypy type narrowing for the gate check - Add integration tests for both code paths Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
| # Re-serialize with updated version number | ||
| with self.write_engine.begin() as conn: | ||
| update_stmt = ( | ||
| update(fv_table) | ||
| .where( | ||
| fv_table.c.feature_view_name == feature_view.name, | ||
| fv_table.c.project_id == project, | ||
| ) | ||
| .values( | ||
| feature_view_proto=snapshot_proto_bytes, | ||
| ) | ||
| ) | ||
| conn.execute(update_stmt) |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
sdk/python/feast/infra/online_stores/elasticsearch_online_store/elasticsearch.py
Show resolved
Hide resolved
Response to Devin Review FindingsThanks for the thorough automated review! Here's a summary of where all the findings stand after the latest commits ( Already Resolved (confirmed by Devin itself in follow-up passes)The following were real issues that have been fixed in prior commits:
Not Bugs / By DesignThe remaining open findings are either false positives or by-design behavior:
|
Update the Feature Services section in the RFC to reflect that feature services now correctly serve versioned FVs when the online versioning flag is enabled (automatic version_tag resolution on projections). Rename CLI command from `feast feature-views versions` to `feast feature-views list-versions` for consistency, and update all docs references. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Show current version badge in feature view headers and listing table, and add a Versions tab with expandable version history across all feature view types (Regular, Stream, OnDemand). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
| ) | ||
|
|
||
| # Update timestamp | ||
| existing_proto.meta.last_updated_timestamp.FromDatetime(datetime.utcnow()) |
There was a problem hiding this comment.
🟡 _update_metadata_fields uses deprecated datetime.utcnow() instead of _utc_now(), producing naive datetimes
The new _update_metadata_fields method at sdk/python/feast/infra/registry/registry.py:615 uses datetime.utcnow() which returns a timezone-naive datetime. The rest of the codebase consistently uses _utc_now() (which returns a timezone-aware datetime via datetime.now(tz=timezone.utc)). This inconsistency means the last_updated_timestamp set by metadata-only updates will be a naive datetime, while timestamps set by all other code paths (e.g., sdk/python/feast/infra/registry/registry.py:674) will be timezone-aware. Additionally, datetime.utcnow() is deprecated since Python 3.12.
| existing_proto.meta.last_updated_timestamp.FromDatetime(datetime.utcnow()) | |
| existing_proto.meta.last_updated_timestamp.FromDatetime(_utc_now()) |
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
FeatureView,StreamFeatureView, andOnDemandFeatureViewfeast applycreates a version snapshot in a separate history store (new SQL table / proto field)FeatureView(name="driver_stats", version="v2")version="latest"or omitted), the latest version is always served — fully backward compatibleChanges
Proto layer:
FeatureViewVersion.protowithFeatureViewVersionRecord/FeatureViewVersionHistoryversionfield toFeatureViewSpec,StreamFeatureViewSpec,OnDemandFeatureViewSpeccurrent_version_number/version_idto meta messagesfeature_view_version_historytoRegistryprotoPython SDK:
version_utils.py— parses"latest","v2","version2"(case-insensitive)FeatureViewVersionNotFounderror classversionparameter onFeatureView,StreamFeatureView,OnDemandFeatureViewapply_feature_viewin both SQL registry and file/proto registrylist_feature_view_versions(name, project)on registries andFeatureStorefeast feature-views versions <name>subcommandversion="latest"Backward compatibility:
versionis fully optional — omitting it is identical toversion="latest"metadata.create_all(checkfirst=True)auto-creates the new history tableTest plan
🤖 Generated with Claude Code
Resolves #2728
--

UI Preview