Skip to content

Add option to fully disable registry cache#3

Closed
jfw-ppi wants to merge 1 commit intomasterfrom
codex/add-option-to-disable-registry-cache
Closed

Add option to fully disable registry cache#3
jfw-ppi wants to merge 1 commit intomasterfrom
codex/add-option-to-disable-registry-cache

Conversation

@jfw-ppi
Copy link
Owner

@jfw-ppi jfw-ppi commented Dec 28, 2025

Motivation

  • Provide a way to completely disable the registry cache so registry state is fetched directly from the registry store and the cache is not initialized at startup.
  • Support use-cases where teams want to avoid memory cache, initial load latency, or ensure every access hits the authoritative store.

Description

  • Add a cache_enabled flag to RegistryConfig and default it to True so existing behavior remains unchanged while allowing disabling the cache via config.
  • Propagate the flag into Registry, CachingRegistry, SqlRegistry, SnowflakeRegistry, and RemoteRegistry, add _should_use_cache helpers, and short-circuit cache initialization and refresh logic when disabled.
  • Ensure remote registry RPCs respect the flag by mapping allow_cache to self._allow_cache(allow_cache) in RemoteRegistry and return direct store results when caching is disabled.
  • Guard FeatureStore refresh behavior and the transformation server async refresh to skip cache refresh when cache_enabled is False.

Testing

  • No automated tests were executed as part of this change.
  • Developers should run make test-python-unit and make test-python-integration to validate behavior in their environments after enabling or disabling the flag.

Codex Task

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +258 to +262
if self.cache_enabled:
try:
registry_proto = self._registry_store.get_registry_proto()
self.cached_registry_proto = registry_proto
self.cached_registry_proto_created = _utc_now()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Load registry before apply_project when cache is disabled

When cache_enabled is false, Registry.__init__ no longer loads the registry proto from the store, so cached_registry_proto stays as the empty default. apply_project (in registry.py, lines 1128–1146) mutates cached_registry_proto directly without calling _prepare_registry_for_changes, so if apply_project is invoked before any other write that loads the store, the subsequent commit() will overwrite the registry store with an almost-empty proto containing only the new project, dropping existing entities/feature views. This regression only occurs when cache_enabled is disabled and a registry already has data.

Useful? React with 👍 / 👎.

@jfw-ppi jfw-ppi closed this Dec 28, 2025
@jfw-ppi jfw-ppi deleted the codex/add-option-to-disable-registry-cache branch December 28, 2025 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant