Skip to content

studio: configurable HuggingFace endpoint via HF_ENDPOINT env var#4989

Draft
AdamPlatin123 wants to merge 15 commits intounslothai:mainfrom
AdamPlatin123:studio-feat-hf-endpoint
Draft

studio: configurable HuggingFace endpoint via HF_ENDPOINT env var#4989
AdamPlatin123 wants to merge 15 commits intounslothai:mainfrom
AdamPlatin123:studio-feat-hf-endpoint

Conversation

@AdamPlatin123
Copy link
Copy Markdown
Contributor

Summary

All HuggingFace API calls in the Studio backend and frontend hardcoded https://huggingface.co. In regions where huggingface.co is
blocked or unreliable (e.g. China), this caused model/dataset search timeouts, empty recommended models, and ERR_CONNECTION_TIMED_OUT
errors flooding the console.

This PR adds HF_ENDPOINT env var support (the same variable huggingface_hub already uses for Python downloads). Setting export HF_ENDPOINT=https://hf-mirror.com routes all Studio HF calls through the mirror.

Changes

Backend (7 files)

  • New studio/backend/utils/hf_endpoint.py — reads HF_ENDPOINT env var (default: https://huggingface.co)
  • studio/backend/main.py — expose endpoint in /api/health response (hf_endpoint field)
  • studio/backend/core/inference/orchestrator.py — replace hardcoded URL
  • studio/backend/utils/transformers_version.py — replace hardcoded URLs (2 locations)
  • studio/backend/utils/models/model_config.py — replace hardcoded URL
  • studio/backend/core/data_recipe/huggingface.py — replace hardcoded URL

Frontend (9 files)

  • New studio/frontend/src/lib/hf-endpoint.ts — reads endpoint from Zustand platform store
  • studio/frontend/src/config/env.ts — populate hfEndpoint from /api/health at startup
  • studio/frontend/src/hooks/use-hf-model-search.ts — pass hubUrl to listModels()
  • studio/frontend/src/hooks/use-hf-dataset-search.ts — pass hubUrl to listDatasets()
  • studio/frontend/src/hooks/use-hf-dataset-splits.ts — use dynamic getHfDatasetsServerBase()
  • studio/frontend/src/features/recipe-studio/stores/recipe-studio.ts — use getHfEndpoint()
  • studio/frontend/src/features/recipe-studio/utils/config-factories.ts — use getHfEndpoint()
  • studio/frontend/src/features/recipe-studio/utils/import/parsers/seed-config-parser.ts — use getHfEndpoint()
  • studio/frontend/src/features/recipe-studio/utils/payload/builders-seed.ts — use getHfEndpoint()

Backward compatibility

  • Default behavior unchanged. Without HF_ENDPOINT set, everything works exactly as before (all URLs resolve to
    https://huggingface.co).
  • Python huggingface_hub library downloads already respect HF_ENDPOINT — no changes needed there.

Data flow

HF_ENDPOINT env var → backend → /api/health → frontend Zustand store → all HF API calls

Test plan

  • curl /api/health returns {"hf_endpoint": "https://huggingface.co"} by default
  • HF_ENDPOINT=https://hf-mirror.com/api/health returns mirror URL
  • Model search returns results when mirror is configured
  • npx vite build passes

Allow users to set HF_ENDPOINT (e.g. https://hf-mirror.com) to route all
HuggingFace API calls through a mirror, solving accessibility issues in
regions where huggingface.co is blocked or unreliable.

Backend:
- Add studio/backend/utils/hf_endpoint.py reading HF_ENDPOINT env var
  (default: https://huggingface.co)
- Replace 5 hardcoded huggingface.co URLs in orchestrator.py,
  transformers_version.py, model_config.py, huggingface.py
- Expose endpoint via /api/health response (hf_endpoint field)

Frontend:
- Rewrite hf-endpoint.ts to read from the Zustand platform store
  (populated at startup from /api/health), removing localStorage dependency
- Pass hubUrl to @huggingface/hub listModels/listDatasets calls
- Replace 4 hardcoded defaults in recipe-studio seed configs with
  getHfEndpoint()

Data flow: env var -> backend -> /api/health -> frontend store -> all HF calls
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request centralizes HuggingFace endpoint configuration across the backend and frontend by introducing a new utility that reads from the HF_ENDPOINT environment variable. Hardcoded URLs have been replaced in several modules, including dataset publishing, model fetching, and the health check endpoint. Review feedback indicates that the backend import paths for the new utility are inconsistent with the project's top-level import style and should be corrected to avoid potential runtime errors. Additionally, one import should be moved outside of a loop to improve efficiency.

Comment thread studio/backend/core/data_recipe/huggingface.py Outdated
Comment thread studio/backend/core/inference/orchestrator.py Outdated
Comment thread studio/backend/main.py Outdated
Comment thread studio/backend/utils/models/model_config.py Outdated
Comment thread studio/backend/utils/transformers_version.py Outdated
Comment thread studio/backend/utils/transformers_version.py Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e61a6abc0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/main.py Outdated
Comment thread studio/backend/utils/hf_endpoint.py Outdated
AdamPlatin123 and others added 3 commits April 13, 2026 02:37
- Replace `studio.backend.utils.hf_endpoint` with `utils.hf_endpoint`
  to match backend's sys.path convention (6 files)
- Move import out of for-loop in model_config.py and cache endpoint
- Normalize empty/whitespace HF_ENDPOINT to default
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 089571ac95

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/frontend/src/lib/hf-endpoint.ts Outdated
Review feedback (PR unslothai#4989): a mirror may only proxy Hub endpoints
(/api/models, repo files) but not the datasets-server API.  Returning
the mirror URL for /splits causes subset/split loading to fail.

Add HF_DATASETS_SERVER env var support:
- Backend: new get_hf_datasets_server() with 3-tier resolution
  (HF_DATASETS_SERVER > same-as-mirror > official datasets-server)
- /api/health: expose hf_datasets_server alongside hf_endpoint
- Frontend: store hfDatasetsServer in Zustand, use it directly
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30c4aa376d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/frontend/src/hooks/use-hf-model-search.ts
@rolandtannous rolandtannous marked this pull request as draft April 12, 2026 21:56
@Datta0
Copy link
Copy Markdown
Collaborator

Datta0 commented Apr 13, 2026

ik this is still a draft but why do you want to evaluate getHfEndpoint everytime? Why can't we just do it once and save the result

AdamPlatin123 and others added 3 commits April 13, 2026 15:27
Review feedback (PR unslothai#4989): cachedModelInfo calls for pinned/priority
models were still hitting the default huggingface.co endpoint, causing
inconsistent behavior when a mirror is configured. Add hubUrl to all
remaining cachedModelInfo and listModels call sites.
…n every call

Per review feedback from Datta0: evaluate the endpoint once after
/api/health returns and cache the result in module-level variables,
rather than calling usePlatformStore.getState() on every invocation.

The new hydrateHfEndpoints() is called once during platform bootstrap.
getHfEndpoint() and getHfDatasetsServerBase() now return the cached
values directly.
- hf_endpoint.py: drop unsafe datasets-server fallback — a mirrored
  HF_ENDPOINT no longer implicitly applies to datasets-server, since
  most Hub mirrors don't proxy /splits. Operators must set
  HF_DATASETS_SERVER explicitly. A startup warning flags the case.
- instruction-from-answer.json: remove hardcoded "endpoint" so the
  parser falls back to getHfEndpoint() and respects HF_ENDPOINT.
- hf-endpoint.ts: read from the Zustand store directly instead of the
  module-level cache; removes hydrateHfEndpoints, the circular import
  with env.ts, and the pre-hydration staleness window.
- env.ts: drop the hydrate call and widen the /api/health failure
  warning to explain that HF_ENDPOINT config will not be applied.
- transformers_version.py: distinguish HTTP 404 (DEBUG, legitimate
  fail-open) from HTTP 5xx and connection errors (WARNING), so mirror
  misconfigurations surface in the logs instead of disappearing.
@rolandtannous
Copy link
Copy Markdown
Collaborator

Pushed commit 40e59750 addressing review feedback. Summary of fixes:

hf_endpoint.py — datasets-server fallback: get_hf_datasets_server() no longer returns the Hub mirror URL when HF_DATASETS_SERVER is unset. Most Hub mirrors don't proxy /splits, so the previous fallback turned every dataset-splits lookup into a silent 404. New behavior: return HF_DATASETS_SERVER if set, otherwise the official datasets-server.huggingface.co. Added a startup WARNING when HF_ENDPOINT is mirrored but HF_DATASETS_SERVER is unset, so operators know they need to opt in explicitly.

instruction-from-answer.json — hardcoded endpoint: Removed "endpoint": "https://huggingface.co". parseSeedSettings reads sourceRaw.endpoint and that value was overriding the getHfEndpoint() default in the parser, so a user on a mirror who loaded this template ended up with the wrong endpoint pre-filled in their seed config.

hf-endpoint.ts + env.ts — cache → direct store reads: Removed the module-level cache and hydrateHfEndpoints(); both getters now read usePlatformStore.getState() on every call. This:

  1. Fixes a pre-hydration race where code (e.g. makeSeedConfig, parseSeedSettings) that ran before /api/health resolved would bake the default endpoint into new configs.
  2. Drops the circular import between env.ts and hf-endpoint.ts.
  3. getState() is O(1) — no measurable perf cost compared to the cache.

env.ts/api/health catch branch: Widened the warning message to state explicitly that HF_ENDPOINT configuration will not be applied when the health fetch fails, so mirror-routing issues surface in the console rather than looking like a generic network blip.

transformers_version.py — log severity: _check_tokenizer_config_needs_v5 and _check_config_needs_550 now distinguish HTTPError 404 (DEBUG, legitimate fail-open) from HTTPError 5xx and URLError (WARNING with a hint about HF_ENDPOINT). Previously a mirror returning 500/connection-refused looked identical to a legitimate 404, so mirror problems surfaced as cryptic transformers crashes with no log trail.

rolandtannous and others added 2 commits April 15, 2026 11:54
Restores the "evaluate once" property requested by @Datta0 without
reintroducing the pre-hydration staleness window: getters read a
module-level variable, and a Zustand subscribe keeps that variable
in sync whenever /api/health populates the platform store. No
explicit hydrate step, no cache-vs-store drift, no circular import.

Everything stays in-memory — usePlatformStore is a plain (non-persist)
store and the module variables live in module scope, so each page
load re-fetches /api/health and the backend's live HF_ENDPOINT wins.
@rolandtannous
Copy link
Copy Markdown
Collaborator

@Datta0 your concern about evaluating getHfEndpoint on every call was valid — commit 40e59750 actually undid the caching (aaddd281) that addressed it, which was a mistake on my part. Restored in ae620853:

let _endpoint = usePlatformStore.getState().hfEndpoint || DEFAULT_HF_ENDPOINT;

usePlatformStore.subscribe((state) => {
  _endpoint = state.hfEndpoint || DEFAULT_HF_ENDPOINT;
});

export function getHfEndpoint(): string {
  return _endpoint;
}

Getters now read a module-level variable (single property lookup, no getState() cost), and a Zustand subscribe keeps the cache in sync whenever /api/health populates the platform store. Compared to the original hydrateHfEndpoints() approach this also removes the circular import with env.ts and the pre-hydration window where a stale default could leak into factories.

Storage-wise everything stays in-memory: usePlatformStore is a plain (non-persist) store and the module variables live in module scope, so each page reload re-fetches /api/health and the backend's live HF_ENDPOINT env var wins.

rolandtannous and others added 2 commits April 15, 2026 15:09
New test_hf_endpoint.py (modeled on test_pytorch_mirror.py) covers
HF_ENDPOINT / HF_DATASETS_SERVER env var resolution:
- unset / empty / whitespace fall back to defaults
- mirror value used verbatim, trailing slash stripped
- get_hf_datasets_server() never silently reuses a mirrored
  HF_ENDPOINT (regression guard for the unsafe fallback)
- startup WARNING fires when a mirror is set without
  HF_DATASETS_SERVER, stays silent for default / explicit cases

Extends test_transformers_version.py with log-severity tests for
the new exception branches in _check_tokenizer_config_needs_v5
and _check_config_needs_550:
- HTTPError 404 → DEBUG, fail-open (legitimate "file not found")
- HTTPError 5xx → WARNING (mirror misconfiguration)
- URLError    → WARNING (mirror unreachable)
@rolandtannous
Copy link
Copy Markdown
Collaborator

pytest tests successfully passed
Screenshot 2026-04-15 at 3 52 53 PM

@rolandtannous
Copy link
Copy Markdown
Collaborator

other tests
Default — no env vars (regression baseline)

unset HF_ENDPOINT HF_DATASETS_SERVER
  # start studio as usual, then:
  curl -s http://localhost:<port>/api/health | jq '{hf_endpoint, hf_datasets_server}'
  # expect: { "hf_endpoint": "https://huggingface.co",
  #           "hf_datasets_server": "https://datasets-server.huggingface.co" }
  Then exercise model search, dataset search, dataset splits in the UI — all should work as before.
Screenshot 2026-04-15 at 3 54 31 PM Screenshot 2026-04-15 at 3 57 11 PM Screenshot 2026-04-15 at 3 56 31 PM Screenshot 2026-04-15 at 3 56 14 PM

@rolandtannous
Copy link
Copy Markdown
Collaborator

Other Tests mirror via hf-mirror.com

export HF_ENDPOINT=https://hf-mirror.com                                                        
# start studio
curl -s http://localhost:<port>/api/health | jq '{hf_endpoint, hf_datasets_server}'             
# expect: { "hf_endpoint": "https://hf-mirror.com",                                             #           "hf_datasets_server": "https://datasets-server.huggingface.co" }
Backend logs should contain the new startup WARNING from hf_endpoint.py:
  HF_ENDPOINT is set to https://hf-mirror.com but HF_DATASETS_SERVER is unset; datasets-server c
alls will still go to https://datasets-server.huggingface.co. Set HF_DATASETS_SERVER to
  override.

  In the browser devtools Network tab:
  - Model search → requests hit hf-mirror.com/api/models ✓
  - Dataset search → requests hit hf-mirror.com/api/datasets ✓


  A new seed node created in recipe-studio should pre-fill hf_endpoint: https://hf-mirror.com (n
ot huggingface.co).
Screenshot 2026-04-15 at 4 00 08 PM Screenshot 2026-04-15 at 4 02 45 PM Screenshot 2026-04-15 at 4 01 46 PM Screenshot 2026-04-15 at 4 00 31 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants