studio: configurable HuggingFace endpoint via HF_ENDPOINT env var#4989
studio: configurable HuggingFace endpoint via HF_ENDPOINT env var#4989AdamPlatin123 wants to merge 15 commits intounslothai:mainfrom
Conversation
Allow users to set HF_ENDPOINT (e.g. https://hf-mirror.com) to route all HuggingFace API calls through a mirror, solving accessibility issues in regions where huggingface.co is blocked or unreliable. Backend: - Add studio/backend/utils/hf_endpoint.py reading HF_ENDPOINT env var (default: https://huggingface.co) - Replace 5 hardcoded huggingface.co URLs in orchestrator.py, transformers_version.py, model_config.py, huggingface.py - Expose endpoint via /api/health response (hf_endpoint field) Frontend: - Rewrite hf-endpoint.ts to read from the Zustand platform store (populated at startup from /api/health), removing localStorage dependency - Pass hubUrl to @huggingface/hub listModels/listDatasets calls - Replace 4 hardcoded defaults in recipe-studio seed configs with getHfEndpoint() Data flow: env var -> backend -> /api/health -> frontend store -> all HF calls
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Code Review
This pull request centralizes HuggingFace endpoint configuration across the backend and frontend by introducing a new utility that reads from the HF_ENDPOINT environment variable. Hardcoded URLs have been replaced in several modules, including dataset publishing, model fetching, and the health check endpoint. Review feedback indicates that the backend import paths for the new utility are inconsistent with the project's top-level import style and should be corrected to avoid potential runtime errors. Additionally, one import should be moved outside of a loop to improve efficiency.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e61a6abc0
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- Replace `studio.backend.utils.hf_endpoint` with `utils.hf_endpoint` to match backend's sys.path convention (6 files) - Move import out of for-loop in model_config.py and cache endpoint - Normalize empty/whitespace HF_ENDPOINT to default
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 089571ac95
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Review feedback (PR unslothai#4989): a mirror may only proxy Hub endpoints (/api/models, repo files) but not the datasets-server API. Returning the mirror URL for /splits causes subset/split loading to fail. Add HF_DATASETS_SERVER env var support: - Backend: new get_hf_datasets_server() with 3-tier resolution (HF_DATASETS_SERVER > same-as-mirror > official datasets-server) - /api/health: expose hf_datasets_server alongside hf_endpoint - Frontend: store hfDatasetsServer in Zustand, use it directly
…github.com/AdamPlatin123/unsloth into studio-feat-hf-endpoint
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 30c4aa376d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
ik this is still a draft but why do you want to evaluate |
Review feedback (PR unslothai#4989): cachedModelInfo calls for pinned/priority models were still hitting the default huggingface.co endpoint, causing inconsistent behavior when a mirror is configured. Add hubUrl to all remaining cachedModelInfo and listModels call sites.
…n every call Per review feedback from Datta0: evaluate the endpoint once after /api/health returns and cache the result in module-level variables, rather than calling usePlatformStore.getState() on every invocation. The new hydrateHfEndpoints() is called once during platform bootstrap. getHfEndpoint() and getHfDatasetsServerBase() now return the cached values directly.
- hf_endpoint.py: drop unsafe datasets-server fallback — a mirrored HF_ENDPOINT no longer implicitly applies to datasets-server, since most Hub mirrors don't proxy /splits. Operators must set HF_DATASETS_SERVER explicitly. A startup warning flags the case. - instruction-from-answer.json: remove hardcoded "endpoint" so the parser falls back to getHfEndpoint() and respects HF_ENDPOINT. - hf-endpoint.ts: read from the Zustand store directly instead of the module-level cache; removes hydrateHfEndpoints, the circular import with env.ts, and the pre-hydration staleness window. - env.ts: drop the hydrate call and widen the /api/health failure warning to explain that HF_ENDPOINT config will not be applied. - transformers_version.py: distinguish HTTP 404 (DEBUG, legitimate fail-open) from HTTP 5xx and connection errors (WARNING), so mirror misconfigurations surface in the logs instead of disappearing.
|
Pushed commit 40e59750 addressing review feedback. Summary of fixes:
|
Restores the "evaluate once" property requested by @Datta0 without reintroducing the pre-hydration staleness window: getters read a module-level variable, and a Zustand subscribe keeps that variable in sync whenever /api/health populates the platform store. No explicit hydrate step, no cache-vs-store drift, no circular import. Everything stays in-memory — usePlatformStore is a plain (non-persist) store and the module variables live in module scope, so each page load re-fetches /api/health and the backend's live HF_ENDPOINT wins.
|
@Datta0 your concern about evaluating let _endpoint = usePlatformStore.getState().hfEndpoint || DEFAULT_HF_ENDPOINT;
usePlatformStore.subscribe((state) => {
_endpoint = state.hfEndpoint || DEFAULT_HF_ENDPOINT;
});
export function getHfEndpoint(): string {
return _endpoint;
}Getters now read a module-level variable (single property lookup, no Storage-wise everything stays in-memory: |
New test_hf_endpoint.py (modeled on test_pytorch_mirror.py) covers HF_ENDPOINT / HF_DATASETS_SERVER env var resolution: - unset / empty / whitespace fall back to defaults - mirror value used verbatim, trailing slash stripped - get_hf_datasets_server() never silently reuses a mirrored HF_ENDPOINT (regression guard for the unsafe fallback) - startup WARNING fires when a mirror is set without HF_DATASETS_SERVER, stays silent for default / explicit cases Extends test_transformers_version.py with log-severity tests for the new exception branches in _check_tokenizer_config_needs_v5 and _check_config_needs_550: - HTTPError 404 → DEBUG, fail-open (legitimate "file not found") - HTTPError 5xx → WARNING (mirror misconfiguration) - URLError → WARNING (mirror unreachable)









Summary
All HuggingFace API calls in the Studio backend and frontend hardcoded
https://huggingface.co. In regions where huggingface.co isblocked or unreliable (e.g. China), this caused model/dataset search timeouts, empty recommended models, and
ERR_CONNECTION_TIMED_OUTerrors flooding the console.
This PR adds
HF_ENDPOINTenv var support (the same variablehuggingface_hubalready uses for Python downloads). Settingexport HF_ENDPOINT=https://hf-mirror.comroutes all Studio HF calls through the mirror.Changes
Backend (7 files)
studio/backend/utils/hf_endpoint.py— readsHF_ENDPOINTenv var (default:https://huggingface.co)studio/backend/main.py— expose endpoint in/api/healthresponse (hf_endpointfield)studio/backend/core/inference/orchestrator.py— replace hardcoded URLstudio/backend/utils/transformers_version.py— replace hardcoded URLs (2 locations)studio/backend/utils/models/model_config.py— replace hardcoded URLstudio/backend/core/data_recipe/huggingface.py— replace hardcoded URLFrontend (9 files)
studio/frontend/src/lib/hf-endpoint.ts— reads endpoint from Zustand platform storestudio/frontend/src/config/env.ts— populatehfEndpointfrom/api/healthat startupstudio/frontend/src/hooks/use-hf-model-search.ts— passhubUrltolistModels()studio/frontend/src/hooks/use-hf-dataset-search.ts— passhubUrltolistDatasets()studio/frontend/src/hooks/use-hf-dataset-splits.ts— use dynamicgetHfDatasetsServerBase()studio/frontend/src/features/recipe-studio/stores/recipe-studio.ts— usegetHfEndpoint()studio/frontend/src/features/recipe-studio/utils/config-factories.ts— usegetHfEndpoint()studio/frontend/src/features/recipe-studio/utils/import/parsers/seed-config-parser.ts— usegetHfEndpoint()studio/frontend/src/features/recipe-studio/utils/payload/builders-seed.ts— usegetHfEndpoint()Backward compatibility
HF_ENDPOINTset, everything works exactly as before (all URLs resolve tohttps://huggingface.co).huggingface_hublibrary downloads already respectHF_ENDPOINT— no changes needed there.Data flow
HF_ENDPOINTenv var → backend →/api/health→ frontend Zustand store → all HF API callsTest plan
curl /api/healthreturns{"hf_endpoint": "https://huggingface.co"}by defaultHF_ENDPOINT=https://hf-mirror.com→/api/healthreturns mirror URLnpx vite buildpasses