feature: Coglet (Rust) HTTP Server Only#2714
Merged
tempusfrangit merged 11 commits intomainfrom Feb 11, 2026
Merged
Conversation
Typically this would be a chore, but now this is a celebration. The http server written in python is no more. Cog now explicitly depends on the Rust based coglet server. 17 files changed, 58 insertions(+), 3305 deletions(-)
Remove all optionality around coglet — it is now always installed: - GetCogletWheelConfig() mirrors GetCogWheelConfig(): env var -> dist/ auto-detect (dev builds) -> PyPI (release builds). Never returns nil. - Dockerfile generator always installs coglet alongside cog. - CI: remove runtime matrix (no more cog vs cog-rust distinction). - Integration tests: remove coglet_rust condition guard. - noxfile.py: always install coglet wheel, trim stale test deps. - pyproject.toml: coglet moved to required dependencies, fastapi and uvicorn removed. - openapi_schema command rewritten to use cog._inspector and cog._schemas directly instead of the deleted FastAPI server. - Docs updated to reflect single Rust/Axum runtime. - .gitattributes: mark llms.txt as linguist-generated.
Flip the LOG_FORMAT default from console to JSON. Production environments expect structured logs. Set LOG_FORMAT=console to get the human-readable format for local development.
Since coglet is now a required dependency, the test-python job needs the coglet wheel artifact. Add build-rust to its dependency chain and download all artifacts with merge-multiple.
…eel configs Wheel auto-detection (env → dist/ → PyPI) was leaking into unit tests, causing failures when local dist/ wheels existed or REPO_ROOT was set. - Add cogWheelConfig/cogletWheelConfig override fields to StandardGenerator - Tests use pypiWheels() helper for deterministic output without env manipulation - Update testInstallCog() to include coglet install lines (coglet is now always installed) - Fix wheels_test.go: isolate from REPO_ROOT and dist/ in dev environment
coglet only has pre-release versions on PyPI (e.g. 0.17.0a4). Without --prerelease=allow, uv cannot resolve coglet>=0.1.0,<1.0. Temporary until coglet 0.17.0 stable is published.
…ibility echo -e behavior differs between macOS and Linux, causing docs:llm:check to fail in CI. printf is portable and produces consistent output.
cog depends on coglet>=0.1.0,<1.0 but coglet only has pre-release versions on PyPI. pip fails resolving cog's dependencies if coglet isn't already installed. Installing coglet first avoids this.
BSD sed (macOS) and GNU sed (Linux) have subtle differences in trailing newline handling with the q command, causing docs:llm:check to fail in CI. awk behavior is consistent across platforms.
docs/ may contain mkdocs-generated copies of CONTRIBUTING.md and README.md that are gitignored. Using git ls-files ensures only tracked markdown files are included in llms.txt generation, preventing CI mismatches between local and CI environments.
092ed2d to
5667cd2
Compare
openapi_schema: use spec_from_file_location to load predictor modules instead of import_module, matching load_predictor_from_ref behavior. This fixes subdirectory predictors (e.g. my-subdir/predict.py:Predictor) where basename stripping lost the directory context. build_pip_freeze: remove pydantic/uvicorn assertions since these are no longer dependencies (they came transitively via fastapi which was removed). Assert coglet instead.
markphelps
added a commit
that referenced
this pull request
Mar 20, 2026
Re-implement the context property on coglet's Scope that was lost when the Python server was replaced by the Rust coglet in #2714. The original feature (#2330) allowed predictors to access per-prediction context via current_scope().context. Thread context (HashMap<String, String>) from the HTTP request body through the full stack: PredictionRequest -> service.predict -> SlotRequest::Predict -> worker -> ScopeGuard -> Scope.context getter. Closes #2852
markphelps
added a commit
that referenced
this pull request
Mar 20, 2026
* feat(coglet): restore Scope.context for per-prediction context Re-implement the context property on coglet's Scope that was lost when the Python server was replaced by the Rust coglet in #2714. The original feature (#2330) allowed predictors to access per-prediction context via current_scope().context. Thread context (HashMap<String, String>) from the HTTP request body through the full stack: PredictionRequest -> service.predict -> SlotRequest::Predict -> worker -> ScopeGuard -> Scope.context getter. Closes #2852 * test: add integration test for Scope.context * fix: correct Python bool casing in scope_context test assertion
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Coglet (Rust/Axum) is now the sole prediction server. The Python FastAPI/Uvicorn server and all supporting modules have been removed (~3300 lines).
notable significant changes: