Skip to content

feature: Coglet (Rust) HTTP Server Only#2714

Merged
tempusfrangit merged 11 commits intomainfrom
tempusfrangit/coglet-only-server
Feb 11, 2026
Merged

feature: Coglet (Rust) HTTP Server Only#2714
tempusfrangit merged 11 commits intomainfrom
tempusfrangit/coglet-only-server

Conversation

@tempusfrangit
Copy link
Copy Markdown
Contributor

Coglet (Rust/Axum) is now the sole prediction server. The Python FastAPI/Uvicorn server and all supporting modules have been removed (~3300 lines).

notable significant changes:

  • fastapi and uvicorn are no longer dependencies
  • coglet is now a required dependency of cog
  • COGLET_WHEEL no longer needed to opt in — coglet is always installed (dev builds auto-detect dist/, releases use PyPI)
  • LOG_FORMAT defaults to JSON; set LOG_FORMAT=console for human-readable output

Typically this would be a chore, but now this is a celebration. The http
server written in python is no more. Cog now explicitly depends on
the Rust based coglet server.

17 files changed, 58 insertions(+), 3305 deletions(-)
Remove all optionality around coglet — it is now always installed:

- GetCogletWheelConfig() mirrors GetCogWheelConfig(): env var -> dist/
  auto-detect (dev builds) -> PyPI (release builds). Never returns nil.
- Dockerfile generator always installs coglet alongside cog.
- CI: remove runtime matrix (no more cog vs cog-rust distinction).
- Integration tests: remove coglet_rust condition guard.
- noxfile.py: always install coglet wheel, trim stale test deps.
- pyproject.toml: coglet moved to required dependencies, fastapi and
  uvicorn removed.
- openapi_schema command rewritten to use cog._inspector and
  cog._schemas directly instead of the deleted FastAPI server.
- Docs updated to reflect single Rust/Axum runtime.
- .gitattributes: mark llms.txt as linguist-generated.
Flip the LOG_FORMAT default from console to JSON. Production
environments expect structured logs. Set LOG_FORMAT=console
to get the human-readable format for local development.
@tempusfrangit tempusfrangit requested a review from a team as a code owner February 10, 2026 21:45
Since coglet is now a required dependency, the test-python job needs
the coglet wheel artifact. Add build-rust to its dependency chain
and download all artifacts with merge-multiple.
…eel configs

Wheel auto-detection (env → dist/ → PyPI) was leaking into unit tests,
causing failures when local dist/ wheels existed or REPO_ROOT was set.

- Add cogWheelConfig/cogletWheelConfig override fields to StandardGenerator
- Tests use pypiWheels() helper for deterministic output without env manipulation
- Update testInstallCog() to include coglet install lines (coglet is now always installed)
- Fix wheels_test.go: isolate from REPO_ROOT and dist/ in dev environment
coglet only has pre-release versions on PyPI (e.g. 0.17.0a4).
Without --prerelease=allow, uv cannot resolve coglet>=0.1.0,<1.0.
Temporary until coglet 0.17.0 stable is published.
…ibility

echo -e behavior differs between macOS and Linux, causing docs:llm:check
to fail in CI. printf is portable and produces consistent output.
cog depends on coglet>=0.1.0,<1.0 but coglet only has pre-release
versions on PyPI. pip fails resolving cog's dependencies if coglet
isn't already installed. Installing coglet first avoids this.
BSD sed (macOS) and GNU sed (Linux) have subtle differences in
trailing newline handling with the q command, causing docs:llm:check
to fail in CI. awk behavior is consistent across platforms.
docs/ may contain mkdocs-generated copies of CONTRIBUTING.md and
README.md that are gitignored. Using git ls-files ensures only tracked
markdown files are included in llms.txt generation, preventing CI
mismatches between local and CI environments.
@tempusfrangit tempusfrangit force-pushed the tempusfrangit/coglet-only-server branch from 092ed2d to 5667cd2 Compare February 10, 2026 23:14
openapi_schema: use spec_from_file_location to load predictor modules
instead of import_module, matching load_predictor_from_ref behavior.
This fixes subdirectory predictors (e.g. my-subdir/predict.py:Predictor)
where basename stripping lost the directory context.

build_pip_freeze: remove pydantic/uvicorn assertions since these are no
longer dependencies (they came transitively via fastapi which was removed).
Assert coglet instead.
Copy link
Copy Markdown
Contributor

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@tempusfrangit tempusfrangit merged commit 4bc0162 into main Feb 11, 2026
35 checks passed
@tempusfrangit tempusfrangit deleted the tempusfrangit/coglet-only-server branch February 11, 2026 16:25
markphelps added a commit that referenced this pull request Mar 20, 2026
Re-implement the context property on coglet's Scope that was lost when
the Python server was replaced by the Rust coglet in #2714. The original
feature (#2330) allowed predictors to access per-prediction context via
current_scope().context.

Thread context (HashMap<String, String>) from the HTTP request body
through the full stack: PredictionRequest -> service.predict ->
SlotRequest::Predict -> worker -> ScopeGuard -> Scope.context getter.

Closes #2852
markphelps added a commit that referenced this pull request Mar 20, 2026
* feat(coglet): restore Scope.context for per-prediction context

Re-implement the context property on coglet's Scope that was lost when
the Python server was replaced by the Rust coglet in #2714. The original
feature (#2330) allowed predictors to access per-prediction context via
current_scope().context.

Thread context (HashMap<String, String>) from the HTTP request body
through the full stack: PredictionRequest -> service.predict ->
SlotRequest::Predict -> worker -> ScopeGuard -> Scope.context getter.

Closes #2852

* test: add integration test for Scope.context

* fix: correct Python bool casing in scope_context test assertion
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants