Skip to content

Restructure coglet Python module to improve separation of structure and prepare for publishing#2710

Merged
tempusfrangit merged 13 commits intomainfrom
fix/coglet-publishing
Feb 10, 2026
Merged

Restructure coglet Python module to improve separation of structure and prepare for publishing#2710
tempusfrangit merged 13 commits intomainfrom
fix/coglet-publishing

Conversation

@tempusfrangit
Copy link
Copy Markdown
Contributor

This PR restructures the coglet Python module to provide a clean public API. Replaces the flat function-based module with frozen pyclasses and submodules, fixes PEP 440 version compliance, rewrites Python string exec blocks in Rust, and sets up a proper mixed-layout build with two-stage stub generation.
Module layout (before -> after)

Before:

  • coglet.serve(...) -- top-level function
  • coglet.active() -- required parentheses
  • coglet._run_worker() / coglet._is_cancelable() -- top-level
  • coglet.SlotLogWriter / coglet.TeeWriter -- publicly visible internals
  • coglet.coglet -- maturin double-export artifact in dir()

After:

  • coglet.__version__ -- PEP 440 compliant
  • coglet.__build__ -- frozen BuildInfo with git sha, build time, rustc version
  • coglet.server.serve(...) -- method on frozen Server pyclass
  • coglet.server.active -- bool property, no parens
  • coglet.server._run_worker() / coglet.server._is_cancelable() -- internal methods
  • coglet._sdk._SlotLogWriter / coglet._sdk._TeeWriter -- submodule for runtime integration

Key changes

  • build.rs: PEP 440 version conversion at compile time, BuildInfo frozen pyclass with build metadata
  • audit.rs: Rewrote 60-line Python string audit hook in pure Rust with AtomicBool re-entrancy guard and Mutex for thread safety
  • cancel.rs: CancelationException and signal handler moved to OnceLock statics and native pyfunction
  • predictor.rs: Async Python helpers moved to OnceLock statics (init once, reuse)
  • lib.rs: CogletServer frozen pyclass, _sdk submodule via add_submodule(), all for export control
  • Mixed layout: Hand-managed coglet/__init__.py with module-name = "coglet._impl" to avoid maturin's auto-generated wrapper
  • Stub pipeline: pyo3-stub-gen for Rust types, then mypy stubgen (via uvx) for Python re-exports
  • All callsites updated: cog SDK, orchestrator, tests, docs

Testing

17 new unit tests for module structure (version format, BuildInfo, frozen protection, server.active, _sdk contents, all). All existing integration tests updated and passing.

Add build script that:
- Converts semver pre-release tags to PEP 440 format
  (e.g., 0.17.0-alpha.2 → 0.17.0a2)
- Captures git SHA, build timestamp, and rustc version
- Emits as compile-time environment variables

Add frozen BuildInfo pyclass for coglet.__build__ with version,
git_sha, build_time, and rustc_version fields.
…ignal handler

Replace module setattr + Python string exec with:
- OnceLock<Py<PyAny>> static for the exception class
- Rust #[pyfunction] signal handler

Eliminates a setattr on the coglet module (needed for upcoming
module write protection) and removes Python-as-string.
…g target

audit.rs:
- Replace 60-line Python string with Rust #[pyfunction] audit hook
- Add AtomicBool re-entrancy guard (replaces threading.Timer hack)
- Add Mutex to serialize stream replacement across threads
- Demote helper functions from #[pyfunction] to pub(crate)
- Rename TeeWriter -> _TeeWriter (private pyclass)

worker_tracing_layer.rs:
- Add coglet::worker_local target filter (excluded from IPC,
  always written to worker stderr fd 101)
- Used by audit.rs for poisoned-lock diagnostics that must not
  traverse the log routing infrastructure
Make the log writer class private in the Python namespace via
pyclass(name = "_SlotLogWriter"). Rust type name unchanged.
…edictor

Move _collect_async_gen and _ctx_wrapper Python async helper functions
into OnceLock statics so they are defined once and reused across calls.
Fix Py<PyAny>::call1() call sites to pass py token as first argument.
…tr__

- Remove standalone active() #[pyfunction], expose via __getattr__ as
  property-like attribute (no parens needed)
- Add __getattr__ for 'active' and '__build__' attribute access
- Add __dir__ returning only public API: __version__, __build__, active, serve
- Add __setattr__ to block writes to module (registered LAST in init)
- Use PEP 440 version from COGLET_PEP440_VERSION env var
- Store BuildInfo as _build_info, exposed as __build__ via __getattr__
- coglet.server: frozen Server pyclass with .active property, .serve(),
  ._run_worker(), ._is_cancelable()
- coglet._sdk: submodule for Python runtime integration classes
  (_SlotLogWriter, _TeeWriter)
- Root module is plain: __version__, __build__, server, _sdk
- No PEP 562 machinery needed — static attrs in dict, dynamic .active
  is a #[getter] on frozen pyclass
- Update all callsites: http.py, orchestrator.rs, tests, docs
…__.py

- coglet/__init__.py: explicit re-exports with __all__ control
- coglet/py.typed: marker for type stub distribution
- pyproject.toml: module-name = coglet._impl for mixed layout
- lib.rs: #[pyo3(name = "_impl")] + __all__ for import control
- .gitignore: add *.so, *.pyd to prevent committing build artifacts
- Maturin now builds as mixed python/rust project, no auto-generated wrapper
- Remove stale root coglet.pyi, replace with coglet/__init__.pyi (mypy
  stubgen) and coglet/_sdk/__init__.pyi (pyo3-stub-gen)
- pyproject.toml: add python-source = '.' for pyo3-stub-gen mixed layout
- mise.toml: update generate:stubs to run pyo3-stub-gen then mypy stubgen
- mise.toml: rewrite stub:check to use git diff on all .pyi files
- mise.toml: update stub:typecheck path
Tests for: PEP 440 version, BuildInfo fields/frozen/repr,
server.active property, server frozen protection, _sdk submodule
contents, __all__ excludes internals. 17 new tests, all passing.
@tempusfrangit tempusfrangit requested a review from a team as a code owner February 10, 2026 08:04
@tempusfrangit tempusfrangit changed the title Restructure coglet Python module for 0.17.0 release Restructure coglet Python module to improve separation of structure and prepare for publishing Feb 10, 2026
@tempusfrangit
Copy link
Copy Markdown
Contributor Author

tempusfrangit commented Feb 10, 2026

Go lint is toast please review as go lint is fixed in #2711 and we'll rebase once that lands.

Ignore the transient gotestyourself mise error as well. I'm sure it'll resolve or we'll figure out another path independent of this PR.

Comment thread crates/coglet-python/src/lib.rs Outdated
Comment on lines +452 to +460
/// Module `__setattr__` — blocks writes to the module to prevent accidental mutation.
///
/// Registered LAST in module init so it doesn't prevent our own setup.
#[pyfunction]
fn __setattr__(name: &str, _value: &Bound<'_, PyAny>) -> PyResult<()> {
Err(pyo3::exceptions::PyAttributeError::new_err(format!(
"cannot set '{name}' on module 'coglet'"
)))
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the pep for this was rejected, restructured in a later commit out.

Copy link
Copy Markdown
Member

@michaeldwan michaeldwan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tempusfrangit walked us through live. looks sensible.

@tempusfrangit tempusfrangit merged commit 818e74a into main Feb 10, 2026
70 of 72 checks passed
@tempusfrangit tempusfrangit deleted the fix/coglet-publishing branch February 10, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants