feat: Agnostic Pipeline Kinds & Internal Runner Refactor#392
feat: Agnostic Pipeline Kinds & Internal Runner Refactor#392pipewrk merged 6 commits intowpkernel:mainfrom
Conversation
… kinds and refactor internal runner implementation.
There was a problem hiding this comment.
💡 Codex Review
wpkernel/packages/pipeline/src/internal/runner/program.ts
Lines 392 to 396 in 44cfc39
When multiple extension lifecycles are configured (see lifecycles array built at lines 577‑579), each call to executeLifecycle overwrites state.extensionCoordinator/extensionState with the current lifecycle’s data (lines 392‑396), but the commit stage later operates on a single extensionState (312‑319). As a result, if you enable more than one lifecycle (e.g., ['prepare','after-fragments']), hooks executed in earlier lifecycles never have their commit/rollback callbacks invoked, leaving their side-effects uncommitted and unrolled. This breaks multi-lifecycle extensions even though this commit advertises supporting them.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…k and sequential commit.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This PR introduces the
makePipeline API, enabling fully agnostic helper kinds (beyond just
fragmentandbuilder). It also refactors the monolithicpipeline-runner.ts into a modular architecture (
context,execution,program) and implements synchronous execution optimizations.Key Changes
1. New
makePipeline API
Agnostic Kinds: Replaced hardcoded
fragment/builderlogic with a generic registry. Pipelines can now define arbitrary stages (e.g., ETL:extract,transform,load).Generic Registration: Updated
Pipeline.use()to accept any validHelper type.
Type Safety: Strictly typed generic implementation without
anycasts.2. Runner Refactor
Modularization: Split
pipeline-runner.ts into:
Sync Optimization: Implemented
maybeThen /
maybeTry patterns to ensure synchronous pipelines execute without Promise overheads.
3. Features
Helper Overrides: Implemented
overridemode support to replace existing helpers properly.Eager Validation: Dependency graphs are now validated eagerly during
prepareContext.
Verification
Automated Tests
New ETL Test Case:
src/tests/agnostic-kinds.test.ts validates custom
extract,transform,loadstages.Sync Execution:
src/tests/sync-execution.test.ts proves that purely sync pipelines return values synchronously.
Regression: Passed full
pnpm testsuite and repo-wide type check.