WendaoSearch.jl is the Julia package boundary for mixed-relation graph search
enhancements that augment Wendao's graph engine.
The package is not a replacement for Wendao itself.
WendaoArrow.jl is a required direct dependency of this package.
GadgetSearch.jl is now a required direct solver dependency for the Phase 2
adapter seam.
In the current development workspace, WendaoArrow.jl resolves from the
sibling checkout at ../WendaoArrow.jl, while GadgetSearch.jl is resolved
from the pinned GitHub source recorded in Project.toml.
GadgetSearch.jl is intentionally locked by GitHub rev in
Project.toml so the solver adapter does not drift with an unpinned sibling
checkout.
WendaoSearch.jl owns:
- mixed-graph query types
- layered retrieval plan types
- candidate-subgraph structural evidence models
- package-local helpers for structural rerank normalization
WendaoSearch.jl does not yet own:
- promoted Rust-owned Arrow Flight services
- full solver integration across every structural route
- Rust-owned route promotion
- the main Wendao graph store
WendaoSearch.jl now depends directly on WendaoArrow.jl.
WendaoSearch.jl now also depends directly on GadgetSearch.jl, but solver
state stays package-local and does not widen the WendaoArrow.jl transport
boundary.
That rule exists because:
WendaoArrow.jlis the Julia transport and contract substrateWendaoSearch.jlshould not grow a parallel Arrow or Flight boundary- future schema and route promotion should build on one shared Julia substrate
The package is being shaped around three ideas:
- Wendao retrieval operates over a mixed-relation graph
- graph retrieval should be layered rather than flat
- structural solver lanes should evaluate bounded candidate subgraphs instead of the full graph
The reference design documents live under docs/.
The initial package surface focuses on stable bootstrap types:
QueryAnchorQueryEdgeConstraintQueryGraphRetrievalLayerSpecLayeredRetrievalPlanCandidateSubgraphStructuralEvaluation
The first package-local contract slice now also defines:
StructuralRerankRequestStructuralRerankResultConstraintFilterRequestConstraintFilterResultCapabilityManifestRequestCapabilityManifestEntry
The first package-local schema slice now also defines:
ArrowColumnDescriptor- package-local request and response column inventories
- package-local route draft descriptors
- schema version
v0-draft
The package-local row-normalization slice also defines:
- request row helpers for
structural_rerankandconstraint_filter - response row helpers for
structural_rerankandconstraint_filter - schema-ordered columntable helpers for those same draft surfaces
- request and response row helpers for
capability_manifest
The package-local draft payload slice now also defines:
- request and response Arrow-table helpers for
structural_rerank - request and response Arrow-table helpers for
constraint_filter - route-metadata helpers under the package-local
wendao_search.*namespace - request and response Arrow-table helpers for
capability_manifest
Those payload helpers are thin adapters over WendaoArrow.schema_table(...).
WendaoSearch.jl owns the route-specific metadata and draft column shape, but
it no longer owns raw Arrow table materialization.
The package-local draft Flight slice now also defines:
- draft route descriptor helpers for the first three package-local routes
- draft request-header helpers for those same routes
- package-local
x-wendao-search-*request headers layered overWendaoArrow.flight_schema_headers(...)
The package-local draft DoExchange slice now also defines:
- draft exchange-request helpers for
structural_rerank - draft exchange-request helpers for
constraint_filter - draft exchange-request helpers for
capability_manifest - thin route adapters over
WendaoArrow.flight_exchange_request(...)
The package-local typed exchange slice now also defines:
- response decoders for
structural_rerankandconstraint_filter - typed exchange helpers that call the upstream
WendaoArrowsend surface - route-shaped local-service or Flight-client helpers that return
StructuralRerankResultorConstraintFilterResult
The package-local result semantics slice now also defines:
- validation helpers for structural rerank and constraint filter result sets
- deterministic summary helpers over those same typed results
- package-local host-facing result checks without changing the transport layer
The package-local host ingest slice now also defines:
- host-facing ingest envelopes for
structural_rerankandconstraint_filter - exchange-to-ingest helpers layered over the typed exchange surface
- stable candidate-id, top-result, and explanation projections for Wendao host integration
The package-local draft service slice now also defines:
SearchServiceDescriptor- deterministic demo processors for
structural_rerankandconstraint_filter - deterministic demo and contract-probe processors for
capability_manifest build_demo_flight_service(...)for local in-process draft route testingbuild_contract_probe_flight_service(...)for exact-contract live narrowing without demo-processor logicwarm_search_flight_service(...)for startup self-probing before external traffic- a scripts entrypoint under
scripts/run_search_service.jl - a live starter example under
examples/search_service.jl - one startup runtime summary log that reports route count, warmup state, and the dominant memory-footprint factors of the current service mode
- one best-effort gateway Flight sniff that logs whether the service can establish a usable link to the current Wendao gateway
- one parser-summary wrapper entrypoint under
scripts/run_parser_summary_service.jlthat mounts the native Julia and Modelica parser-summary routes over the shared Search Flight service
The package-local capability-manifest demo or probe service now advertises the
canonical provider id xiuxian-wendao-julia, so Rust-side plugin discovery can
validate the live Julia manifest without reinterpreting provider identity in
the host crate.
That startup surface is intentionally broader than "just high-performance
compute". WendaoSearch.jl currently runs as a long-lived Julia Flight
service, so resident memory also reflects:
- Julia runtime and JIT state
- Arrow Flight and gRPC server state
- request and response Arrow conversion buffers
- startup warmup compilation
GadgetSearchandHiGHSstate insolver_demo- multi-route dispatch state when more than one route is enabled
The package also exposes a narrow dependency bridge:
WendaoArrowDependencyStatusarrow_flight_available()wendaoarrow_schema_versions()wendaoarrow_dependency_status()
The first Phase 2 solver slice now also exposes:
GadgetSearchDependencyStatusSolverCandidateEdgeSolverCandidateGraphSolverGadgetResultSolverSearchOutcomegadgetsearch_dependency_status()gadgetsearch_graph6(...)gadgetsearch_default_layout(...)gadgetsearch_loader(...)gadgetsearch_node_index(...)gadgetsearch_pin_indices(...)gadgetsearch_pin_candidates(...)gadgetsearch_simple_graph(...)gadgetsearch_truth_table_constraint(...)gadgetsearch_solve_truth_table(...)
These types are intended to lock the early package vocabulary before the first Arrow contracts and solver adapters land.
The current solver dependency lock is:
https://github.com/isPANN/GadgetSearch.jlrev = "4720cc1453d387404a09113f7c3fa722f6436586"- solver verification currently uses
HiGHSas a test-only optimizer
Current solver status:
- the package can now convert one topology-aware candidate graph into a real
single-candidate
GraphLoader - the package can now normalize node-id based pin candidates into
GadgetSearchpin indices - the package now owns one real
gadgetsearch_solve_truth_table(::Type{<:EnergyModel}, ...)helper for bounded single-candidate solves - a focused
RydbergModelregression now proves the first real solver call on a two-node candidate graph withHiGHS
The package now ships one draft local Flight service entrypoint:
direnv exec . julia scripts/run_search_service.jl \
--route-name structural_rerank \
--host 127.0.0.1 \
--port 41020The same entrypoint can now mount WendaoCodeParser.jl routes behind the same
gRPC listener:
direnv exec . julia scripts/run_search_service.jl \
--route-name capability_manifest \
--code-parser-route-names julia_ast_query \
--mode demo \
--host 127.0.0.1 \
--port 41075Supported route names:
structural_rerankconstraint_filtercapability_manifestrerankfiltercapabilitiesplugin_capabilities
Supported mounted code parser route names:
julia_file_summaryjulia_root_summarymodelica_file_summaryjulia_ast_querymodelica_ast_query
Mounted Modelica runtime note:
- mounted Modelica routes do not only depend on
OMParser.jlbeing built correctly - the current
OMParser.jlnative bridge also expectsAbsyn,ImmutableList, andMetaModelicato be visible fromMainin the live child process WendaoCodeParser.jlnow installs that runtime shim before the first mounted Modelica parse, and the mounted live regression keeps this contract under test- repeated mounted Modelica AST queries now also reuse a bounded
same-source parse-state cache in the parser service process, which removes
repeated
OMParserparse work for unchanged source text
Supported service modes:
democontract_probesolver_demo
Current live solver status:
-
solver_demois now landed for bothstructural_rerankandconstraint_filter -
those live paths now run real bounded
GadgetSearchsolves against explicit request edges carried bycandidate_edge_sources,candidate_edge_destinations, andcandidate_edge_kinds -
demoremains the stable heuristic service mode for existing multi-route graph-structural smoke and plugin discovery flows -
capability_manifeststill uses the current demo or probe processors even when the bounded solver lane is enabled -
plugin-owned
xiuxian-wendao-julialive smoke now proves both explicit transport and manifest-discovery downcalls against solver-backed rerank and filter traffic -
host-through-language-seam
xiuxian-wendaolive smoke now also proves the same manifest-discoverysolver_demodowncalls throughxiuxian_wendao::analyzers::languages -
the staged request contract now carries explicit edge endpoints for both
structural_rerankandconstraint_filter -
plugin-owned generic non-pair topology helpers now stage explicit node and edge arrays for
structural_rerankwithout collapsing them into a two-node pair helper first -
live
solver_demosmoke now also covers one three-node chain through those generic helpers, both in the Julia plugin crate and through the host language seam -
those generic-topology live proofs now also promote one real connected
LinkGraphAgenticExpansionPlanpair collection through the same helper seam, so the three-node chain no longer depends on hand-built node and edge arrays above the service contract -
the Julia-owned generic-topology staging seam now also supports one scored raw pair collection above the pair DTO layer, so candidate-level semantic aggregation no longer has to be hand-coded in the host proofs
-
the Julia-owned generic-topology staging seam now also supports one raw connected-pair collection above that scored pair helper, so host proofs no longer map
LinkGraphAgenticCandidatePairinto scored pair DTOs before live downcall -
the same generic-topology live lane now also proves one multi-candidate batch against
WendaoSearch.jl --mode solver_demo, both in the Julia plugin crate and through the host language seam, without changing the current service contract -
the Rust-side
xiuxian-wendao-juliaprojection seam is now feature-folderized undersrc/plugin/graph_structural_projection/without changing the live contract, anddirenv exec . julia --project=. -e 'using Pkg; Pkg.test()'remains green after that refactor -
the paired Rust exchange proofs now also live in external
#[cfg(test)] #[path = "..."]modules instead of a large inline test block, and the boundedWendaoSearch.jllive regressions remain green after that test-harness refactor -
the same host-through-language-seam live lane now also proves one seed-centered generic-topology batch derived from a real
LinkGraphAgenticExpansionPlan, so the currentsolver_demoroute now covers one more realistic mixed-graph candidate batch above connected-pair collections without changing the service contract -
that same host-through-language-seam live lane now also proves one worker-partition generic-topology batch derived from real
LinkGraphAgenticWorkerPlanpartitions, so the currentsolver_demoroute now covers one more planner-shaped candidate batch above seed-centered groups while accepting mixed feasible and infeasible solver rows inside the same returned batch -
that same host-through-language-seam live lane now also derives one batch-level generic-topology query context from the real expansion-plan query plus selected worker seed metadata, so the current
solver_demoroute is now proven above a less synthetic host query-context seam -
that same host-through-language-seam live lane now also derives worker-batch dependency, keyword, and tag scores from real plan-aware batch semantics and validates those staged request-batch columns before the live downcall, so the current
solver_demoroute is now proven above a less synthetic host score-staging seam -
that same host-through-language-seam live lane now also validates the staged
semantic_scorerequest column derived from real worker-partition pair semantics before the live downcall, so the currentsolver_demoroute is now proven above a less implicit host-to-Julia semantic-score handoff -
that same host-through-language-seam live lane now also validates the staged
query_id,retrieval_layer,query_max_layers,anchor_planes,anchor_values, andedge_constraint_kindsrequest columns against the same plan-aware batch fixture before the live downcall, so the currentsolver_demoroute is now proven above a less implicit host-to-Julia query-context handoff -
that same host-through-language-seam live lane now also validates the staged
candidate_node_ids,candidate_edge_sources,candidate_edge_destinations, andcandidate_edge_kindsrequest columns against the same plan-aware batch fixture before the live downcall, so the currentsolver_demoroute is now proven above a less implicit host-to-Julia topology handoff -
that same host-through-language-seam live lane now also proves one plan-aware worker-partition generic-topology
constraint_filterbatch above the same raw connected-pair collection seam, and it now validates the stagedconstraint_kindandrequired_boundary_sizerequest columns before reusing that batch against the manifest-discoveredWendaoSearch.jl --mode solver_demofilter route -
the paired Julia-plugin live lane now also proves one multi-candidate generic-topology
constraint_filterbatch against that same manifest- discoveredWendaoSearch.jl --mode solver_demomulti-route endpoint, so the current filter route is now covered for both single-candidate and multi-candidate generic-topology downcalls -
the paired host-through-language-seam live lane now also derives generic- topology fallback edge labels and staged
edge_constraint_kindsfrom the normalized Wendao agentic execution relation, so the current manifest-discoveredsolver_demodowncall no longer relies on a host-local placeholder"related"edge label -
that same host-through-language-seam filter lane now also derives the staged
required_boundary_sizefrom plan-aware anchor and candidate- topology semantics, and it validates filter-side anchor and topology list columns before reusing the same batch against the current manifest-discoveredsolver_demoroute -
that same host-through-language-seam filter lane now also derives the staged
constraint_kindfrom the same plan-aware batch shape, and the paired Julia-plugin live proof now exercises the non-defaultboundary_matchfilter mode against the current solver-demo multi-route endpoint Current structural request topology fields: -
candidate_node_ids -
candidate_edge_sources -
candidate_edge_destinations -
candidate_edge_kinds
This entrypoint keeps transport on WendaoArrow.jl while letting
WendaoSearch.jl expose one package-local draft service for local loopback
and gateway smoke work.
The same entrypoint now also accepts a comma-separated multi-route form:
direnv exec . julia scripts/run_search_service.jl \
--route-names capability_manifest,structural_rerank,constraint_filter \
--mode demo \
--host 127.0.0.1 \
--port 41070That live service now advertises /plugin/capabilities and serves the
graph-structural routes on the same base URL.
Current status:
- package tests cover the new demo service helpers
- package tests now also cover the new exact-contract
contract_probeservice helpers - package tests now also cover
warm_search_flight_service(...)for contract-probe and demo startup paths - live narrowing showed the first external request after startup could fail even when the same request succeeded on the second attempt
- this first-hit failure was not specific to the structural response columns
or
v0-draft; it was a startup warmup seam on the Search service path examples/search_service.jlnow performs an in-process self-probe before binding the Flight listener- a real native Julia first-hit probe against
scripts/run_search_service.jl --mode contract_probe --probe-profile id_plus_floatnow succeeds and returns(:candidate_id, :structural_score) - the draft live entrypoint is now usable for local structural-rerank loopback
smoke work over the package-local
v0-draftcontract - package tests now launch
scripts/run_search_service.jlas a child process - the package now also exposes
/plugin/capabilitiesas a package-local draft Flight route for Julia-side capability discovery - package tests now cover in-process and cross-process capability-manifest service paths
- the package now also supports one same-port multi-route live service for capability manifest, structural rerank, and constraint filter
- package tests now cover that same-port multi-route service through a real child-process live regression
- the cross-process regression harness runs the child from the repository
root and restores
JULIA_LOAD_PATH=@:@stdlib, soPkg.test()now matches the manual live startup path instead of failing beforeusing Pkg - the multi-route implementation stays Search-owned by dispatching within one
Flight
DoExchangeservice on request descriptor path, instead of trying to register multipleFlightServiceinstances on the same gRPC server WendaoCodeParser.jlis now a direct sibling dependency so Search can mount native Julia and Modelica parser routes into the existing live gRPC service without introducing a second process manager- mounted parser routes stay opt-in through
--code-parser-route-nameor--code-parser-route-names, so the default Search-only service shape is unchanged - package tests now cover one same-port live regression that serves
capability_manifestandjulia_ast_queryfrom the same child process
The package test tree is now feature-folderized so test/runtests.jl remains
an entrypoint only.
Current grouped test surfaces:
test/support/for shared live-service helperstest/solver/forGadgetSearch.jladapter coveragetest/core/for query, contract, and payload model coveragetest/exchange/for Flight request, typed exchange, and ingest coveragetest/service/for descriptor, demo, observability, and contract-probe coveragetest/integration/for warmup and cross-process live service coveragetest/bridge/for narrow dependency-status coverage
When adding new coverage, prefer placing it under the closest feature folder
and keep test/runtests.jl as a thin include-only interface.
direnv exec . julia --project=. -e 'using Pkg; Pkg.test()'direnv exec . julia scripts/run_search_service.jl \
--route-name structural_rerank \
--mode contract_probe \
--probe-profile id_plus_float \
--host 127.0.0.1 \
--port 41060direnv exec . julia scripts/run_search_service.jl \
--route-names capability_manifest,structural_rerank,constraint_filter \
--mode demo \
--host 127.0.0.1 \
--port 41070direnv exec . julia scripts/run_search_service.jl \
--route-name capability_manifest \
--code-parser-route-names julia_ast_query \
--mode demo \
--host 127.0.0.1 \
--port 41075direnv exec . julia --project=. scripts/probe_search_service.jl \
--route-name structural_rerank \
--host 127.0.0.1 \
--port 41060Active live listener and transport validation now belongs to the Python client
lanes owned by arrow-julia and the workspace Python transport packages. The
package-local Julia probe script above remains a compatibility check for
runtime flows that still build request tables in Julia while delegating Flight
transport to pyarrow.flight. Set WENDAOSEARCH_PYTHON=python when your
environment already has pyarrow, or let the default bridge use
uv run --with pyarrow python when uv is available.
WendaoSearch.jl now owns one formal parser-summary live-service descriptor at:
config/live/parser_summary.toml
That file is the canonical mounted parser-summary service contract for Rust
repo-intelligence and gateway code_search or code_ast proofs. The
repository devenv process layer is only the launcher for that package-owned
config; it does not define Search route, mode, host, or port semantics itself.
When a fresh workspace is missing the package checkout under .data/,
the shared launcher may clone the configured WendaoSearch.jl repo before
startup. That bootstrap is intentionally one-way and bounded: existing git
checkouts stay untouched, and any existing non-git directory at the package
path is treated as a hard error instead of being overwritten.
Start the canonical process-managed parser-summary service:
cd .. && direnv exec . devenv processes up -d wendaosearch-parser-summaryThe startup log now emits one service-footprint line like
WendaoSearch service startup ... memory_factors=[...]. When gateway sniffing
is enabled, it also emits either:
WendaoSearch gateway link establishedWendaoSearch gateway sniff failedWendaoSearch gateway sniff skipped
The current sniff is a bounded TCP transport reachability check against the
configured Wendao gateway Flight port. It runs after the local listener bind
succeeds, so a successful link log no longer appears before local listener
startup. Pass --no-gateway-sniff or set [gateway_sniff].enabled = false in
the service config when you want isolated package-local startup without an
external gateway transport check.
The canonical parser-summary managed service mirrors stdout and stderr into:
.run/logs/wendaosearch-parser-summary.stdout.log
.run/logs/wendaosearch-parser-summary.stderr.log
The package also still owns one repo-local multi-route demo descriptor at:
config/live/solver_demo.toml
That config remains useful for broad multi-route demos and benchmark scripts, but it is no longer the formal managed startup surface for Rust gateway parser integration.
Start the parser-summary service directly through the package-owned wrapper:
direnv exec . julia --project=. scripts/run_parser_summary_service.jlOverride the bind port without changing the package-owned route contract:
direnv exec . julia --project=. scripts/run_parser_summary_service.jl \
--host 127.0.0.1 \
--port 41081If those files do not appear after startup, the usual cause is a stale
process-compose generation that predates the current log-sink launcher. In
that case, restart the managed service from the current devenv generation
instead of trusting an inherited long-lived shell session.
Wait for the canonical parser-summary service readiness:
cd .. && direnv exec . bash -lc 'for _ in $(seq 1 60); do if bash scripts/channel/wendaosearch-healthcheck.sh >/dev/null 2>&1; then echo ready; exit 0; fi; sleep 2; done; echo not-ready >&2; exit 1'Start the optional process-managed solver-demo service:
cd .. && direnv exec . devenv processes up -d wendaosearch-solver-demoWait for solver-demo readiness when you intentionally need the multi-route demo surface:
cd .. && direnv exec . bash -lc 'export WENDAOSEARCH_SERVICE_NAME=wendaosearch-solver-demo; export WENDAOSEARCH_RUNTIME_DIR=.run/wendaosearch; export WENDAOSEARCH_CONFIG=.data/WendaoSearch.jl/config/live/solver_demo.toml; for _ in $(seq 1 60); do if bash scripts/channel/wendaosearch-healthcheck.sh >/dev/null 2>&1; then echo ready; exit 0; fi; sleep 2; done; echo not-ready >&2; exit 1'Probe the live service through the package-owned probe script:
direnv exec . julia --project=. \
scripts/probe_search_service.jl \
--config config/live/solver_demo.toml \
--route-name structural_rerankRun one opt-in broad-query live performance probe against the current gateway
and the local solver_demo service:
direnv exec . julia --project=. \
scripts/benchmark_search_service.jl \
--config config/live/solver_demo.toml \
--query topology \
--knowledge-limit 10 \
--fixture-limit 3 \
--samples 3The performance summary always records the actual fixture source:
fixture_source=graph_neighborsmeans the broad query produced graph-resolvable live nodes and the benchmark used real graph-neighbor topology.fixture_source=knowledge_path_chainmeans the broad query produced real knowledge-search hits but the current gateway did not expose those hit paths as graph nodes, so the benchmark fell back to one explicit path-chain workload derived from the returned live search hits.
The current measured run is documented in
docs/04_chronicles/research/2026-04-07-live-gateway-performance-report.md.
Run one mounted native Arrow Flight AST code search benchmark against the shared Search-plus-parser service:
direnv exec . julia --project=. \
scripts/benchmark_ast_code_search.jl \
--host 127.0.0.1 \
--port 49872 \
--route-names julia_ast_query,modelica_ast_query \
--fixture-scale 32 \
--samples 10 \
--node-kind function \
--name-contains foo \
--limit 5The current measured AST run is documented in
docs/04_chronicles/research/2026-04-08-native-arrow-flight-ast-code-search-performance-report.md.
The current post-cache-fix confirmation run recorded:
route=julia_ast_query backend=JuliaSyntax.jl source_id=FlightAstBenchmark.jl source_bytes=1705 query_fields=3 rows=5 matches=5 samples=20 median_ms=3.866 min_ms=3.519 max_ms=12.908 median_alloc_bytes=168064
route=modelica_ast_query backend=OMParser.jl source_id=FlightAstBenchmark.mo source_bytes=1810 query_fields=3 rows=5 matches=5 samples=20 median_ms=3.439 min_ms=3.0 max_ms=19.933 median_alloc_bytes=167552
route=modelica_ast_query backend=OMParser.jl source_id=FlightAstBenchmark.mo source_bytes=1810 query_fields=3 rows=5 matches=5 samples=50 median_ms=3.261 min_ms=2.899 max_ms=4.515 median_alloc_bytes=167552
Run the opt-in package regression that exercises the same live perf path:
direnv exec . env \
WENDAOSEARCH_RUN_LIVE_GATEWAY_PERF=1 \
WENDAOSEARCH_LIVE_GATEWAY_QUERY=topology \
WENDAOSEARCH_LIVE_GATEWAY_SEARCH_LIMIT=10 \
WENDAOSEARCH_LIVE_GATEWAY_FIXTURE_LIMIT=3 \
WENDAOSEARCH_LIVE_GATEWAY_SAMPLES=3 \
julia --project=. -e 'using Pkg; Pkg.test()'Run the opt-in Rust consumer live proof against the managed background service:
cd .. && direnv exec . bash -lc 'RUN_PROCESS_MANAGED_WENDAOSEARCH_TEST=1 cargo test -p xiuxian-wendao-julia plugin::graph_structural_exchange::tests::fetch_graph_structural_solver_demo_rows_for_repository_against_process_managed_wendaosearch_service -- --exact --nocapture'Stop the background service:
direnv exec . devenv processes downThe Rust plugin and host test suites still self-spawn their own Julia services for bounded live proofs. That keeps isolated ports, isolated lifecycles, and deterministic cleanup in test scope while the repo-level process-managed service remains available for operator smoke, cwd-independent process-managed launch validation, and one explicit background-service Rust live proof.