fix(pgwire): wire quantization params through CREATE/ALTER VECTOR INDEX DDL#27
Merged
farhan-syah merged 4 commits intomainfrom Apr 14, 2026
Merged
Conversation
jieba-rs pulls include-flate → libflate → core2 0.4.0, which is yanked on crates.io. The workspace [patch.crates-io] bypassed this locally but was ignored by cargo publish's isolated packaging resolve, so releases kept failing on nodedb-fts. Drop jieba-rs entirely and let lang-zh fall back to CJK bigram segmentation (the same path used when the feature is disabled). The feature flag is retained as a no-op so downstream configs keep compiling. Can be restored when the upstream flate chain moves off yanked core2.
dsl.rs has grown to cover DDL parsing for search indexes, CRDT merge, FTS, vector, sparse, and fusion in a single flat file. Split it into a dsl/ directory with one module per concern: crdt_merge, fulltext_index, search_fusion, search_index, search_vector, sparse_index, vector_index, helpers, mod No logic changes — pure file reorganisation.
ALTER VECTOR INDEX SET now accepts index_type, pq_m, ivf_cells, and ivf_nprobe in addition to the existing m, m0, and ef_construction keys. Quantization-shape parameters (index_type, pq_m, ivf_cells, ivf_nprobe) dispatch VectorOp::SetParams to update the stored IndexConfig before the index materializes. HNSW structure parameters (m, m0, ef_construction) dispatch VectorOp::Rebuild as before. Both groups may appear in a single ALTER and are dispatched independently; omitted fields preserve current values. The executor handler (execute_set_vector_params) is wired alongside the existing Rebuild path, and integration tests cover HNSW-only, PQ-only, IVF-PQ, and combined parameter updates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #25
Summary
CREATE VECTOR INDEXnow parsesINDEX_TYPE,PQ_M,IVF_CELLS,IVF_NPROBEand propagates them intoVectorOp::SetParamsinstead of the hardcoded zeros that forced every SQL-created index to full-FP32 HNSW.INDEX_TYPEvalues and invalid combinations (PQ_Mnot dividingDIM, quantization params without a PQ index type,ivf_nprobe > ivf_cells) are now rejected at the DDL layer instead of silently falling through.ALTER VECTOR INDEX ... SET (...)accepts the same quantization keyword set; quantization-shape changes route throughSetParams, HNSW parameter changes route throughRebuild, and a single statement may mix both.execute_set_vector_paramsnow reads the existingIndexConfigand treats zero / empty inputs as "preserve", so quantization-only ALTERs no longer clobberm/ef_construction/metricset at CREATE time.pgwire/ddl/dsl.rs(594 lines, over the 500-line hard limit and on the fix's code path) split into adsl/directory with one handler per file so the fix could be added without pushing the file further over the limit.Test plan
nodedb/tests/sql_vector_index_ddl.rs— 5 tests covering: unknownINDEX_TYPErejection,PQ_M/DIMdivisibility validation, validhnsw_pqacceptance, validivf_pqacceptance, andALTER VECTOR INDEX SET (index_type = ...)acceptance.cargo nextest run -p nodedb --test sql_vector_index_ddl— 5/5 green.sql_index_naming,sql_transactions,sql_where_expressions,sql_update_expressions) — 28/28 green.cargo fmt --allclean,cargo clippy -p nodedb --tests -- -D warningsclean.