Add missing deregister methods to SessionContext#1473
Add missing deregister methods to SessionContext#1473timsaucer wants to merge 4 commits intoapache:mainfrom
Conversation
Expose upstream DataFusion deregister methods (deregister_udf, deregister_udaf, deregister_udwf, deregister_udtf, deregister_object_store) in both the Rust PyO3 bindings and Python wrappers, closing the gap identified in apache#1457. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
There was a problem hiding this comment.
Pull request overview
Exposes additional SessionContext deregistration APIs in the Python bindings to match upstream DataFusion capabilities, and adds tests for UDF/UDAF/UDWF/UDTF deregistration.
Changes:
- Add Python
SessionContextwrappers forderegister_object_store,deregister_udf,deregister_udaf,deregister_udwf, andderegister_udtf. - Add corresponding Rust
PySessionContextbindings for the missing deregister methods. - Add Python unit tests verifying UDF/UDAF/UDWF/UDTF deregistration behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
python/datafusion/context.py |
Adds new public Python API methods on SessionContext to deregister object stores and user-defined functions. |
crates/core/src/context.rs |
Adds PyO3-exposed Rust bindings that call through to DataFusion SessionContext/RuntimeEnv deregistration methods. |
python/tests/test_context.py |
Adds tests covering deregistration of scalar UDFs, aggregate UDAFs, window UDWFs, and table UDTFs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
DataFusion raises ValueError for planning errors when a deregistered function is used in a query. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Url::parse() can fail on invalid input. Use .map_err() to convert the error into a Python exception instead of panicking. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
| """Remove an object store from the session. | ||
|
|
||
| Args: | ||
| schema: The data source schema (e.g. ``"s3://"``). |
There was a problem hiding this comment.
Would be nice to add this (e.g. to the register variant as well.
| """Register a user-defined window function (UDWF) with the context.""" | ||
| self.ctx.register_udwf(udwf._udwf) | ||
|
|
||
| def deregister_udwf(self, name: str) -> None: |
There was a problem hiding this comment.
For all these deregister calls I assume they are for long running sessions and people aren't regularly registering and unregistering in quick succession. If they were then a context manager might be nice to manage scoped lifetimes. Probably out of scope for here or unless someone asks for it.
Which issue does this PR close?
Closes #1457
Rationale for this change
These methods exist in the upstream repository but were not exposed to the Python API.
What changes are included in this PR?
Expose methods to python
Add unit tests
Are there any user-facing changes?
Addition only