Add functional test for IPC interface#33201
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33201. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
|
Concept ACK. 27bd67857dd0e145d7539b949df2b7aaa53c6b1b is really nice and simpler than I would have expected. In case it's useful #30437 adds an interface_ipc_mining.py test, and #32297 adds an interface_ipc_cli.py test which are similar to this test and have similar test framework updates. Code from those PRs might be useful here or vice versa. |
b4cb8f6 to
b87de46
Compare
|
Concept ACK So far I achieved test coverage for the Mining interface with a combination of unit tests and by having RPC methods such as CentOS CI fails with |
e6bfad6 to
f5b8da2
Compare
74fc236 to
eb482c1
Compare
There was a problem hiding this comment.
On macOS 15.6 with capnp 1.2.0 installed via Homebrew and Python 3.10.14 installed via PyEnv I get:
pip3 install pycapnp
...
Successfully installed pycapnp-2.0.0
cmake -B build
cmake --build build
...
% build/test/functional/interface_ipc.py
2025-08-17T07:05:49.321522Z TestFramework (INFO): PRNG seed is: 6039316184654231320
2025-08-17T07:05:49.322566Z TestFramework (INFO): Initializing test directory /var/folders/8p/qqdl2fsj7rd6q096jtmsfydc0000gn/T/bitcoin_func_test_z3jtr5u1
2025-08-17T07:05:50.595292Z TestFramework (ERROR): Unexpected exception
Traceback (most recent call last):
File "/Users/sjors/dev/bitcoin/test/functional/test_framework/test_framework.py", line 200, in main
self.run_test()
File "/Users/sjors/dev/bitcoin/build/test/functional/interface_ipc.py", line 82, in run_test
self.run_echo_test()
File "/Users/sjors/dev/bitcoin/build/test/functional/interface_ipc.py", line 63, in run_echo_test
asyncio.run(capnp.run(async_routine()))
File "/Users/sjors/.pyenv/versions/3.10.14/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/Users/sjors/.pyenv/versions/3.10.14/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "capnp/lib/capnp.pyx", line 1944, in run
File "capnp/lib/capnp.pyx", line 1945, in capnp.lib.capnp.run
File "/Users/sjors/dev/bitcoin/build/test/functional/interface_ipc.py", line 57, in async_routine
init, ctx = await self.make_capnp_init_ctx()
File "/Users/sjors/dev/bitcoin/build/test/functional/interface_ipc.py", line 45, in make_capnp_init_ctx
modules = self.capnp_modules()
File "/Users/sjors/dev/bitcoin/build/test/functional/interface_ipc.py", line 38, in capnp_modules
"proxy": capnp.load(str(mp_dir / "mp" / "proxy.capnp"), imports=imports),
File "capnp/lib/capnp.pyx", line 4385, in capnp.lib.capnp.load
File "capnp/lib/capnp.pyx", line 3574, in capnp.lib.capnp.SchemaParser.load
capnp.lib.capnp.KjException: mp/proxy.capnp:6: failed: Import failed: /capnp/c++.capnp
Maybe related: capnproto/pycapnp#377 (though pycapnp-1.3.0 doesn't work at all, since it doesn't have the run method)
Suggested test doc improvement:
diff --git a/test/README.md b/test/README.md
index 37f2c07217..3b2979cdf6 100644
--- a/test/README.md
+++ b/test/README.md
@@ -34,6 +34,9 @@ The ZMQ functional test requires a python ZMQ library. To install it:
- on Unix, run `sudo apt-get install python3-zmq`
- on mac OS, run `pip3 install pyzmq`
+The IPC functional test requires a python IPC library. To install it:
+
+- `pip3 install pycapnp`
On Windows the `PYTHONUTF8` environment variable must be set to 1:4d2a182 to
35498c9
Compare
35498c9 to
9baa7a1
Compare
Expose ENABLE_IPC build option to functional tests so new tests can test IPC-only features.
Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in bitcoin#30437 and `bitcoin-cli` in bitcoin#32297 to find the `bitcoin` binary and call `bitcoin -m` to start nodes with IPC support. This way the new tests can run whenever the ENABLE_IPC build option is enabled, instead of only running when the `BITCOIN_CMD` environment variable is set to `bitcoin -m`
With this change, tests can specify `self.extra_init = [{ipcbind: True}]` to
start a node listening on an IPC socket, instead of needing to choose which
node binary to invoke and what `self.extra_args=[["-ipcbind=..."]]` value to
pass to it.
The eliminates boilerplate code bitcoin#30437 (interface_ipc_mining.py), bitcoin#32297
(interface_ipc_cli.py), and bitcoin#33201 (interface_ipc.py) previously needed in
their test setup.
9baa7a1 to
6173d84
Compare
|
Strange error, https://github.com/bitcoin/bitcoin/actions/runs/17491932432/job/49683489249?pr=33201: in line template4 = await template2.result.waitNext(ctx, waitoptions)Any idea, @ryanofsky? |
6173d84 to
6f1eb21
Compare
|
Debugged on IRC: So the culprit was: waitnext = template2.result.waitNext(ctx, waitoptions)
template4 = await template2.result.waitNext(ctx, waitoptions)Which resulted in attempting two parallel IPC calls being executed in the same thread on the server. |
6f1eb21 to
d0b8af4
Compare
Co-Authored-By: ismaelsadeeq <[email protected]> Co-Authored-By: ryanofsky <[email protected]> Co-Authored-By: TheCharlatan <[email protected]> Co-Authored-By: Sjors Provoost <[email protected]>
Install pycapnp on all (active) CI hosts which have IPC enabled and run the functional tests. Except for previous_releases, which uses an older version of pip that doesn't support --break-system-packages.
ryanofsky
left a comment
There was a problem hiding this comment.
Code review ACK a341e11. Changes since last review: rebasing, switching to miniwallet and expanding wallet test, improving pycapnp install steps in instructions and CI.
Looking at CI changes here, it's interesting that low-level way we configure CI jobs allows dependencies to be installed that might not be used, and things to be built that are not tested. I think a more ideal config / script boundary might make the config only responsible for specifying what should be tested, and let the script be responsible for figuring out what dependencies need to be installed. That's a wider issue though and CI changes here all look right from what I can tell.
re: libmultiprocess assert failure #33201 (comment), I opened bitcoin-core/libmultiprocess#206 to improve this
|
ACK a341e11 As if by magic, |
|
@Sjors Not magic, it needed a |
|
Reviewers may find #33566 interesting. |
This adds support to the functional test framework to run the multiprocess
bitcoin-nodebinary, and then tests it in a newinterface_ipc.pyfunctional test through thepycapnpmodule.