[EXPERIMENTAL] Schnorr batch verification for blocks#29491
[EXPERIMENTAL] Schnorr batch verification for blocks#29491fjahr wants to merge 10 commits intobitcoin:masterfrom
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29491. ReviewsSee the guideline for information on the review process. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. LLM Linter (✨ experimental)Possible typos and grammar issues:
Possible places where named args for integral literals may be used (e.g.
2026-01-31 14:01:32 |
454783c to
0a5e612
Compare
|
🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the Possibly this is due to a silent merge conflict (the changes in this pull request being Leave a comment here, if you need help tracking down a confusing failure. |
|
I guess this is most interesting to test against the last 50K blocks on mainnet, i.e. since early 2023? Given the high percentage of taproot transactions. Such testing could benefit from a assume utxo snapshot, saving everyone the pain of rewinding. I can bake one, though ideally I'd like #26045 to land first. Is the current PR already using batches? It's unclear to me by glancing over Maybe add a |
Yeah, that would be great. I have to get the tracing part to work in my setup again before I can start.
Yes, the rough architecture is as as follows (sorry for not writing a better explanation above but it's still very rough and I expect it to change). So, the The 106 is an implementation detail from Everything should be conditional on the presence of a
Yeah, I have been thinking that or maybe even a compiler argument |
0a5e612 to
1825555
Compare
|
@Sjors maybe what confused you is that |
That clarifies a lot, and should be documented :-) |
|
@willcl-ark and I ran a IBD benchmark vs master for this PR: IBD from a local node to height 840000 and currently only looking at the connect_block duration. In the current draft implementation:
Used this bpftrace script to collect CSV data: #!/usr/bin/env bpftrace
usdt:./src/bitcoind:validation:block_connected
{
$height = (int32) arg1;
$tx = (uint64) arg2;
$ins = (int32) arg3;
$sigops = (int64) arg4;
$duration = (int64) arg5; // in nanoseconds
printf("%d,%ld,%d,%ld,%ld\n", $height, $tx, $ins, $sigops, $duration);
} |
From a quick skim of the changes ISTM that we always use the |
1825555 to
6f271d7
Compare
|
It seems in this approach the I think a better approach would be to have a thread local batch for each worker thread and the main thread, so each thread can add all schnorr sigs without locking, and then each thread can verify their batches in parallel at the end. |
6f271d7 to
cafdd14
Compare
bcbbb57 to
452a375
Compare
fjahr
left a comment
There was a problem hiding this comment.
Rebased on latest master and included the latest API changes in the batch module PR in secp256k1.
src/batchverify.cpp
Outdated
There was a problem hiding this comment.
Not discarding it anymore in the latest push.
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
c5a124f to
3979539
Compare
46d1cf7 to
41e0ea6
Compare
15ea24cb8c batch: make add functions void & introduce reset bfcc479a35 batch: remove `batch_usable` api 15e388e096 batch: make tests functions internal & static aac054a373 fix typos & index the right inputs for benchmarks c07e710003 batch: remove experimental status 49fb753393 test: fix ci failures e96dabb4af batch: Generate speedup graphs b0b3425cd4 batch, extrakeys: Add benchmarks 9d5115156b batch: Add tests for batch_add_* APIs 668199c917 batch,ecmult: Add tests for core batch APIs and strauss_batch refactor 53a158203f batch: Add example b40b4186b8 batch: Add batch_add_* APIs 2bed1cb6ee batch, ecmult: Add batch_verify and refactor strauss_batch 8f13eeae31 batch: Add create and destroy APIs 0b6b0c87ad batch: Initialize an experimental batch module REVERT: 14e56970cb Merge bitcoin-core/secp256k1#1794: ecmult: Use size_t for array indices REVERT: c7a52400d6 Merge bitcoin-core/secp256k1#1809: release cleanup: bump version after 0.7.1 REVERT: ae7eb729c0 release cleanup: bump version after 0.7.1 REVERT: 1a53f4961f Merge bitcoin-core/secp256k1#1808: Prepare for 0.7.1 REVERT: 20a209f11c release: prepare for 0.7.1 REVERT: c4b6a81a60 changelog: update in preparation for the v0.7.1 release REVERT: ebb35882da Merge bitcoin-core/secp256k1#1796: bench: fail early if user inputs invalid value for SECP256K1_BENCH_ITERS REVERT: c09215f7af bench: fail early if user inputs invalid value for SECP256K1_BENCH_ITERS REVERT: 471e3a130d Merge bitcoin-core/secp256k1#1800: sage: verify Eisenstein integer connection for GLV constants REVERT: 29ac4d8491 sage: verify Eisenstein integer connection for GLV constants REVERT: 4721e077b4 Merge bitcoin-core/secp256k1#1793: doc/bench: added help text for SECP256K1_BENCH_ITERS env var for bench_ecmult REVERT: bd5ced1fe1 doc/bench: added help text for SECP256K1_BENCH_ITERS env var for bench_ecmult REVERT: 47eb70959a ecmult: Use size_t for array indices in _odd_multiplies_table REVERT: bb1d199de5 ecmult: Use size_t for array indices into tables REVERT: 2d9137ce9d Merge bitcoin-core/secp256k1#1764: group: Avoid using infinity field directly in other modules REVERT: f9a944ff2d Merge bitcoin-core/secp256k1#1790: doc: include arg -DSECP256K1_USE_EXTERNAL_DEFAULT_CALLBACKS=ON for cmake REVERT: 0406cfc4d1 doc: include arg -DUSE_EXTERNAL_DEFAULT_CALLBACKS=1 for cmake REVERT: 8d445730ec Merge bitcoin-core/secp256k1#1783: Add VERIFY_CHECKs and documentation that flags must be 0 or 1 REVERT: aa2a39c1a7 Merge bitcoin-core/secp256k1#1778: doc/bench: Added cmake build options to bench error messages REVERT: 540fec8ae9 Merge bitcoin-core/secp256k1#1788: test: split monolithic ellswift test into independent cases REVERT: d822b29021 test: split monolithic ellswift test into independent cases REVERT: ae00c552df Add VERIFY_CHECKs that flags are 0 or 1 REVERT: 5c75183344 Merge bitcoin-core/secp256k1#1784: refactor: remove ret from secp256k1_ec_pubkey_serialize REVERT: be5e4f02fd Merge bitcoin-core/secp256k1#1779: Add ARG_CHECKs to ensure "array of pointers" elements are non-NULL REVERT: 3daab83a60 refactor: remove ret from secp256k1_ec_pubkey_serialize REVERT: 8bcda186d2 test: Add non-NULL checks for "pointer of array" API functions REVERT: 5a08c1bcdc Add ARG_CHECKs to ensure "array of pointers" elements are non-NULL REVERT: 3b5b03f301 doc/bench: Added cmake build options to bench error messages REVERT: e7f7083b53 Merge bitcoin-core/secp256k1#1774: refactor: split up internal pubkey serialization function into compressed/uncompressed variants REVERT: b6c2a3cd77 Merge bitcoin-core/secp256k1#1761: ecmult_multi: reduce strauss memory usage by 30% REVERT: f5e815f430 remove secp256k1_eckey_pubkey_serialize function REVERT: 0d3659c547 use new `_eckey_pubkey_serialize{33,65}` functions in modules (ellswift,musig) REVERT: adb76f82ea use new `_eckey_pubkey_serialize{33,65}` functions in public API REVERT: fc7458ca3e introduce `secp256k1_eckey_pubkey_serialize{33,65}` functions REVERT: 2f73e5281d group: Avoid using infinity field directly in other modules REVERT: 26166c4f5f ecmult_multi: reduce strauss memory usage by 30% git-subtree-dir: src/secp256k1 git-subtree-split: 15ea24cb8c1bd239a7a39939da1952cf6d3a35b0
… verification Co-authored-by: Eunovo <[email protected]>
41e0ea6 to
f178c6c
Compare
|
🐙 This pull request conflicts with the target branch and needs rebase. |


This is a draft implementation of Schnorr signature batch verification in blocks, put here for conceptual discussion, and collaborative benchmarking.
The
secp256k1code is still under review itself, please spend some time giving feedback there if you can:TODOs