SegWit wallet support#11403
Conversation
promag
left a comment
There was a problem hiding this comment.
Light review, it would be nice to push base PR's.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
In commit Expose method to find key for a single-key destination:
Use boost::apply_visitor instead?
There was a problem hiding this comment.
I tried that, but the result is much longer and harder to get right (see below). It's also the wrong approach I think. A visitor is used when you want to handle all possible types - here we specifically only want to support a specific subset of cases (P2PKH, P2WPKH, P2SH-P2WPKH).
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
In commit Expose method to find key for a single-key destination:
Recursive call?
return GetKeyForDestination(inner_dest);There was a problem hiding this comment.
It's not that easy, as it could match a superset (for example P2SH-P2PKH), or even invalid things (P2SH-P2SH-P2WPKH?).
There was a problem hiding this comment.
Right.
Isn't boost::get kind of expensive? Avoid 2nd call?
There was a problem hiding this comment.
I think it's very cheap - look at the tag, and return the argument with a cast.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
In commit SegWit wallet support:
Remove?
src/wallet/init.cpp
Outdated
There was a problem hiding this comment.
Add init tests (follow up maybe)?
src/wallet/rpcwallet.cpp
Outdated
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Missing test for this error.
src/wallet/rpcwallet.cpp
Outdated
src/wallet/init.cpp
Outdated
There was a problem hiding this comment.
Add init tests (follow up maybe)?
src/wallet/rpcwallet.cpp
Outdated
test/functional/segwit.py
Outdated
There was a problem hiding this comment.
Remove , True so that default value is tested.
src/wallet/init.cpp
Outdated
There was a problem hiding this comment.
These should mention the defaults, which likely means refactoring how the "default" string is handled.
There was a problem hiding this comment.
Added explanation of defaults.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
Default should probably remain legacy until support for newer styles is complete (eg, import* RPC)?
There was a problem hiding this comment.
I'd rather not. This makes the existing tests far more powerful.
There was a problem hiding this comment.
Tests don't need to (and probably shouldn't) rely on defaults...
There was a problem hiding this comment.
Yes and no.
I think that many tests, which are primarily testing things unrelated to addresses, should use the default address type. They don't strictly rely on them, but it's better if they work with either type, and it's certainly easier to not need to change all of them.
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Seems like it would be better to change the deprecated "account" into a JSON Object options.
There was a problem hiding this comment.
I think that should be done separately.
|
A potential problem with supporting |
|
maybe it should be renamed to |
8213838 [Qt] tolerate BIP173/bech32 addresses during input validation (Jonas Schnelli) 06eaca6 [RPC] Wallet: test importing of native witness scripts (NicolasDorier) fd0041a Use BIP173 addresses in segwit.py test (Pieter Wuille) e278f12 Support BIP173 in addwitnessaddress (Pieter Wuille) c091b99 Implement BIP173 addresses and tests (Pieter Wuille) bd355b8 Add regtest testing to base58_tests (Pieter Wuille) 6565c55 Convert base58_tests from type/payload to scriptPubKey comparison (Pieter Wuille) 8fd2267 Import Bech32 C++ reference code & tests (Pieter Wuille) 1e46ebd Implement {Encode,Decode}Destination without CBitcoinAddress (Pieter Wuille) Pull request description: Builds on top of #11117. This adds support for: * Creating BIP173 addresses for testing (through `addwitnessaddress`, though by default it still produces P2SH versions) * Sending to BIP173 addresses (including non-v0 ones) * Analysing BIP173 addresses (through `validateaddress`) It includes a reformatted version of the [C++ Bech32 reference code](https://github.com/sipa/bech32/tree/master/ref/c%2B%2B) and an independent implementation of the address encoding/decoding logic (integrated with CTxDestination). All BIP173 test vectors are included. Not included (and intended for other PRs): * Full wallet support for SegWit (which would include automatically adding witness scripts to the wallet during automatic keypool topup, SegWit change outputs, ...) [see #11403] * Splitting base58.cpp and tests/base58_tests.cpp up into base58-specific code, and "address encoding"-code [see #11372] * Error locating in UI for BIP173 addresses. Tree-SHA512: 238031185fd07f3ac873c586043970cc2db91bf7735c3c168cb33a3db39a7bda81d4891b649685bb17ef90dc63af0328e7705d8cd3e8dafd6c4d3c08fb230341
|
Needs rebase after #11167 merge but that'll make it clearer what's new here to review 👍 |
src/rpc/misc.cpp
Outdated
There was a problem hiding this comment.
This field needs to be added anywhere it's used in help text for RPC calls.
There was a problem hiding this comment.
You might want to copy the field to the help text rather than move it, though :p
(Use after move here)
src/wallet/wallet.h
Outdated
There was a problem hiding this comment.
Remove plurals from description.
a3a79cd to
20da344
Compare
|
Rebased after #11167 merge. |
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
pedantic nit, but this only supports v0 of each witness version
There was a problem hiding this comment.
I've always considered P2WPKH as specifically referring to v0 20-byte witness programs, and P2WSH as v0 32-byte ones. We'll need to come up with even clumsier names later ;)
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
startup options strings being different than getnewaddress options is a bit odd to me?
'p2sh_p2wpkh' vs p2sh etc
There was a problem hiding this comment.
Fixed. Standardized on 'legacy', 'p2sh', and 'bech32'. It's not entirely accurate, as P2SH is a scriptPubKey type and not its address encoding, but meh - they're probably the most recognizable names.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
No solution off the top of my head, but it makes sense that this would be set per-wallet. If not, could we name it g_address_style?
There was a problem hiding this comment.
Fixed as a global now, as we don't really have a way to configure it per wallet. Can be revised if there ever is.
src/wallet/init.cpp
Outdated
src/wallet/rpcwallet.cpp
Outdated
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
change seems way out of place here
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
Same suggestion as address_style
20da344 to
5632374
Compare
|
Regarding petertodd's concern, I think something like flack's suggestion would address it. |
kallewoof
left a comment
There was a problem hiding this comment.
utACK 5632374f2f71bca384ceed07ca6842fd87102346
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
The help for e.g. getnewaddress says "p2wpkh" etc. One or the other needs to be changed.
There was a problem hiding this comment.
Fixed, I think it's consistent now.
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
I think you want "getnewaddress ( \"account\" \"style\" )\n"
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
This shouldn't just be tacked on the end. Accounts are deprecated. Just replace "account" with an options Object.
There was a problem hiding this comment.
That would be API breaking for a lot of parties which is at odds with rapid deployment.
There was a problem hiding this comment.
This shouldn't just be tacked on the end. Accounts are deprecated. Just replace "account" with an options Object.
In this particular context, account will just be renamed to label, rather than removed entirely. (At least this is what #11536 does.)
5632374 to
f542dae
Compare
src/script/standard.h
Outdated
There was a problem hiding this comment.
You could instead inherit the ctors from uint160/uint256 to shortcut the construction from WitnessV0KeyHash(uint160(foo) to WitnessV0KeyHash(foo):
using uint160::uint160;
theuni
left a comment
There was a problem hiding this comment.
Added a few more comments.
I suspect I'm missing something about the generic-ish functions in CWallet. Do you maybe have plans to make those per-wallet as @instagibbs suggested?
src/rpc/misc.cpp
Outdated
There was a problem hiding this comment.
Why dupe this? Helps with existing tooling?
There was a problem hiding this comment.
Yes, tests that expect to see a pubkey field when calling validateaddress on the output of getnewaddress.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
imo, it's clearer (and less redundant) to test the result of the assignment:
if (const CKeyID* ret = boost::get<CKeyID>(&dest)) {
return *ret;
}Edit: That's also the common paradigm for using std::weak_ptr::lock()
src/keystore.cpp
Outdated
There was a problem hiding this comment.
I'm not sure about the sequencing rules here. I'd feel safer with a temporary CScriptID.
There was a problem hiding this comment.
Ugh, thanks for pointing that out.
There was a problem hiding this comment.
I'm not sure about the sequencing rules here. I'd feel safer with a temporary CScriptID.
For the record, there should be no issue here. std::move just does an rvalue reference cast to influence template deduction and function overloading. It doesn't actually actually move anything or do anything else at runtime.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
Can't this pass in nullptr and move out of CWallet? Unless I'm missing something, it's not actually related.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
If IsSolvable moves out of CWallet, I believe these can too.
There was a problem hiding this comment.
Done. IsSolvable is moved to script/sign, GetDestinationForKey is moved to keystore.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
Seems the caller should be responsible for adding this if it's a WitnessV0ScriptHash? Like the others, this doesn't seem like wallet functionality to me.
There was a problem hiding this comment.
Yes, it's ugly. I don't immediately see a nicer solution though, except for a callback to pass in, or duplicating part of this function just to find the correct program.
Note that if it weren't for the implicitly-know-about-witness-versions-of-keys logic, an equivalent line would be needed in GetDestinationForKey or its callers.
|
The wallet probably needs to be able to generate both For example, I might send an email requesting payment to a So rather than I think this was brought up elsewhere, but If |
|
@Sjors There is an RPC argument added to address-creating RPCs to override the default. |
|
@sipa I see, perhaps |
|
Post-merge utACK |
New global variables were introduced in bitcoin#11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4199: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "ListCoins": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/ListCoins Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug.
…s_type, g_change_type f765bb3 Fix ListCoins test failure due to unset g_address_type, g_change_type (Russell Yanofsky) Pull request description: New global variables were introduced in bitcoin#11403 and not setting them causes: ``` test_bitcoin: wallet/wallet.cpp:4199: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "ListCoins": signal: SIGABRT (application abort requested) ``` It's possible to reproduce the failure reliably by running: ``` src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/ListCoins ``` Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug. Example travis failure: https://travis-ci.org/bitcoin/bitcoin/builds/327642495 Tree-SHA512: 3e0875716f66bc0304cf92a26457e6b54ecfe15ed962f4343577b05fc56bb577554422b7f53949ad6085ac5798ad7816b8176c5b01e050ddbfbb925d2732767a
16f6f59 [qa] Test fundrawtransaction with change_type option (João Barbosa) 536ddeb [rpc] Add change_type option to fundrawtransaction (João Barbosa) 31dbd5a [wallet] Add change type to CCoinControl (João Barbosa) Pull request description: Adds a new option `change_type` to `fundrawtransaction` RPC. This is useful to override the node `-changetype` argument. The new option is exclusive to `changeAddress` option, setting both raises a RPC error. See also #11403, #12119. Tree-SHA512: 654686444f6125e37015a62f167064d54ec335701534988447be4687fa5ef9c7980a8a07cc0a03fff6ea6c4c1abf0f77a8843d535c4f3fe0bf93f968a4e676e6
New global variables were introduced in bitcoin#11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4199: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "ListCoins": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/ListCoins Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug.
New global variables were introduced in bitcoin#11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug.
…change_type b7f6002 Fix rescan test failure due to unset g_address_type, g_change_type (Russell Yanofsky) Pull request description: New global variables were introduced in #11403 and not setting them causes: ``` test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested) ``` It's possible to reproduce the failure reliably by running: ``` src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan ``` Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug. This is similar to bug #12150. Example travis failure is https://travis-ci.org/bitcoin/bitcoin/jobs/340642010 Tree-SHA512: ab40662b3356892b726f1f552e22d58d86b5e982538741e52b37ee447a0c97c76c24ae543687edf2e25d9dd925722909d37abfae95d93bf09e23fa245a4c3351
New global variables were introduced in #11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug. Github-Pull: #12424 Rebased-From: b7f6002 Tree-SHA512: 1cc64db3b1d886d793e9d194b318dde3d5f628bde778a50513de4bf54dcfc77152885e72608927e3e490d253350ca0381847539a904cb31862f3a6fceac88dc1
New global variables were introduced in bitcoin#11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug.
Suggested by Matt Corallo <[email protected]> bitcoin#11403 (comment) Combining the maps was probably never a good arrangement but is more problematic now in presence of WitnessV0ScriptHash and WitnessV0KeyHash types. Conflicts: src/wallet/wallet.h src/wallet/walletdb.cpp
New global variables were introduced in bitcoin#11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4199: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "ListCoins": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/ListCoins Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug.
New global variables were introduced in bitcoin#11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug. Github-Pull: bitcoin#12424 Rebased-From: b7f6002 Tree-SHA512: 1cc64db3b1d886d793e9d194b318dde3d5f628bde778a50513de4bf54dcfc77152885e72608927e3e490d253350ca0381847539a904cb31862f3a6fceac88dc1
Suggested by Matt Corallo <[email protected]> bitcoin/bitcoin#11403 (comment) Combining the maps was probably never a good arrangement but is more problematic now in presence of WitnessV0ScriptHash and WitnessV0KeyHash types.
New global variables were introduced in bitcoin#11403 and not setting them causes: test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed. unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested) It's possible to reproduce the failure reliably by running: src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug.
This implements a minimum viable implementation of SegWit wallet support, based on top of #11389, and includes part of the functionality from #11089.
Two new configuration options are added:
-addresstype, with optionslegacy,p2sh, andbech32. It controls what kind of addresses are produced bygetnewaddress,getaccountaddress, andcreatemultisigaddress.-changetype, with the same options, and by default equal to-addresstype, that controls what kind of change is used.All wallet private and public keys can be used for any type of address. Support for address types dependent on different derivation paths will need a major overhaul of how our internal detection of outputs work. I expect that that will happen for a next major version.
The above also applies to imported keys, as having a distinction there but not for normal operations is a disaster for testing, and probably for comprehension of users. This has some ugly effects, like needing to associate the provided label to
importprivkeywith each style address for the corresponding key.To deal with witness outputs requiring a corresponding redeemscript in wallet, three approaches are used:
getnewaddressor multisig RPCs explicitly get their redeemscripts added to the wallet file. This means that downgrading after creating a witness address will work, as long as the wallet file is up to date.These approaches correspond to solutions 3a, 1a, and 5a respectively from https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2. As argued there, there is no full solution for dealing with the case where you both downgrade and restore a backup, so that's also not implemented.
dumpwallet,importwallet,importmulti,signmessageandverifymessagedon't work with SegWit addresses yet. They're remaining TODOs, for this PR or a follow-up. Because of that, several tests unexpectedly run with-addresstype=legacyfor now.