Factor out PSBT utilities from RPCs for use in GUI code; related refactoring.#14978
Conversation
src/core_read.cpp
Outdated
There was a problem hiding this comment.
Ok, I thought I was very clever converting this from a std::vector<unsigned char> to a std::string, but the reality appears to be that Clang is happy with this line and g++ hates it, so I'm going to have to massage this back towards the original version a bit, in order to get it to build anywhere but macOS.
|
I cleverly made a small change that apparently g++ does not accept but Clang does, so I will have to fix that before it will build on Linux, oops. (I commented on the offending line in the diff.) |
3c91a3d to
2fe27d6
Compare
|
Hmm, something is also legitimately broken in the tests, will investigate tomorrow. :-\ |
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
Concept ACK.
Can you detail the GUI part?
Is it available? |
2fe27d6 to
909bda5
Compare
I'm adding an interface for offline and multisig signing in the GUI, using PSBT as the on-disk format for the intermediate files. See e.g. how Bitcoin Armory handles offline transactions, which I'm basically copying.
I pasted it into #bitcoin-core-dev awhile ago -- I will link it here in the comment thread once I'm sure the tip of my branch is still actually building. |
|
@gwillen writes:
* Move most PSBT definitions into psbt.h.
* Move most PSBT RPC utilities into psbt.{h,cpp}.
* Move wallet-touching PSBT RPC utilities (FillPSBT) into
wallet/psbtwallet.{h,cpp}.
* Switch exceptions from JSONRPCError() to new PSBTException class.
* Split DecodePSBT into DecodeBase64PSBT (old behavior) and DecodeRawPSBT.
* Add one new version of DecodeBase64 utility in strencodings.h (and
corresponding DecodeBase32 for completeness).
* Factor BroadcastTransaction utility function out of sendrawtransaction RPC
handler in rpc/rawtransaction.cpp
Is it possible to split this commit into one for each of the points
here? It makes review harder when everything is squashed into a single commit.
|
|
The GUI work is WIP here: https://github.com/gwillen/bitcoin/tree/feature-offline-v1 . I made that branch at a known-working version, since I'm making significant changes to it right now. It's not currently based on this PR, which I will have to rebase it onto later; this PR is an extraction of some refactoring that also appears in that PR, plus some other stuff that doesn't yet. |
|
@jb55 Yeah, I can definitely split it out if people prefer. Will take a day or two to get to. I might ask slight forgiveness in some cases where a reasonable intermediate state between two combined changes never existed, and would be hard to recreate. |
|
Would be helpful to split the commit up, for review now and for intelligibility when looking back in the future. |
|
Concept ACK. Agree with above, split this into multiple commits. |
|
I definitely agree in splitting it. @gwillen checked out the branch and looks promising. |
71290c3 to
e155e95
Compare
|
Sorry for the delay. I have split the PR into more reasonable commits. The one that does the bulk of the code movement between files ("Move PSBT definitions and code to separate files") is now only movement (and adjusting headers / Makefile), and no other substantive changes. The final result is exactly the same as before, except for a single stray blank line I had accidentally introduced in the previous version, which is removed. Every intermediate step is intended to be sane and self-contained (and buildable), but I haven't actually verified (by compiler rather than by eye) that every intermediate step cleanly builds yet. (EDIT: Wellll, I was pretty close. All intermediate states now build and pass tests.) |
e155e95 to
dd45d3f
Compare
|
This looks much better now! Thanks! utACK dd45d3febacb40cf68f58eff08f76db2854fc340 The refactorings look pretty good in general. I did a quick skim of the code, I didn't see anything obviously wrong but I'm no expert on this part of the codebase. |
src/util/strencodings.cpp
Outdated
There was a problem hiding this comment.
nit: variable name should be snake_case. See doc/developer-notes.md for naming convention.
There was a problem hiding this comment.
This was copied from surrounding context (it just wraps the c_str version of the same function, which has the same parameter named the same thing, and I'm propagating it outwards to the wrapper.)
There was a problem hiding this comment.
From https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md:
When writing patches, favor the new style over attempting to mimic the surrounding style, except for move-only commits.
There was a problem hiding this comment.
Nit: we're only calling DecodeBase64 in a handful of places, so you could also change the return type to bool and pass the output string as a reference param. Though it also makes sense to remain consistent with the existing (char*) methods.
There was a problem hiding this comment.
I will fix the argument name, but otherwise leave things alone unless you would strongly prefer the more invasive changes. (I am changing pfInvalid to pf_invalid everywhere in the surrounding context, because it will drive me completely nuts to make it inconsistent. Please don't make me do that.)
There was a problem hiding this comment.
Yeah, let's not do too much in this PR.
src/psbt.h
Outdated
There was a problem hiding this comment.
This results in RPC errors with a code RPC_MISC_ERROR. I think it would be better if you could somehow make the error code more specific (the message still comes through).
There was a problem hiding this comment.
Yeah, I'd love any thoughts on how to accomplish this without gunking up non-RPC code too much when it calls this codepath. I don't really want this to depend on the RPC stuff.
There was a problem hiding this comment.
Thread 617247c#r246575002
One simple way this could be achieved is to add custom error handler to CRPCCommand which would translate this exception to the desired output.
One non intrusive way to add the error handler is to set it in
bitcoin/src/wallet/rpcwallet.cpp
Lines 4194 to 4197 in 391a273
This way all desired commands would share the same error handling and then we can start removing duplicate error messages in the code by introducing more specific exceptions.
There was a problem hiding this comment.
I'm slightly worried about the use of exceptions in code that use usable in more places than just RPC, as C++ can't guarantee that all possible exceptions are being dealt with at the right layer.
Would an enum with PSBT-related error codes, plus a enum to string conversion function (like in script/script_error) make sense?
|
FWIW I believe the travis failure is spurious -- it failed a single unrelated test, on a single platform, and the code hasn't changed since the last time it succeeded. |
dd45d3f to
c062d74
Compare
|
(Minor edit to remove a single unnecessary #include line that was bugging me, no significant change.) |
promag
left a comment
There was a problem hiding this comment.
utACK c062d74 .Verified the move only commit, just adds necessary includes and updates the makefile with the new file. Some nits and other comments for your consideration.
src/core_read.cpp
Outdated
There was a problem hiding this comment.
Commit 801333b
Could add test for this new error?
src/core_read.cpp
Outdated
There was a problem hiding this comment.
Commit 801333b
To avoid changing this line you could add a commit before with something along:
--- a/src/streams.h
+++ b/src/streams.h
@@ -236,17 +236,8 @@ public:
Init(nTypeIn, nVersionIn);
}
- CDataStream(const vector_type& vchIn, int nTypeIn, int nVersionIn) : vch(vchIn.begin(), vchIn.end())
- {
- Init(nTypeIn, nVersionIn);
- }
-
- CDataStream(const std::vector<char>& vchIn, int nTypeIn, int nVersionIn) : vch(vchIn.begin(), vchIn.end())
- {
- Init(nTypeIn, nVersionIn);
- }
-
- CDataStream(const std::vector<unsigned char>& vchIn, int nTypeIn, int nVersionIn) : vch(vchIn.begin(), vchIn.end())
+ template <typename T>
+ CDataStream(const T& vchIn, int nTypeIn, int nVersionIn) : vch(vchIn.begin(), vchIn.end())
{
Init(nTypeIn, nVersionIn);
}There was a problem hiding this comment.
Is this safe (does it not "prove too much")? Certainly I can imagine it might cause worse error messages in some cases. I assume it wasn't done that way for some reason originally?
There was a problem hiding this comment.
Just for fun, I went and checked -- this code is essentially unchanged since the repo was first imported to github in 2009. So the rationale is probably lost to history...
src/core_io.h
Outdated
There was a problem hiding this comment.
Commit 801333b
nit, would be nice see brief comments.
There was a problem hiding this comment.
nit: base64_psbt (see next comment)
src/core_io.h
Outdated
There was a problem hiding this comment.
Commit 801333b
nit, s/tx_data/raw_tx?
There was a problem hiding this comment.
Both tx_data and raw_tx suggest a transaction hex. Something like raw_psbt is probably more clear, and consistent with the function name.
src/rpc/rawtransaction.h
Outdated
There was a problem hiding this comment.
Commit 745c540
Shouldn't this be elsewhere if the goal is to use in the GUI?
I think this should return uint256.
nit, could document that the transaction id is returned.
nit, s/allowhighfees/allow_high_fees.
There was a problem hiding this comment.
Returning uint256 seems more natural indeed.
There was a problem hiding this comment.
I also need access to this function in #14912 from other wallet RPC methods. Not really an issue, because I can always include src/rpc/rawtransaction.h. But perhaps create a new src/wallet/transaction.cpp?
I might add BroadcastTransaction(PartiallySignedTransaction psbt, bool allowhighfees = false) in my own PR.
There was a problem hiding this comment.
Agreed that this should be elsewhere, I will create src/wallet/transaction.cpp for it. (I was reluctant to be too original just to move a single function, but I prefer it that way also.) And agree regarding the return value, will change it.
I don't think there's too much additional value in taking a PartiallySignedTransaction, since finalization always results in a CTransaction.
There was a problem hiding this comment.
(agree with the latter, I ended up with a different approach too, the seperate file is useful enough)
There was a problem hiding this comment.
Done, but this now makes it more obvious that BroadcastTransaction has the same question as the PSBT functions as to how it shall signal an error, since right now it throws UniValues like the other RPC stuff.
src/rpc/rawtransaction.cpp
Outdated
There was a problem hiding this comment.
Commit 745c540
nit, despite being valid I think it's more readable in the form:
bool allowhighfees = false;
if (!request.params[1].isNull()) allowhighfees = request.params[1].get_bool();Because it is more clear the default value.
src/psbt.h
Outdated
There was a problem hiding this comment.
Thread 617247c#r246575002
One simple way this could be achieved is to add custom error handler to CRPCCommand which would translate this exception to the desired output.
One non intrusive way to add the error handler is to set it in
bitcoin/src/wallet/rpcwallet.cpp
Lines 4194 to 4197 in 391a273
This way all desired commands would share the same error handling and then we can start removing duplicate error messages in the code by introducing more specific exceptions.
src/psbt.h
Outdated
There was a problem hiding this comment.
Commit 08ef528
If this is only used once and you now it's called from Merge then I suggest to inline it there.
src/psbt.h
Outdated
There was a problem hiding this comment.
Commit 08ef528
Add comment explaining the return value.
src/psbt.cpp
Outdated
There was a problem hiding this comment.
Commit c062d74
Make the argument a const reference.
nit, brace on new line.
src/util/strencodings.cpp
Outdated
There was a problem hiding this comment.
From https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md:
When writing patches, favor the new style over attempting to mimic the surrounding style, except for move-only commits.
src/rpc/rawtransaction.h
Outdated
There was a problem hiding this comment.
Returning uint256 seems more natural indeed.
src/psbt.h
Outdated
There was a problem hiding this comment.
I'm slightly worried about the use of exceptions in code that use usable in more places than just RPC, as C++ can't guarantee that all possible exceptions are being dealt with at the right layer.
Would an enum with PSBT-related error codes, plus a enum to string conversion function (like in script/script_error) make sense?
… GUI code; related refactoring Only contains 162ffef
Merge bitcoin#19660, bitcoin#19373, bitcoin#19841, bitcoin#13862, bitcoin#13866, bitcoin#17280, bitcoin#17682 and partial bitcoin#19326, bitcoin#14978: Auxiliary Backports
…UI code; related refactoring
…UI code; related refactoring
merge bitcoin#16117, bitcoin#18358, bitcoin#17383, bitcoin#21052, bitcoin#14424, bitcoin#15159, bitcoin#14689, bitcoin#14978, partial bitcoin#16908, bitcoin#14978, bitcoin#13932: Auxillary Backports
wallet/psbtwallet.{h,cpp}.
corresponding DecodeBase32 for completeness).
handler in rpc/rawtransaction.cpp
Note: For those keeping score at home wondering why refactor, this is in anticipation of (and developed in parallel with) a change to actually introduce GUI use of all this stuff, which is already under development and working-ish.