wallet: introduce setfeerate (an improved settxfee, in sat/vB)#20391
wallet: introduce setfeerate (an improved settxfee, in sat/vB)#20391jonatack wants to merge 12 commits intobitcoin:masterfrom
Conversation
|
Thanks to @xekyo for suggesting this in #20305 (comment). |
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
In the 'Successful responses' and 'Error responses' above shouldn't it be |
|
@prayank23 thanks for looking. Those will be automatically updated from sat/B to sat/vB by #20305, which updates the ToString() helper in |
fa492af to
a96417d
Compare
|
Rebased and updated per the plan outlined in #20484 (comment). |
maflcko
left a comment
There was a problem hiding this comment.
Concept ACK on adding the feature for users to set -paytxfee via rpc at runtime in sat/b
14c5e50 to
2176a3a
Compare
Make settxfee a hidden rpc to avoid confusion for new users and guide them to use the setfeerate rpc in sat/vB, while allowing existing scripts based on settxfee in BTC/kvB to continue using it without breaking. A hidden RPC means that `bitcoin-cli help settxfee` continues to display the help documentation while removing it from the results of the general `bitcoin-cli help`. Updates the release note.
per PR 20403 review feedback by MarcoFalke
|
I start reviewing this one. |
S3RK
left a comment
There was a problem hiding this comment.
I did a high level pass, didn't look at functional tests at all yet. Overall seems good, but I need to give it more thorough look. Please check my comments.
| static_assert(std::is_integral<I>::value, "CFeeRate should be used without floats"); | ||
| } | ||
| static CFeeRate FromSatB(CAmount fee_rate); | ||
| static CFeeRate FromBtcKb(CAmount fee_rate); |
There was a problem hiding this comment.
On line 47 I believe it should be (nFeePaid * 1e3 / 1e8) == (nFeePaid / 1e5)
and not (nFeePaid * 1e8 / 1e3) == (nFeePaid / 1e5)
| switch (mode) { | ||
| case FeeEstimateMode::SAT_VB: | ||
| return strprintf("%d.%03d %s/vB", nSatoshisPerK / 1000, nSatoshisPerK % 1000, CURRENCY_ATOM); | ||
| default: |
There was a problem hiding this comment.
Do we want to assert some other FeeEstimateMode isn't passed by mistake?
| BOOST_CHECK_EQUAL(CFeeRate(3141).ToString(FeeEstimateMode::SAT_VB, /* with_units */ false), "3.141"); | ||
| BOOST_CHECK_EQUAL(CFeeRate(10002).ToString(FeeEstimateMode::SAT_VB, /* with_units */ false), "10.002"); | ||
| BOOST_CHECK_EQUAL(CFeeRate(3141).ToString(FeeEstimateMode::BTC_KVB, /* with_units */ false), "0.00003141"); | ||
| BOOST_CHECK_EQUAL(CFeeRate(10002).ToString(FeeEstimateMode::BTC_KVB, /* with_units */ false), "0.00010002"); |
There was a problem hiding this comment.
what happens if we pass an absurdly big number?
| BOOST_CHECK_THROW(AmountFromValue(ValueFromString("93e+9")), UniValue); //overflow error | ||
| } | ||
|
|
||
| BOOST_AUTO_TEST_CASE(rpc_parse_fee_rate_values) |
There was a problem hiding this comment.
Aren't CFeeRate various constructors and ToString() already covered by other unit tests? Unit tests have a non-zero maintenance cost and I'm missing the point of such duplication. The ValueFromFeeRate fn is pretty trivial and I believe the test should be trivial as well.
| default: return strprintf("%d.%08d %s/kvB", nSatoshisPerK / COIN, nSatoshisPerK % COIN, CURRENCY_UNIT); | ||
| if (with_units) { | ||
| switch (mode) { | ||
| case FeeEstimateMode::SAT_VB: |
There was a problem hiding this comment.
I'm confused why do we put fee estimate mode and fee units, which are different things, into the same enum. I understand this is a bit out of scope for this PR, but I'd appreciate if you could provide some historical context for educational purposes.
There was a problem hiding this comment.
IIRC that was done in #11413, but good point. I'll look at fixing that.
There was a problem hiding this comment.
Ah, just remembered that I wrote a patch in November 2020 that does that but I have not opened it yet to not have too many PRs open.
| LOCK(wallet.cs_wallet); | ||
|
|
||
| const CFeeRate amount{CFeeRate::FromSatB(AmountFromValue(request.params[0]))}; | ||
| const CFeeRate relay_min_feerate{wallet.chain().relayMinFee().GetFeePerK()}; |
There was a problem hiding this comment.
Do we need GetFeePerK() call here and below?
| BOOST_CHECK(CFeeRate::FromSatB(CAmount(100000)) == CFeeRate(1)); | ||
| BOOST_CHECK(CFeeRate::FromSatB(CAmount(100001)) == CFeeRate(1)); | ||
| BOOST_CHECK(CFeeRate::FromSatB(CAmount(2690000)) == CFeeRate(26)); | ||
| BOOST_CHECK(CFeeRate::FromSatB(CAmount(123456789)) == CFeeRate(1234)); |
There was a problem hiding this comment.
what happens if we pass an absurdly big number?
|
Thank you @S3RK for your feedback, much appreciated. I'll reply soon. |
S3RK
left a comment
There was a problem hiding this comment.
I finished the review. A bunch of small questions/comments, but nothing serious. Happy to discuss and do further reviews.
| @@ -717,7 +719,7 @@ def test_option_feerate(self): | |||
|
|
|||
| result = node.fundrawtransaction(rawtx) # uses self.min_relay_tx_fee (set by settxfee) | |||
| btc_kvb_to_sat_vb = 100000 # (1e5) | |||
There was a problem hiding this comment.
This is already defined earlier
| fee_per_byte = Decimal('0.001') / 1000 | ||
| self.nodes[2].settxfee(fee_per_byte * 1000) |
There was a problem hiding this comment.
| fee_per_byte = Decimal('0.001') / 1000 | |
| self.nodes[2].settxfee(fee_per_byte * 1000) | |
| fee_sat_per_byte = 100 | |
| self.nodes[2].setfeerate(fee_sat_per_byte) |
| @@ -717,7 +719,7 @@ def test_option_feerate(self): | |||
|
|
|||
| result = node.fundrawtransaction(rawtx) # uses self.min_relay_tx_fee (set by settxfee) | |||
There was a problem hiding this comment.
| result = node.fundrawtransaction(rawtx) # uses self.min_relay_tx_fee (set by settxfee) | |
| result = node.fundrawtransaction(rawtx) # uses self.min_relay_tx_fee (set by setfeerate) |
| result = [self.nodes[3].fundrawtransaction(rawtx), # uses self.min_relay_tx_fee (set by setfeerate) | ||
| self.nodes[3].fundrawtransaction(rawtx, {"subtractFeeFromOutputs": []}), # empty subtraction list | ||
| self.nodes[3].fundrawtransaction(rawtx, {"subtractFeeFromOutputs": [0]}), # uses self.min_relay_tx_fee (set by settxfee) | ||
| self.nodes[3].fundrawtransaction(rawtx, {"subtractFeeFromOutputs": [0]}), # uses self.min_relay_tx_fee (set by setfeerate) |
There was a problem hiding this comment.
There are couple more comments to fix below
| test_response(requested=fee_rate, expected=fee_rate, msg="Fee rate for transactions with this wallet successfully set to 25.000 sat/vB") | ||
| bumped_tx = rbf_node.bumpfee(rbfid) | ||
| actual_feerate = bumped_tx["fee"] * COIN / rbf_node.getrawtransaction(bumped_tx["txid"], True)["vsize"] | ||
| assert_greater_than(Decimal("0.01"), abs(fee_rate - actual_feerate)) |
There was a problem hiding this comment.
Why this specific value of 0.01? In the test case with settxfee we have a different value of 0.00001000. Should we have a testcase-wise constant?
There was a problem hiding this comment.
Even 0.01 is too small for this. Signatures can be 71-73 bytes long, but even 72 bytes throws this off.
| assert_greater_than(Decimal("0.01"), abs(fee_rate - actual_feerate)) | |
| bumped_txdetails = rbf_node.getrawtransaction(bumped_tx["txid"], True) | |
| allow_for_bytes_offset = len(bumped_txdetails['vout']) * 2 # potentially up to 2 bytes per output | |
| actual_fee = bumped_tx["fee"] * COIN | |
| assert_approx(actual_fee, fee_rate * bumped_txdetails['vsize'], fee_rate * allow_for_bytes_offset) |
| def test_response(*, wallet="RBF wallet", requested=0, expected=0, error=None, msg): | ||
| assert_equal(rbf_node.setfeerate(requested), {"wallet_name": wallet, "fee_rate": expected, ("error" if error else "result"): msg}) | ||
|
|
||
| # Test setfeerate with too high/low values returns expected errors |
There was a problem hiding this comment.
This test has nothing to do with bumpfee RPC. I believe it's better to have it in a separate test. wallet_bumpfee.py is already huge
| assert_greater_than(Decimal("0.01"), abs(fee_rate - actual_feerate)) | ||
| test_response(msg="Fee rate for transactions with this wallet successfully unset. By default, automatic fee selection will be used.") | ||
|
|
||
| # Test setfeerate with a different -maxtxfee |
There was a problem hiding this comment.
Same as above. This belongs to setfeerate test, not bumpfee test
| strprintf("%s%d.%08d", sign ? "-" : "", quotient, remainder)); | ||
| } | ||
|
|
||
| UniValue ValueFromFeeRate(const CFeeRate& fee_rate, FeeEstimateMode mode) |
There was a problem hiding this comment.
While I think it's a great idea to have this function, I'm a bit concerned that it'll be not used and forgotten. Do you have plans to do a follow up and put it to use by replacing ValuefromAmount in appropriate places ?
| throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot specify both estimate_mode and fee_rate"); | ||
| } | ||
| cc.m_feerate = CFeeRate(AmountFromValue(fee_rate), COIN); | ||
| cc.m_feerate = CFeeRate::FromSatB(AmountFromValue(fee_rate)); |
There was a problem hiding this comment.
I love the idea of named constructors. Should we strive for consistency and use them in more places?
| def test_response(*, requested=0, expected=0, error=None, msg): | ||
| assert_equal(node.setfeerate(requested), {"wallet_name": "test_wallet", "fee_rate": expected, ("error" if error else "result"): msg}) | ||
|
|
||
| # Test setfeerate with 10.0001 (CFeeRate rounding), "10.001" and "4" sat/vB |
There was a problem hiding this comment.
Again I believe those check would be better places in a separate setfeerate test
|
🐙 This pull request conflicts with the target branch and needs rebase. Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft". |
| throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot specify both estimate_mode and fee_rate"); | ||
| } | ||
| cc.m_feerate = CFeeRate(AmountFromValue(fee_rate), COIN); | ||
| cc.m_feerate = CFeeRate::FromSatB(AmountFromValue(fee_rate)); |
There was a problem hiding this comment.
In commit "wallet: update rpcwallet to CFeeRate named constructors" (c72083c)
Wow wow. I see that this is doing the right thing, but these constructor names seem absurdly misleading! The AmountFromValue function converts a floating point number to a fixed-point representation of that number multiplied by 10e8. So even though the fee_rate univalue argument has units of sat/B, the AmountFromValue integer return value is no longer in the same units, because it has been multiplied by 100000000. So FromSatB should really be called CFeeRate::FromSatB_Fixed_Point_e8 or something like that because it is taking sat/B values with a very specific, atypical number representation.
An analogous thing is happening with FromBtcKb below. It doesn't really take a BTC/Kb argument. That's basically false advertising. What it really takes is a BTC/Kb*10e8 argument which is the same as a sat/Kb argument, so this constructor would be more accurately called CFeeRate::FromSatKb
I guess I'm having a negative reaction to these constructor names, but I do think this could be cleaned up pretty easily. I think it would be better if CFeeRate class had no involvement with univalue parsing or fixed point representations, and if it didn't (mis)use the CAmount type as a way to represent rates when CAmount is used almost everywhere else only to represent quantities.
My suggestion to be less confusing / misleading would be to drop the two new CFeeRate constructors, and just add standalone RPC util functions:
CFeeRate FeeRateFromSatB(const UniValue& value);
CFeeRate FeeRateFromBtcKb(const UniValue& value);
Then call these to simplify CFeeRate::FromSatB(AmountFromValue(feerate)) as FeeRateFromSatB(feerate) here and in setfeerate and to simplify CFeeRate::FromBtcKb(AmountFromValue(feerate)) as FeeRateFromBtcKb(feerate) in settxfee
There was a problem hiding this comment.
Wow wow. I see that this is doing the right thing, but these constructor names seem absurdly misleading!
The current code in master isn't exactly clear right now either:
/** Constructor for a fee rate in satoshis per kvB (sat/kvB).
*
* Passing a num_bytes value of COIN (1e8) returns a fee rate in satoshis per vB (sat/vB),
* e.g. (nFeePaid * 1e8 / 1e3) == (nFeePaid / 1e5),
* where 1e5 is the ratio to convert from BTC/kvB to sat/vB.
*/
CFeeRate(const CAmount& nFeePaid, uint32_t num_bytes);I think it would be good to figure out how to clear this up before introducing further features that depend on this.
There was a problem hiding this comment.
Wow wow. I see that this is doing the right thing, but these constructor names seem absurdly misleading!
The current code in master isn't exactly clear right now either:
/** Constructor for a fee rate in satoshis per kvB (sat/kvB). * * Passing a num_bytes value of COIN (1e8) returns a fee rate in satoshis per vB (sat/vB), * e.g. (nFeePaid * 1e8 / 1e3) == (nFeePaid / 1e5), * where 1e5 is the ratio to convert from BTC/kvB to sat/vB. */ CFeeRate(const CAmount& nFeePaid, uint32_t num_bytes);I think it would be good to figure out how to clear this up before introducing further features that depend on this.
Ok, seems best to close this then. It's almost a year old and needs to be done differently. I think using sat/vB is still a popular request from users and plan to try again a bit later.
ryanofsky
left a comment
There was a problem hiding this comment.
Started review (will update list below with progress). I got a little sidetracked on the cfeerate constructor names, but overall this seems like a very nice, well thought out PR
- c4b288d policy: add CFeeRate::FromSatB/FromBtcKb constructors (1/12)
- c72083c wallet: update rpcwallet to CFeeRate named constructors (2/12)
- 5ed3ca1 policy: allow CFeeRate::ToString to not append fee rate units (3/12)
- aee3571 core_io: Add ValueFromFeeRate helper (4/12)
- 0479268 test: add ValueFromFeeRate/CFeeRate unit tests (5/12)
- 8e863e3 wallet: introduce setfeerate, an improved settxfee in sat/vB (6/12)
- 529bfc1 test: add setfeerate functional coverage in wallet_create_tx.py (7/12)
- c907f15 test: add setfeerate functional coverage in wallet_bumpfee.py (8/12)
- d87f0f3 test: update functional tests from settxfee to setfeerate (9/12)
- c594a00 wallet: update settxfee help (10/12)
- 3725a3f wallet, test: make settxfee a hidden rpc (11/12)
- 1002e2d wallet, test: upgradewallet follow-ups (12/12)
|
Thanks! Huge apologies, since the merge of #21786 that resolved some of these things differently (at a lower level), the approach of the initial refactoring commits is obsolete and being changed. I need to push an update and have been intending to do so for some time. 🙏 |
Github-Pull: bitcoin#20391 Rebased-From: 8e863e3 (diff-minimised)
Github-Pull: bitcoin#20391 Rebased-From: 529bfc1
Github-Pull: bitcoin#20391 Rebased-From: c907f15
Was: test: update functional tests from settxfee to setfeerate Github-Pull: bitcoin#20391 Rebased-From: d87f0f3 (partial)
Was: test: update functional tests from settxfee to setfeerate Github-Pull: bitcoin#20391 Rebased-From: d87f0f3 (partial)
Github-Pull: bitcoin#20391 Rebased-From: 8e863e3 (diff-minimised)
Github-Pull: bitcoin#20391 Rebased-From: 529bfc1
Github-Pull: bitcoin#20391 Rebased-From: c907f15
Was: test: update functional tests from settxfee to setfeerate Github-Pull: bitcoin#20391 Rebased-From: d87f0f3 (partial)
Continuing the work on issue #19543, this proposes a
setfeerateRPC in sat/vB and makessettxfeea hidden RPC per the plan described in #20484 (comment).