Fee estimation: avoid serving stale fee estimate#27622
Fee estimation: avoid serving stale fee estimate#27622glozow merged 4 commits intobitcoin:masterfrom
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
a2ad5e4 to
0d2329a
Compare
0d2329a to
d24f874
Compare
aecd92e to
3a9fee2
Compare
3a9fee2 to
b02e3fc
Compare
pinheadmz
left a comment
There was a problem hiding this comment.
concept ACK
This should solve the stale fee issue, the test is clever and effective. I ensured the test fails without the patch and also checked the test failed when the mtime was set < 12 hours. Reviewed code and left some notes / questions
ef2ee18 to
473946b
Compare
473946b to
1315c9f
Compare
5f79a92 to
cd9828f
Compare
de80c10 to
948edc9
Compare
d2b39e0 test: ensure old fee_estimate.dat not read on restart and flushed (ismaelsadeeq) cf219f2 tx fees, policy: read stale fee estimates with a regtest-only option (ismaelsadeeq) 3eb241a tx fees, policy: do not read estimates of old fee_estimates.dat (ismaelsadeeq) 5b886f2 tx fees, policy: periodically flush fee estimates to fee_estimates.dat (ismaelsadeeq) Pull request description: Fixes bitcoin#27555 The issue arises when an old `fee_estimates.dat` file is sometimes read during initialization. Or after an unclean shutdown, the latest fee estimates are not flushed to `fee_estimates.dat`. If the fee estimates in the old file are old, they can cause transactions to become stuck in the mempool. This PR ensures that nodes do not use stale estimates from the old file during initialization. If `fee_estimates.dat` has not been updated for 60 hours or more, it is considered stale and will not be read during initialization. To avoid having old estimates, the `fee_estimates.dat` file will be flushed periodically every hour. As mentioned bitcoin#27555 > "The immediate improvement would be to store fee estimates to disk once an hour or so to reduce the chance of having an old file. From there, this case could probably be detected, and refuse to serve estimates until we sync." In addition, I will follow-up PR to persist the `mempoolminfee` across restarts. ACKs for top commit: willcl-ark: ACK d2b39e0 instagibbs: reACK bitcoin@d2b39e0 glozow: ACK d2b39e0. One nit if you follow up. Tree-SHA512: 4f6e0c296995d0eea5cf80c6aefdd79b7295a6a0ba446f2166f32afc105fe4f831cfda1ad3abd13c5c752b4fbea982cf4b97eaeda2af1fd7184670d41edcfeec
This reduces chances of having old estimates in fee_estimates.dat. Github-Pull: bitcoin#27622 Rebased-From: 5b886f2
Old fee estimates could cause transactions to become stuck in the mempool. This commit prevents the node from using stale estimates from an old file. Github-Pull: bitcoin#27622 Rebased-From: 3eb241a
If -acceptstalefeeestimates option is passed stale fee estimates can now be read when operating in regtest environments. Additionally, this commit updates all declarations of the CBlockPolicyEstimator class to include a the second constructor variable. Github-Pull: bitcoin#27622 Rebased-From: cf219f2
This commit adds tests to ensure that old fee_estimates.dat files are not read and that fee_estimates are periodically flushed to the fee_estimates.dat file. Additionaly it tests the -regtestonly option -acceptstalefeeestimates. Github-Pull: bitcoin#27622 Rebased-From: d2b39e0
…` option is only supported on regtest chain ee5a036 test: ensure acceptstalefeeestimates is supported only on regtest chain (ismaelsadeeq) 22d5d4b tx fees, policy: doc: update and delete unnecessary comment (ismaelsadeeq) Pull request description: This PR Follow up comments from [#27622](bitcoin/bitcoin#27622) It test that the new `regtest-only` option `acceptstalefeeestimates` is not supported on [main, signet and test chains](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235218268), removes an unnecessary [comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235204323), and update fee estimator `MAXFILEAGE` [description comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1233887314). ACKs for top commit: jonatack: ACK ee5a036 glozow: utACK ee5a036 Tree-SHA512: 4755f25b08db62f37614ea768272b12580ee0d481fb7fa339379901a6132c66828777c6747d3fe67490ceace3a6ff248bf13bdf65720f6e5ba8642eb762acd3c
… is only supported on regtest chain ee5a036 test: ensure acceptstalefeeestimates is supported only on regtest chain (ismaelsadeeq) 22d5d4b tx fees, policy: doc: update and delete unnecessary comment (ismaelsadeeq) Pull request description: This PR Follow up comments from [bitcoin#27622](bitcoin#27622) It test that the new `regtest-only` option `acceptstalefeeestimates` is not supported on [main, signet and test chains](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235218268), removes an unnecessary [comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235204323), and update fee estimator `MAXFILEAGE` [description comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1233887314). ACKs for top commit: jonatack: ACK ee5a036 glozow: utACK ee5a036 Tree-SHA512: 4755f25b08db62f37614ea768272b12580ee0d481fb7fa339379901a6132c66828777c6747d3fe67490ceace3a6ff248bf13bdf65720f6e5ba8642eb762acd3c
|
Added to #28487 for backporting to 25.1. |
|
Pushed into #28535 for backporting to 24.x as well. |
This reduces chances of having old estimates in fee_estimates.dat. Github-Pull: bitcoin#27622 Rebased-From: 5b886f2
Old fee estimates could cause transactions to become stuck in the mempool. This commit prevents the node from using stale estimates from an old file. Github-Pull: bitcoin#27622 Rebased-From: 3eb241a
If -acceptstalefeeestimates option is passed stale fee estimates can now be read when operating in regtest environments. Additionally, this commit updates all declarations of the CBlockPolicyEstimator class to include a the second constructor variable. Github-Pull: bitcoin#27622 Rebased-From: cf219f2
This commit adds tests to ensure that old fee_estimates.dat files are not read and that fee_estimates are periodically flushed to the fee_estimates.dat file. Additionaly it tests the -regtestonly option -acceptstalefeeestimates. Github-Pull: bitcoin#27622 Rebased-From: d2b39e0
This reduces chances of having old estimates in fee_estimates.dat. Github-Pull: bitcoin#27622 Rebased-From: 5b886f2
Old fee estimates could cause transactions to become stuck in the mempool. This commit prevents the node from using stale estimates from an old file. Github-Pull: bitcoin#27622 Rebased-From: 3eb241a
Fixes #27555
The issue arises when an old
fee_estimates.datfile is sometimes read during initialization.Or after an unclean shutdown, the latest fee estimates are not flushed to
fee_estimates.dat.If the fee estimates in the old file are old, they can cause transactions to become stuck in the mempool.
This PR ensures that nodes do not use stale estimates from the old file during initialization. If
fee_estimates.dathas not been updated for 60 hours or more, it is considered stale and will not be read during initialization. To avoid
having old estimates, the
fee_estimates.datfile will be flushed periodically every hour. As mentioned #27555In addition, I will follow-up PR to persist the
mempoolminfeeacross restarts.