Clean up calculations of pcoinsTip memory usage#10133
Merged
laanwj merged 3 commits intobitcoin:masterfrom Apr 5, 2017
Merged
Conversation
Contributor
|
utack 99b814f. |
sipa
reviewed
Mar 31, 2017
src/validation.cpp
Outdated
Member
There was a problem hiding this comment.
The changes in the rollback test are never flushed, so there is no need for compensating for a peak?
Contributor
Author
There was a problem hiding this comment.
agreed. assumed coins was built on top of pcoinstip, missed that. will change.
Since we are more accurately measuring pcoinsTip peak usage at twice the current in dynamic usage, it makes sense to double the default (this will lead to the same effective usage and peak usage as previously). We should also double the buffer used to avoid flushing if above 90% but still sufficient space remaining.
Always leave a reasonable buffer of 50MB for usage from newly connected block (once over 50%) and increase the high water mark buffer to 200MB.
Contributor
Author
|
Fixed sipa's issue. After discussion I decided to remove the lowering of maxmempool until we are more confident that maxes sense. I also made the flushing buffer a bit smarter. |
Member
|
utACK 1b55e07 We should tag this for 0.14.1 |
Member
|
Benchmarking. |
laanwj
added a commit
that referenced
this pull request
Apr 5, 2017
1b55e07 Make threshold for flushing more conservative. (Alex Morcos) f33afd3 Lower default memory footprint slightly (Alex Morcos) 5b95a19 Make pcoinsTip memory calculations consistent (Alex Morcos) Tree-SHA512: d0061138596cf89008397b8729d9b25293938b1ad454cc99a6fe2f6210e94f76dfa78a8f0fce4c1ba3efec4e742a9c1a3ab26676a4a8346d3e7c3055d032669b
laanwj
pushed a commit
that referenced
this pull request
Apr 5, 2017
Since we are more accurately measuring pcoinsTip peak usage at twice the current in dynamic usage, it makes sense to double the default (this will lead to the same effective usage and peak usage as previously). We should also double the buffer used to avoid flushing if above 90% but still sufficient space remaining. Github-Pull: #10133 Rebased-From: 5b95a19
Member
codablock
pushed a commit
to codablock/dash
that referenced
this pull request
Oct 24, 2017
1b55e07 Make threshold for flushing more conservative. (Alex Morcos) f33afd3 Lower default memory footprint slightly (Alex Morcos) 5b95a19 Make pcoinsTip memory calculations consistent (Alex Morcos) Tree-SHA512: d0061138596cf89008397b8729d9b25293938b1ad454cc99a6fe2f6210e94f76dfa78a8f0fce4c1ba3efec4e742a9c1a3ab26676a4a8346d3e7c3055d032669b
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In #10126 we changed the calculation of when to flush so that the requested dbcache memory limit would be respected even by the peak usage occurring at flush.
Without doubling the default dbcache limit, this would lead to an effective default of half the previous which I believe is too low.
First commit doubles the default and applies the DB_PEAK_USAGE_FACTOR in 2 other places.
The debug logging of the usage still reports the current usage.
The second commit lowers the defaults slightly to ensure 0.14.1 has a lower memory footprint than 0.13.2.
The third commit makes the delayed flushing on large dbcaches a bit less aggressive, so the limit is less likely to be violated in practice.