Skip to content

[Main] Write to the zerocoinDB in batches#636

Merged
Mrs-X merged 1 commit intoPIVX-Project:masterfrom
Fuzzbawls:2018_zerocoin-batchwrite
Jun 28, 2018
Merged

[Main] Write to the zerocoinDB in batches#636
Mrs-X merged 1 commit intoPIVX-Project:masterfrom
Fuzzbawls:2018_zerocoin-batchwrite

Conversation

@Fuzzbawls
Copy link
Collaborator

instead of using a separate write operation for each and every bit of
data that needs to be flushed to disk, utilize leveldb's batch writing
capability.

The primary area of improvement this offers is when reindexing the zerocoinDB (-reindexzerocoin), which went from needing multiple hours on my system to mere minutes.

Secondary improvement area is in ConnectBlock() when multiple zerocoin transactions are involved.

instead of using a separate write operation for each and every bit of
data that needs to be flushed to disk, utilize leveldb's batch writing
capability.
@Fuzzbawls
Copy link
Collaborator Author

thinking about this further...I could probably add a guard in ConnectBlock() that only calls the WriteBatch() function if the vSpends/vMints vectors have a size greater than zero...just to save a couple cycles in processing.

@Fuzzbawls Fuzzbawls added the Needs Release Notes Placeholder tag for anything needing mention in the "Notable Changes" section of release notes label Jun 19, 2018
@Warrows
Copy link

Warrows commented Jun 19, 2018

Are you going to add this guard, or is the PR ready for test and review?

@Fuzzbawls
Copy link
Collaborator Author

@Warrows can be tested as-is. The internal batch write functions already don't do anything except a LogPrint() if the supplied vector has a size of zero.

Copy link

@Warrows Warrows left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK cd672cd. I can confirm the vast performance improvement on reindex.

Copy link

@Mrs-X Mrs-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK (and yes, it's indeed faster ;-) ) and merging...

@Mrs-X Mrs-X merged commit cd672cd into PIVX-Project:master Jun 28, 2018
Mrs-X added a commit that referenced this pull request Jun 28, 2018
cd672cd [Main] Write to the zerocoinDB in batches (Fuzzbawls)

Tree-SHA512: 031d41bf5d09b2f3636e191e0de5fbf8062cd967f76a25fe2705dceaf4e9a65875c394af5fb6e2ad08739a4acd3fa7945eff4594a11040e5aa9cf2b93ca04cd8
@ghost ghost removed the review label Jun 28, 2018
@Fuzzbawls Fuzzbawls added this to the 3.1.1 milestone Jul 5, 2018
Fuzzbawls added a commit to Fuzzbawls/PIVX that referenced this pull request Jul 6, 2018
instead of using a separate write operation for each and every bit of
data that needs to be flushed to disk, utilize leveldb's batch writing
capability.

Github-Pull: PIVX-Project#636
Rebased-From: cd672cd
@Fuzzbawls Fuzzbawls removed Needs Release Notes Placeholder tag for anything needing mention in the "Notable Changes" section of release notes labels Feb 23, 2019
@Fuzzbawls Fuzzbawls deleted the 2018_zerocoin-batchwrite branch January 10, 2020 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants