Mainnet release diff 09142019#1
Open
rlan35 wants to merge 23 commits intomainnet_release_09112019from
Open
Conversation
Signed-off-by: Leo Chen <[email protected]>
Fix explorer node stuck issue
add rpc call to resend missing cross shard receipt
…enesis [cleanup] remove is_genesis flag
Signed-off-by: Leo Chen <[email protected]>
[nodetype] add nodetype to replace is_explorer
…cking; use sleep instead of ticker
fix beacon sync channel blocking issue
Signed-off-by: Leo Chen <[email protected]>
Fix the beacon committee check
-d download only Signed-off-by: Leo Chen <[email protected]>
-T node_type support node type (validator/explorer) Signed-off-by: Leo Chen <[email protected]>
Signed-off-by: Leo Chen <[email protected]>
-i shard_id specify shard_id, this is applicable only to explorer node Signed-off-by: Leo Chen <[email protected]>
…script_support Non validator node script support
chaosma
reviewed
Sep 14, 2019
| cxReceipts, err := node.Blockchain().ReadCXReceipts(uint32(i), newBlock.NumberU64(), newBlock.Hash(), false) | ||
| if err != nil || len(cxReceipts) == 0 { | ||
| utils.Logger().Warn().Err(err).Uint32("ToShardID", uint32(i)).Int("numCXReceipts", len(cxReceipts)).Msg("[BroadcastCXReceipts] No ReadCXReceipts found") | ||
| go node.BroadcastCXReceiptsWithShardID(newBlock, commitSig, commitBitmap, uint32(i)) |
There was a problem hiding this comment.
after double check, no need to add closure or j:=i
rlan35
commented
Sep 14, 2019
| if *isExplorer { | ||
| startMsg = "==== New Explorer Node ====" | ||
| } | ||
| startMsg := fmt.Sprintf("==== New %s Node ====", strings.Title(*nodeType)) |
Owner
Author
There was a problem hiding this comment.
TODO: keep the original log
rlan35
commented
Sep 14, 2019
| return 0, false | ||
| } | ||
| tx := txs[int(index)] | ||
| if tx.ShardID() == tx.ToShardID() || blk.Header().ShardID() != tx.ShardID() { |
Owner
Author
There was a problem hiding this comment.
TODO(chao): add comment for this logic.
rlan35
commented
Sep 14, 2019
| cxReceipts, err := node.Blockchain().ReadCXReceipts(uint32(i), newBlock.NumberU64(), newBlock.Hash(), false) | ||
| if err != nil || len(cxReceipts) == 0 { | ||
| utils.Logger().Warn().Err(err).Uint32("ToShardID", uint32(i)).Int("numCXReceipts", len(cxReceipts)).Msg("[BroadcastCXReceipts] No ReadCXReceipts found") | ||
| go node.BroadcastCXReceiptsWithShardID(newBlock, commitSig, commitBitmap, uint32(i)) |
Owner
Author
There was a problem hiding this comment.
TODO(chao): add closure or j := i
rlan35
commented
Sep 14, 2019
| utils.Logger().Info().Uint32("ToShardID", toShardID).Msg("[BroadcastCXReceiptsWithShardID] ReadCXReceipts and MerkleProof Found") | ||
|
|
||
| groupID := p2p.ShardID(toShardID) | ||
| go node.host.SendMessageToGroups([]p2p.GroupID{p2p.NewGroupIDByShardID(groupID)}, host.ConstructP2pMessage(byte(0), proto_node.ConstructCXReceiptsProof(cxReceipts, merkleProof, block.Header(), commitSig, commitBitmap))) |
Owner
Author
There was a problem hiding this comment.
TODO(EK): check whether SendMessageToGroups is blocking or not.
rlan35
commented
Sep 14, 2019
| go node.host.SendMessageToGroups([]p2p.GroupID{p2p.NewGroupIDByShardID(groupID)}, host.ConstructP2pMessage(byte(0), proto_node.ConstructCXReceiptsProof(cxReceipts, merkleProof, newBlock.Header(), commitSig, commitBitmap))) | ||
| sig := nextHeader.LastCommitSignature() | ||
| bitmap := nextHeader.LastCommitBitmap() | ||
| go node.BroadcastCXReceiptsWithShardID(blk, sig[:], bitmap, toShardID) |
Owner
Author
There was a problem hiding this comment.
TODO(chao): use only one "go" routine.
rlan35
commented
Sep 14, 2019
| } | ||
|
|
||
| // BroadcastCXReceiptsWithShardID broadcasts cross shard receipts to given ToShardID | ||
| func (node *Node) BroadcastCXReceiptsWithShardID(block *types.Block, commitSig []byte, commitBitmap []byte, toShardID uint32) { |
Owner
Author
There was a problem hiding this comment.
TODO(chao): should be called with thread-safety in mind.
rlan35
commented
Sep 14, 2019
| } | ||
| } | ||
|
|
||
| node.BroadcastMissingCXReceipts() |
Owner
Author
There was a problem hiding this comment.
TODO(chao): explain why this line.
rlan35
commented
Sep 14, 2019
| } | ||
| if node.beaconSync.GetActivePeerNumber() == 0 { | ||
| utils.Logger().Info().Msg("no peers; bootstrapping beacon sync config") | ||
| peers, err := node.SyncingPeerProvider.SyncingPeers(0) |
Owner
Author
There was a problem hiding this comment.
TODO(EK): explain the magic "0" number.
rlan35
commented
Sep 14, 2019
| if [ -e $BINDIR/$bin ]; then | ||
| $AWSCLI s3 cp $BINDIR/$bin s3://${PUBBUCKET}/$FOLDER/$bin --acl public-read | ||
| else | ||
| echo "!! MISSGIN $bin !!" |
rlan35
pushed a commit
that referenced
this pull request
Nov 22, 2019
rlan35
pushed a commit
that referenced
this pull request
Nov 29, 2019
* [reward] Add Accumulator of rewards in DB * [reward] Update Accumulator after checking all block rewards to state * [reward] Address PR comments #1 * [reward] Make fresh base 18 block reward always * [reward] Remove dead method on fakeChainReader * [committee] Adjust logging of prev,next committee on each usage of updateconsensusinformation * [reward] Set block accumulator only once to 0 when flip to staking epoch * [reward] Separate out condition to write block reward to 0 once past staking epoch (fix mistake) * [reward] lint * [reward] Initial write of reward accumulator needs lower-level first usage * [reward] Give block reward stake adjustment in right denomination
rlan35
added a commit
that referenced
this pull request
Mar 2, 2020
…aconchain (harmony-one#2253) * [double-sign] Commit changes in consensus needed for double-sign * [double-sign] Leader captures when valdator double signs, broadcasts to beaconchain * [slash] Add quick iteration tool for testing double-signing * [slash] Add webhook example * [slash] Add http server for hook to trigger double sign behavior * [double-sign] Use bin/trigger-double-sign to cause a double-sign * [double-sign] Full feedback loop working * [slash] Thread through the slash records in the block proposal step * [slash] Compute the slashing rate * [double-sign] Generalize yaml malicious for many keys * [double-sign][slash] Modify data structures, verify via webhook handler * [slash][double-sign] Find one address of bls public key signer, seemingly settle on data structures * [slash] Apply to state slashing for double signing * [slash][double-sign] Checkpoint for working code that slashes on beaconchain * [slash] Keep track of the total slash and total reporters reward * [slash] Dump account state before and after the slash * [slash] Satisfy Travis * [slash][state] Apply slash to the snapshot at beginning of epoch, now need to capture also the new delegates * [slash] Capture the unique new delegations since snapshot as well * [slash] Filter undelegation by epoch of double sign * [slash] Add TODO of correctness needed in slash needs on off-chain data * [rpc] Fix closure issue on shardID * [slash] Add delegator to double-sign testing script * [slash] Expand crt-validator.sh with commenting printfs and make delegation * [slash] Finish track payment of leftover slash debt after undelegation runs out * [slash] Now be explicit about error wrt delegatorSlashApply * [slash] Capture specific sanity check on slash paidoff * [slash] Track slash from undelegation piecemeal * [slash][delegation] Named slice types, .String() * [slash] Do no RLP encode twice, once is enough * [slash] Remove special case of validators own delegation * [slash] Refactor approach to slash state application * [slash] Begin expanding out Verify * [slash] Slash on snapshot delegations, not current * [slash] Fix Epoch Cmp * [slash] Third iteration on slash logic * [slash] Use full slash amount * [slash] More log, whitespace * [slash] Remove Println, add log * [slash] Remove debug Println * [slash] Add record in unit test * [slash] Build Validator snapshot, current. Fill out slash record * [slash] Need to get RLP dump of a header to use in test * [slash] Factor out double sign test constants * [slash] Factor out common for validator, stub out slash application, finish out deserialization setup * [slash] Factor out data structure creation because of var lexical scoping * [slash] Seem to have pipeline of unit test e2e executing * [slash] Add expected snitch, slash amounts * [slash] Checkpoint * [slash] Unit test correctly checks case of validator own stake which could drop below 1 ONE in slashing * [config] add double-sign testnet config (#1) Signed-off-by: Leo Chen <[email protected]> * [slash] Commit for as is code & data of current dump.json * [slash] Order of state operation not correct in test, hence bad results, thank you dlv * [slash] Add snapshot state dump * [slash] Pay off slash of validator own delegation correctly * [slash] Pay off slash debt with special case for min-self * [slash] Pass first scenario conclusively * [slash] 2% slash passes unit test for own delegation and external * [slash] Parameterize unit test to easily test .02 vs .80 slash * [slash] Handle own delegation correctly at 80% slash * [slash] Have 80% slash working with external delegator * [slash] Remove debug code from slash * [slash] Adjust Apply signature, test again for 2% slash * [slash] Factor out scenario in testing so can test 2% and 80% at same time * [slash] Correct balance deduction on plan delegation * [slash] Mock out ChainReader for TestVerify * [slash] Small surface area interface, now feedback loop for verify * [slash] Remove development json * [slash] trigger-double-sign consumes yaml * [slash] Remove dead code * [slash][test] Factor ValidatorWrapper into scenario * [slash][test] Add example from local-testing dump - caution might be off * [slash] Factor out mutation of slashDebt * [slash][test] Factor out tests so can easily load test-case from bytes * [slash] Fix payment mistake in validator own delegation wrt min-self-delgation respected * [slash] Satisfy Travis * [slash] Begin cleanup of PR * [slash] Apply slash from header to Finalize via state processor * [slash] Productionize code, Println => logs; adjust slash picked in newblock * [slash] Need pointer for rlp.Decode * [slash] ValidatorInformation use full wrapper * Fix median stake * [staking] Adjust MarshalJSON for Validator, Wrapper * Refactor offchain data commit; Make block onchain/offchain commit atomic (harmony-one#2279) * Refactor offchain data; Add epoch to ValidatorSnapshot * Make block onchain/offchain data commit atomically * [slash][committee] Set .Active to false on double sign, do not consider banned or inactive for committee assignment * [effective] VC eligible.go * [consensus] Redundant field in printf * [docker] import-ks for a dev account * [slash] Create BLS key for dockerfile and crt-validator.sh * [slash][docker] Easy deployment of double-sign testing * [docker] Have slash work as single docker command * [rpc] Fix median-stake RPC * [slash] Update webhook with default docker BLS key * [docker][slash] Fresh yaml copy for docker build, remove dev code in main.go * [slash] Remove helper binary, commented out code, change to local config * [params] Factor out test genesis value * Add shard checking to Tx-Pool & correct blacklist (harmony-one#2301) * [core] Fix blacklist & add shardID check * [staking + node + cmd] Fix blacklist & add shardID check * [slash] Adjust to PR comments part 1 * [docker] Use different throw away funded account * [docker] Create easier testing for delegation with private keys * [docker] Update yaml * [slash] Remove special case for slashing validator own delegation wrt min-self-delegate * [docker] Install nano as well * [slash] Early error if banned * [quorum] Expose earning account in decider marshal json * Revert "Refactor offchain data commit; Make block onchain/offchain commit atomic (harmony-one#2279)" This reverts commit 9ffbf68. * [slash] Add non-sanity check way to update validator * [reward] Increase percision on percentage in schedule * [slash] Adjust logs * [committee] Check eligibility of validator before doing sanity check * [slash] Update docker * [slash] Move create validator script to test * [slash] More log * [param] Make things faster * [slash][off-chain] Clear out slashes from pending in writeblockwithstate * [cross-link] Log is not error, just info * [blockchain] Not necessary to guard DeletePendingSlashingCandidates * [slash][consensus] Use plain []byte for signature b/c bls.Sign has private impl fields, rlp does not encode that * [slash][test] Use faucet as sender, assume user imported * [slash] Test setup * [slash] reserve error for real error in logs * [slash][availability] Apply availability correct, bump signing count each block * [slash][staking] Consider banned field in sanity check, pay snitch only half of what was actually slashed * [slash] Pay as much as can * [slash] use right nowAmt * [slash] Take away from rewards as well * [slash] iterate faster * [slash] Remove dev based timing * [slash] Add more log, sanity check incoming slash records, only count external for slash rate * [availability][state] Adjust signature of ValidatorWrapper wrt state, filter out for staked validators, correct availaibility measure on running counters * [availability] More log * [slash] Simply pre slash erra slashing * [slash] Remove development code * [slash] Use height from recvMsg, todo on epoch * [staking] Not necessary to touch LastEpochInCommittee in staking_verifier * [slash] Undo ds in endpoint pattern config * [slash] Add TODO and log when delegation becomes 0 b/c slash debt payment * [slash] Abstract staked validators from shard.State into type, set slash rate based BLSKey count Co-authored-by: Leo Chen <[email protected]> Co-authored-by: flicker-harmony <[email protected]> Co-authored-by: Rongjian Lan <[email protected]> Co-authored-by: Daniel Van Der Maden <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.