Tags: trusts-stack-network/tsn
Tags
v2.3.6: anti-spam middleware, fork-work recovery, auto-wipe disabled,…
… snapshot auto-trigger fix
Anti-spam middleware on sync routes
- Per-IP escalating ban (1h / 6h / 24h) for peers presenting outdated
X-TSN-Version, wrong X-TSN-Network, or wrong X-TSN-Genesis headers.
- Ban triggers after 3 offenses in a sliding window; WARN log is
deduplicated per-IP on a 5-min window to prevent log flood.
- Missing headers pass through (internal HTTP callers do not always set
them). Outbound /tip and /blocks requests now include Network + Genesis
headers so peer-to-peer sync self-identifies the chain.
Fork-work recovery (blockchain.rs::calculate_chain_work)
- When the walk back from a fork tip cannot reach a known-work ancestor
(block missing from LRU+DB), the previous version returned a wildly
under-estimated accumulated difficulty, causing longer forks to be
rejected as REJECT LESS_WORK. The fix adds a bounded prefix estimate
drawn from local cumulative_work at a plausible fork_base height,
guarded by MAX_REORG_DEPTH so long-range attacks are still rejected.
Auto-wipe removed from watchdog paths
- The watchdog used to wipe the local chain on height-stuck, peers-far-
ahead, solo-fork, and resync-loop signals. Each path is now log-only.
Operator triggers re-sync manually via /admin/force-resync. This stops
the cascade of self-destructive wipes that followed transient mesh
instabilities and re-imported stale snapshots.
No wipe on rollback-below-finalization (sync.rs)
- When a peer proposes a fork that would require rolling past our
finalized height, we now reject the peer instead of wiping our chain.
Finalized blocks are canonical; the peer is wrong.
Snapshot auto-trigger fixed
- The guard `tip_h <= max_reorg + interval` skipped the first interval
and the `prev_eligible` crossing check never fired after restart above
the boundary. Replaced with `tip_h < max_reorg + interval` and rely on
the atomic last_snapshot_auto_trigger as the single-fire guard.
- Net effect: first snapshot now fires at h=1100 as intended, not h=2100.
Snapshot cache staleness
- Invalidate the HTTP snapshot cache not only when it is ahead of the
chain (rollback case) but also when it is more than 500 blocks behind
the tip. Without this, a stale cache from a previous stuck period
could be served to fast-syncing peers and cause cascading wipes.
MINIMUM_VERSION bumped to 2.3.6
- Community miners stuck on 2.3.3/2.3.4/2.3.5 will be rejected at the
HTTP sync layer. Auto-update should pull 2.3.6 within 5 minutes if
reachable.
394 library tests pass.
v2.3.5: testnet-v5 chain reset + HTTP accept-queue fix + snapshot Git… …Hub mirror This release resets the chain to testnet-v5 after a chain split on April 19 left multiple nodes on incompatible forks. Nodes on pre-v2.3.5 are rejected by the new MINIMUM_VERSION; obsolete blockchain/ directories are auto-wiped on boot when the genesis hash no longer matches; wallets on the old testnet have their notes archived and the scan cursor reset. Network identity ---------------- - NETWORK_NAME bumped from tsn-testnet-v4 to tsn-testnet-v5. - MINIMUM_VERSION bumped from 2.2.0 to 2.3.5 so pre-v2.3.5 peers are refused. - Genesis block binds its timestamp to u16::from_be_bytes(sha256(NETWORK_NAME)[..2]) so every future testnet rename produces a fresh genesis hash automatically. timestamp is part of the PoW header hash (state_root is not), and the u16 range keeps genesis safely in the past so real mined blocks stay monotonic. - EXPECTED_GENESIS_HASH locked to the deterministic value produced by this build (dadfa2a3..). The `print_genesis_hash` test in src/main.rs prints the hash for future bumps; it is #[ignore]-marked so it never runs by default. - Snapshot manifest chain_id now reads from config::NETWORK_NAME instead of the hardcoded "tsn-mainnet". Auto-wipe gracieux ------------------ - cmd_node catches the "Genesis hash mismatch" error returned by ShieldedBlockchain::open and, when observed, wipes <data_dir>/blockchain and <data_dir>/snapshots before retrying. Wallet DB is left alone — the wallet code handles obsolescence itself. Wallet obsolescence ------------------- - wallet_db schema v2 adds `network_name TEXT` to scan_state with an idempotent ALTER TABLE migration. New `network_name()`, `set_network_name()`, and `archive_for_network_reset()` methods. - ShieldedWallet::open detects a recorded network that differs from config::NETWORK_NAME and archives all notes + tx_history, resets the scan cursor, and records the current network. Keys are preserved, so the user's address stays the same across testnet resets. - Pre-v2.3.5 wallets (empty network_name) get stamped with the current network on first touch without archiving. cmd_balance 429 behaviour ------------------------- - `pre_validate_orphan_positions` now takes a `best_effort` flag. cmd_balance calls it with best_effort=true so a user running `./tsn balance` against a rate-limited node bails after two consecutive 429 errors instead of waiting out minutes of exponential backoff. cmd_send keeps the strict path. - 100ms pacing delay is added between every position query so a single wallet stays under the seed's sync rate limiter (200 rps burst 400). HTTP accept-queue fix (from v2.3.4 hotfix) ------------------------------------------ - The P2P NewBlock mempool cleanup introduced in v2.3.4 held mempool.write() + blockchain.read() on the event loop, which under external /tip polling load starved axum's accept loop and caused CLOSE-WAIT accumulation at ~5000/hour per seed. Cleanup now runs on tokio::task::spawn_blocking and short-circuits when the mempool is empty (the common case on seeds), so the blockchain read-lock is not taken during every gossip block event. - Real stress validation: local relay under ab -c 80 for 180s, receiving live P2P blocks — 12k rps sustained, 0 CLOSE-WAIT accumulated. Snapshot GitHub mirror ---------------------- - auto_snapshot_export now publishes signed snapshots to trusts-stack-network/tsn-snapshots as releases tagged `snapshot-<height>`, each carrying snapshot.tar.gz + manifest.json. Publishing is gated on TSN_SNAPSHOT_GH_TOKEN env var (empty → silent no-op), so only seeds configured as publishers push mirrors. - Retention keeps only the 10 most recent `snapshot-` releases; older releases and their git tags are deleted via the GitHub REST API. - Upload runs on a spawned task so the P2P-critical auto-snapshot path returns immediately; a slow or failing GitHub API cannot stall chain operation. Tests ----- - 394 library + 13 binary tests pass. - `print_genesis_hash` (ignored-by-default) prints deterministic config. - Live relay on this host boots cleanly on testnet-v5, chain_info reports the expected genesis hash, stress test confirms the HTTP fix.
v2.3.4: P2P mempool cleanup + snapshot disk persistence + wallet orph… …an display P2P NewBlock path now drops confirmed transactions and spent nullifiers from the mempool, matching the HTTP receive_block behaviour. This fixes a stuck mempool on miners that only ingest blocks via GossipSub (observed as "Nullifier already spent" loops while other miners extended the chain). Auto snapshots are now persisted to <data_dir>/snapshots/ with a 24h retention policy, so they survive process restarts instead of only living in RAM. cmd_balance pre-validates unspent note witnesses against the node's Merkle tree and displays Total / Spendable / Stuck so users see orphan notes without having to attempt a send. Consolidation wait revert 600s -> 180s now that the P2P mempool cleanup fix keeps mempool state fresh across all propagation paths. Tests: 394 lib + 13 bin (including two new persist_snapshot_to_disk round-trip tests). Smoke tested against the live network as a light client; accepts new P2P blocks cleanly.
v2.3.3: pre-validate unspent note witnesses before send
Adds pre_validate_orphan_positions, an async helper called at the top
of cmd_send that queries the node for every unspent note's Merkle leaf
and flags the ones whose stored pq_commitment no longer matches. The
returned set seeds both auto_consolidate's and the final send's
bad_positions HashSet, so orphan notes are excluded from the very first
batch instead of being discovered round-by-round via proof failure.
This addresses the root cause of the v2.3.1 observation that notes
become silently orphan after chain reorgs: the wallet scan stores a
commitment that is correct at scan time, but a later reorg can replace
the block at that height with a different one, leaving a commitment in
the wallet that no longer matches the server's Merkle tree leaf at the
same position. The wallet has no reorg detection, so these notes stay
in the unspent set until the next send attempts to use them.
The proper long-term fix is a reorg-aware partial re-scan driven off
tip-hash comparison at scan time. This release ships the reactive
counterpart that runs before each send — much cheaper to implement and
sufficient to unblock users whose wallets have accumulated orphan dust.
Behaviour
- At the top of cmd_send, after loading the wallet and filtering
already-spent nullifiers, the new helper queries
/witness/v2/position/<pos> for every unspent note in sequence. It
uses get_with_429_backoff so a rate-limited node does not break the
validation. The returned HashSet contains only positions where the
wallet's stored pq_commitment disagrees with a real server leaf.
- A line is printed showing the spendable balance after subtracting
orphan notes, e.g. "Balance 1149.9990 TSN" on one line and
"Spendable 460 TSN (5 orphan note(s) excluded, 689.9990 TSN stuck)"
on the next. This replaces the previous silent behaviour where the
user saw the full balance and could not explain why send kept
failing.
- If spendable is below the requested amount+fee, the send bails
immediately with a clear error pointing to the rescan menu.
- auto_consolidate gets a new initial_bad_positions argument which it
merges into its internal bad_positions HashSet at the top of the
function, then behaves as before.
- The final send selection loop in cmd_send seeds its
final_bad_positions with the same set so it never re-selects an
orphan for the final transaction.
Verification
End-to-end test on the live testnet-v4 chain with two wallets.
wallet-chainb.json, 10 notes all orphan after prior multi-round
consolidation experiments:
Balance 3035.9930 TSN
Validating 10 note witness(es)... done (10 checked, 10 orphan(s) detected)
Spendable 0 TSN (10 orphan(s) excluded, 3035.99 TSN stuck)
Error: Insufficient spendable balance after excluding 10 orphan
note(s): have 0 TSN, need 100.001 TSN.
Total: 1 s.
wallet-fresh.json, 17 notes with 5 orphans from mid-consolidation
fallout:
Balance 1149.9990 TSN
Validating 17 note witness(es)... done (17 checked, 5 orphan(s) detected)
Spendable 460 TSN (5 orphan(s) excluded, 689.99 TSN stuck)
Notes 5 selected (230 TSN, change 29.99 TSN)
Proof generating... done
Submit confirmed (relayed to 5 seeds)
TX fc2d91efa033f57774223a48751006d7fa72afc017b41be33a969557f72e413d
Total: 6 s. Destination wallet balance verified.
No retry loop, no proof attempt on an orphan, no change to
consensus, schema, or migration. All 11 resolve_pq_commitment unit
tests still pass.
v2.3.2: wallet rescan actually deletes rows + metrics port auto-fallb… …ack + actionable wallet-lock error Three small but user-visible fixes, all in code paths that previously misled users about their real state. 1. Wallet rescan now actually clears the database. ShieldedWallet::clear_notes used to only clear the in-memory notes Vec and reset last_scanned_height, then save() called insert_notes_batch with an empty slice. insert_notes_batch inserts, it does not delete — so the SQLite wallet.db kept every note that was there before. The interactive "Rescan wallet" menu item and the POST /wallet/rescan HTTP endpoint both reported success while leaving the DB untouched. clear_notes now calls db.clear_notes() (DELETE FROM notes) when a SQLite backend is attached. 2. Metrics server auto-falls-back when port 9090 is taken. Running a miner and a relay on the same host — a common local testing setup — used to log "Failed to start metrics server: Address already in use" on the second process. The node still worked, but the error was noisy and implied something was broken. Now the metrics server tries ports 9090 through 9099 in order, uses the first free one, and logs a single warn if all ten are taken. 3. cmd_send gives an actionable error when the wallet is locked. Previously WalletLock::acquire inside cmd_send did a blocking flock, so `./tsn send` against a wallet currently held by the user's own miner would hang forever with no output. cmd_send now uses try_acquire and bails immediately with a message that names the likely cause (concurrent miner on the same wallet file) and points to the fix (run the miner on a dedicated wallet). Cargo.toml version bumped 2.3.1 to 2.3.2. No consensus change, no schema change, no new migration. All 11 resolve_pq_commitment unit tests still pass. Explorer miner attribution from the backlog is intentionally not in this release — a proper fix requires exposing miner identity at the consensus layer (new field on CoinbaseTransaction) and is not something that fits in a point release. Wallet scan cannot detect chain reorgs and will keep storing commitments that later become stale at their position. This manifests as notes being silently marked orphan by the v2.3.1 orphan-skip at send time. A proper reorg-aware wallet scan is tracked for a future release.
v2.3.1: wallet Merkle witness orphan detection + HTTP 429 backoff Main goal: fix the multi-round auto-consolidation flow so that notes that became orphan after a reorg (stored pq_commitment no longer matches the server's Merkle tree leaf at the same position) no longer abort the whole send. They are now detected early, excluded from the batch, and the send retries with a fresh selection. Changes in src/main.rs resolve_pq_commitment: rewritten with an explicit four-case truth table (stored only / server only / agreeing / disagreeing). The disagreeing case now returns an ORPHANED_NOTE error that carries the offending position so callers can skip it. The placeholder case (server returns all-zeros leaf in a fast-sync blind zone) still trusts the wallet's stored value. send_single_tx: collects orphan positions during witness resolution and bails once at the end with a structured ORPHANED_NOTE_POSITIONS=p1,p2:.. marker instead of giving up on the first bad note. auto_consolidate + cmd_send final selection: now keep a HashSet of known-bad positions and rebuild the batch (or re-greedy the final selection) after each orphan error, until a fully clean batch proves. The final send has a bounded retry loop so a pathologically corrupt wallet can not spin forever. New helpers get_with_429_backoff and post_json_with_429_backoff retry HTTP requests on status 429 with exponential backoff (1s, 2s, 4s, 8s, 16s, 32s, then bail). Applied to the critical paths of send_single_tx: /witness/v2/position/<pos> fetch and /tx/v2 submit. Other polling paths already had their own sleep loops and are unaffected. Eleven unit tests in tests:: cover the new resolve_pq_commitment truth table, the orphan marker, the placeholder case, and the legacy fast-sync case. Changes in src/network/api.rs New admin endpoint POST /admin/mempool/purge. Loopback-only via ConnectInfo (not spoofable x-forwarded-for). Accepts a 64-hex tx hash and removes the tx from v1 or v2 mempool, automatically clearing its nullifiers from pending_nullifiers so the originating wallet can resubmit. Responds with a structured JSON describing what was removed. Changes in src/network/mempool.rs add_v2 now emits a warn! log on every reject path (hash already in v1 mempool, hash already in v2 mempool, nullifier conflicts with pending tx). Previously the hash-dup and nullifier-conflict paths were silent, making it impossible to diagnose "conflicts with pending" reports from users. End-to-end validation on testnet-v4 chain: Fresh wallet mined 79 notes / 3634 TSN. cmd_send 2500 TSN triggered auto-consolidation. Six consolidation rounds confirmed on chain with four orphan positions excluded mid-round, then the final 400 TSN send from a non-consolidating path reached the destination wallet in a single tx 5cc95fdcfb9bb9b5e923b81e5aba11cf4bcde26e481cb823b0cd0144abea6d9d Destination wallet balance verified at 400.0000 TSN (1 note). Known issue, deferred to v2.3.2 The wallet scan for v2 self-send outputs stores a pq_commitment that does not match the leaf the chain places at the same Merkle tree position. These notes are now skipped cleanly by the fix in this release, so they no longer break cmd_send, but they remain on the wallet as unspendable dust until the scan itself is fixed. A root-cause audit of the scan's pq_randomness / pq_commitment derivation is scheduled for v2.3.2. Cargo.toml version bumped 2.3.0 to 2.3.1. No consensus change, no schema change, no new migration.
v2.2.0: wallet rewrite — SQLite persistence, atomic writes, file locking - Replace wallet.json with crash-safe SQLite WAL database - Fix balance-reset-on-restart bug (concurrent process race condition) - Atomic file writes (write tmp then rename) - Advisory file locking (flock) to prevent concurrent corruption - Automatic migration from wallet.json to wallet.db - WalletService (Arc<Mutex>) for thread-safe wallet access - 5 new wallet API endpoints (/wallet/balance, history, address, scan, rescan) - Graceful shutdown with WAL checkpoint flush - MINIMUM_VERSION raised to 2.2.0 - EXPECTED_GENESIS_HASH locked for testnet-v4 - NETWORK_NAME: tsn-testnet-v4 Tested: 386/386 tests pass, migration verified on real wallets, restart persistence confirmed on 6-node live network.
v2.1.6: sync recovery, block confirmation, address command Fixes: - Auto-detect and reset corrupted chain state on startup (sanity check) - Auto-reset when reorg fails in fast-sync blind zone (prevents infinite loop) - Clear stale blocks from DB during chain reset (prevents poisoned work calc) - Patch D exception: allow reset when cumulative_work is at genesis level Features: - Block confirmation feedback: "Potential mined block" then CONFIRMED/ORPHANED - Console `address` command to display mining wallet address - MINIMUM_VERSION bumped to 2.1.6 Maintenance: - Full English codebase (comments, errors, metrics, docs)
PreviousNext