v2.3.4: P2P mempool cleanup + snapshot disk persistence + wallet orph…

…an display

P2P NewBlock path now drops confirmed transactions and spent nullifiers from
the mempool, matching the HTTP receive_block behaviour. This fixes a stuck
mempool on miners that only ingest blocks via GossipSub (observed as
"Nullifier already spent" loops while other miners extended the chain).

Auto snapshots are now persisted to <data_dir>/snapshots/ with a 24h
retention policy, so they survive process restarts instead of only living
in RAM.

cmd_balance pre-validates unspent note witnesses against the node's Merkle
tree and displays Total / Spendable / Stuck so users see orphan notes
without having to attempt a send.

Consolidation wait revert 600s -> 180s now that the P2P mempool cleanup
fix keeps mempool state fresh across all propagation paths.

Tests: 394 lib + 13 bin (including two new persist_snapshot_to_disk
round-trip tests). Smoke tested against the live network as a light
client; accepts new P2P blocks cleanly.

v2.3.3: pre-validate unspent note witnesses before send

Adds pre_validate_orphan_positions, an async helper called at the top
of cmd_send that queries the node for every unspent note's Merkle leaf
and flags the ones whose stored pq_commitment no longer matches. The
returned set seeds both auto_consolidate's and the final send's
bad_positions HashSet, so orphan notes are excluded from the very first
batch instead of being discovered round-by-round via proof failure.

This addresses the root cause of the v2.3.1 observation that notes
become silently orphan after chain reorgs: the wallet scan stores a
commitment that is correct at scan time, but a later reorg can replace
the block at that height with a different one, leaving a commitment in
the wallet that no longer matches the server's Merkle tree leaf at the
same position. The wallet has no reorg detection, so these notes stay
in the unspent set until the next send attempts to use them.

The proper long-term fix is a reorg-aware partial re-scan driven off
tip-hash comparison at scan time. This release ships the reactive
counterpart that runs before each send — much cheaper to implement and
sufficient to unblock users whose wallets have accumulated orphan dust.

Behaviour

  - At the top of cmd_send, after loading the wallet and filtering
    already-spent nullifiers, the new helper queries
    /witness/v2/position/<pos> for every unspent note in sequence. It
    uses get_with_429_backoff so a rate-limited node does not break the
    validation. The returned HashSet contains only positions where the
    wallet's stored pq_commitment disagrees with a real server leaf.
  - A line is printed showing the spendable balance after subtracting
    orphan notes, e.g. "Balance 1149.9990 TSN" on one line and
    "Spendable 460 TSN (5 orphan note(s) excluded, 689.9990 TSN stuck)"
    on the next. This replaces the previous silent behaviour where the
    user saw the full balance and could not explain why send kept
    failing.
  - If spendable is below the requested amount+fee, the send bails
    immediately with a clear error pointing to the rescan menu.
  - auto_consolidate gets a new initial_bad_positions argument which it
    merges into its internal bad_positions HashSet at the top of the
    function, then behaves as before.
  - The final send selection loop in cmd_send seeds its
    final_bad_positions with the same set so it never re-selects an
    orphan for the final transaction.

Verification

  End-to-end test on the live testnet-v4 chain with two wallets.

  wallet-chainb.json, 10 notes all orphan after prior multi-round
  consolidation experiments:
      Balance 3035.9930 TSN
      Validating 10 note witness(es)... done (10 checked, 10 orphan(s) detected)
      Spendable 0 TSN (10 orphan(s) excluded, 3035.99 TSN stuck)
      Error: Insufficient spendable balance after excluding 10 orphan
      note(s): have 0 TSN, need 100.001 TSN.
      Total: 1 s.

  wallet-fresh.json, 17 notes with 5 orphans from mid-consolidation
  fallout:
      Balance 1149.9990 TSN
      Validating 17 note witness(es)... done (17 checked, 5 orphan(s) detected)
      Spendable 460 TSN (5 orphan(s) excluded, 689.99 TSN stuck)
      Notes 5 selected (230 TSN, change 29.99 TSN)
      Proof generating... done
      Submit confirmed (relayed to 5 seeds)
      TX fc2d91efa033f57774223a48751006d7fa72afc017b41be33a969557f72e413d
      Total: 6 s. Destination wallet balance verified.

  No retry loop, no proof attempt on an orphan, no change to
  consensus, schema, or migration. All 11 resolve_pq_commitment unit
  tests still pass.

v2.3.2: wallet rescan actually deletes rows + metrics port auto-fallb…

…ack + actionable wallet-lock error

Three small but user-visible fixes, all in code paths that previously
misled users about their real state.

1. Wallet rescan now actually clears the database.

   ShieldedWallet::clear_notes used to only clear the in-memory notes
   Vec and reset last_scanned_height, then save() called
   insert_notes_batch with an empty slice. insert_notes_batch inserts,
   it does not delete — so the SQLite wallet.db kept every note that
   was there before. The interactive "Rescan wallet" menu item and the
   POST /wallet/rescan HTTP endpoint both reported success while
   leaving the DB untouched. clear_notes now calls db.clear_notes()
   (DELETE FROM notes) when a SQLite backend is attached.

2. Metrics server auto-falls-back when port 9090 is taken.

   Running a miner and a relay on the same host — a common local
   testing setup — used to log "Failed to start metrics server:
   Address already in use" on the second process. The node still
   worked, but the error was noisy and implied something was broken.
   Now the metrics server tries ports 9090 through 9099 in order, uses
   the first free one, and logs a single warn if all ten are taken.

3. cmd_send gives an actionable error when the wallet is locked.

   Previously WalletLock::acquire inside cmd_send did a blocking
   flock, so `./tsn send` against a wallet currently held by the
   user's own miner would hang forever with no output. cmd_send now
   uses try_acquire and bails immediately with a message that names
   the likely cause (concurrent miner on the same wallet file) and
   points to the fix (run the miner on a dedicated wallet).

Cargo.toml version bumped 2.3.1 to 2.3.2. No consensus change, no
schema change, no new migration. All 11 resolve_pq_commitment unit
tests still pass.

Explorer miner attribution from the backlog is intentionally not
in this release — a proper fix requires exposing miner identity at
the consensus layer (new field on CoinbaseTransaction) and is not
something that fits in a point release.

Wallet scan cannot detect chain reorgs and will keep storing
commitments that later become stale at their position. This
manifests as notes being silently marked orphan by the v2.3.1
orphan-skip at send time. A proper reorg-aware wallet scan is
tracked for a future release.

v2.3.1: wallet Merkle witness orphan detection + HTTP 429 backoff

Main goal: fix the multi-round auto-consolidation flow so that notes that
became orphan after a reorg (stored pq_commitment no longer matches the
server's Merkle tree leaf at the same position) no longer abort the whole
send. They are now detected early, excluded from the batch, and the send
retries with a fresh selection.

Changes in src/main.rs

  resolve_pq_commitment: rewritten with an explicit four-case truth table
  (stored only / server only / agreeing / disagreeing). The disagreeing
  case now returns an ORPHANED_NOTE error that carries the offending
  position so callers can skip it. The placeholder case (server returns
  all-zeros leaf in a fast-sync blind zone) still trusts the wallet's
  stored value.

  send_single_tx: collects orphan positions during witness resolution and
  bails once at the end with a structured ORPHANED_NOTE_POSITIONS=p1,p2:..
  marker instead of giving up on the first bad note.

  auto_consolidate + cmd_send final selection: now keep a HashSet of
  known-bad positions and rebuild the batch (or re-greedy the final
  selection) after each orphan error, until a fully clean batch proves.
  The final send has a bounded retry loop so a pathologically corrupt
  wallet can not spin forever.

  New helpers get_with_429_backoff and post_json_with_429_backoff retry
  HTTP requests on status 429 with exponential backoff (1s, 2s, 4s, 8s,
  16s, 32s, then bail). Applied to the critical paths of send_single_tx:
  /witness/v2/position/<pos> fetch and /tx/v2 submit. Other polling paths
  already had their own sleep loops and are unaffected.

  Eleven unit tests in tests:: cover the new resolve_pq_commitment truth
  table, the orphan marker, the placeholder case, and the legacy
  fast-sync case.

Changes in src/network/api.rs

  New admin endpoint POST /admin/mempool/purge. Loopback-only via
  ConnectInfo (not spoofable x-forwarded-for). Accepts a 64-hex tx hash
  and removes the tx from v1 or v2 mempool, automatically clearing its
  nullifiers from pending_nullifiers so the originating wallet can
  resubmit. Responds with a structured JSON describing what was removed.

Changes in src/network/mempool.rs

  add_v2 now emits a warn! log on every reject path (hash already in v1
  mempool, hash already in v2 mempool, nullifier conflicts with pending
  tx). Previously the hash-dup and nullifier-conflict paths were silent,
  making it impossible to diagnose "conflicts with pending" reports from
  users.

End-to-end validation on testnet-v4 chain:

  Fresh wallet mined 79 notes / 3634 TSN. cmd_send 2500 TSN triggered
  auto-consolidation. Six consolidation rounds confirmed on chain with
  four orphan positions excluded mid-round, then the final 400 TSN send
  from a non-consolidating path reached the destination wallet in a
  single tx 5cc95fdcfb9bb9b5e923b81e5aba11cf4bcde26e481cb823b0cd0144abea6d9d
  Destination wallet balance verified at 400.0000 TSN (1 note).

Known issue, deferred to v2.3.2

  The wallet scan for v2 self-send outputs stores a pq_commitment that
  does not match the leaf the chain places at the same Merkle tree
  position. These notes are now skipped cleanly by the fix in this
  release, so they no longer break cmd_send, but they remain on the
  wallet as unspendable dust until the scan itself is fixed. A root-cause
  audit of the scan's pq_randomness / pq_commitment derivation is
  scheduled for v2.3.2.

Cargo.toml version bumped 2.3.0 to 2.3.1. No consensus change, no schema
change, no new migration.

v2.2.1: fix wallet auto-detection after SQLite migration

v2.2.0: wallet rewrite — SQLite persistence, atomic writes, file locking

- Replace wallet.json with crash-safe SQLite WAL database
- Fix balance-reset-on-restart bug (concurrent process race condition)
- Atomic file writes (write tmp then rename)
- Advisory file locking (flock) to prevent concurrent corruption
- Automatic migration from wallet.json to wallet.db
- WalletService (Arc<Mutex>) for thread-safe wallet access
- 5 new wallet API endpoints (/wallet/balance, history, address, scan, rescan)
- Graceful shutdown with WAL checkpoint flush
- MINIMUM_VERSION raised to 2.2.0
- EXPECTED_GENESIS_HASH locked for testnet-v4
- NETWORK_NAME: tsn-testnet-v4

Tested: 386/386 tests pass, migration verified on real wallets,
restart persistence confirmed on 6-node live network.

v2.1.6: sync recovery, block confirmation, address command

Fixes:
- Auto-detect and reset corrupted chain state on startup (sanity check)
- Auto-reset when reorg fails in fast-sync blind zone (prevents infinite loop)
- Clear stale blocks from DB during chain reset (prevents poisoned work calc)
- Patch D exception: allow reset when cumulative_work is at genesis level

Features:
- Block confirmation feedback: "Potential mined block" then CONFIRMED/ORPHANED
- Console `address` command to display mining wallet address
- MINIMUM_VERSION bumped to 2.1.6

Maintenance:
- Full English codebase (comments, errors, metrics, docs)

v2.1.5 — Mining efficiency, wallet recovery, auto snapshots

TSN v2.1.4 — Post-quantum privacy blockchain

release: v2.1.2 — consensus fix, sync improvements, interactive console

Consensus:
- Reorg uses rollback_to_height with cached state (deterministic, no replay)
- Missing block during replay triggers auto-wipe + re-sync (no more crash)
- Orphan pool evicts oldest instead of rejecting new blocks
- Checkpoint framework: hardcoded enforced, dynamic warning, genesis bypass

Network:
- HTTP relay: fire-and-forget with semaphore (max 8), no more API blocking
- Peer cleanup: ghost peers removed after 5 failures, max 50 HTTP peers
- Sync prioritizes seed nodes for large batch downloads
- Sync yields every 5 blocks to prevent HTTP handler starvation
- receive_block takes reorg_lock (consistent with P2P/watchdog paths)
- /network/status endpoint: server-side seed heights, P2P raw heights

Security:
- Ed25519 release signature now mandatory (unsigned updates rejected)
- Signing key rotated (old key was leaked)

UX:
- Interactive console: id, status, peers, version, difficulty, uptime, help
- Commands displayed in violet to distinguish from logs
- Peer ID shown in color at startup (no more truncated box)

Cleanup:
- ForkChoice dead code removed (836 lines)
- README version badge updated
- MINIMUM_VERSION = 2.1.2