Skip to content

Bookkeeper migration part 1: the cleanups#8445

Merged
rustyrussell merged 29 commits intoElementsProject:masterfrom
rustyrussell:bkpr-part1
Aug 14, 2025
Merged

Bookkeeper migration part 1: the cleanups#8445
rustyrussell merged 29 commits intoElementsProject:masterfrom
rustyrussell:bkpr-part1

Conversation

@rustyrussell
Copy link
Contributor

This does various cleanups in anticipation of moving the audit log of moves into the core.

The next part will do the rest (unfortunately, that's a giant PR because we need to do the bookeeper -> internal migration at the end, and we must do that before anyone records coin movements).

Copy link
Collaborator

@niftynei niftynei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easy build error from CI, otherwise lgtm tho I wish the currency types were justifiable to be left in :)

Include(s) from common/coin_mvt.h duplicated in common/coin_mvt.c:
#include <bitcoin/tx.h>

assert(mvt_tags_valid(tags));

/* We put the *primary* tag first */
for (size_t i = 0; i < NUM_MVT_TAGS; i++) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh this is fun.


enum mvt_tag *new_tag_arr(const tal_t *ctx, enum mvt_tag tag);
/* Convenience macro for creating tag bitmaps */
#define mk_mvt_tags(...) mk_mvt_tags_(__VA_ARGS__, 999)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is cool.

Copy link
Member

@cdecker cdecker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

We can use these to test migrations.

Signed-off-by: Rusty Russell <[email protected]>
Print the error!

Signed-off-by: Rusty Russell <[email protected]>
This means it doesn't have to be a tal ptr.

Signed-off-by: Rusty Russell <[email protected]>
This makes our final balance not match our wallet:
1. We only spend the anchor when we need to boost the commitment tx,
   which we don't always do (sometimes the peer does, sometimes it's
   not worth it).
2. We don't put the UTXO in our wallet, because we don't consider it
   "ours": anyone can spend it after 16 blocks.

We used to use the tag "ignored" for this, but that's overly complex
IMHO.

Signed-off-by: Rusty Russell <[email protected]>
We don't actually set it any more.  The bookkeeper db does a migration
for old anchors.

Signed-off-by: Rusty Russell <[email protected]>
This is how we handle amount_msat and amount_sat everywhere these days, and this
wasn't updated.

Signed-off-by: Rusty Russell <[email protected]>
…cations.

Rather than converting to a generic coin_mvt struct, use these directly in
the notification, which is more explicit.

Signed-off-by: Rusty Russell <[email protected]>
Now we only ever use `struct chain_coin_mvt` or `struct channel_coin_mvt`.

Signed-off-by: Rusty Russell <[email protected]>
The part id is *only* unique within a group.  The payment_hash / partid / groupid tuple is unique.

Signed-off-by: Rusty Russell <[email protected]>
Changelog-Added: Plugins: `coin_movement` notification with `part_id` field now always has `group_id` field.
More readable for me.  Also, change order so we definitely break
compilation on all callers (putting enum before amount).

Signed-off-by: Rusty Russell <[email protected]>
…others.

This means we can keep a pointer to the channel directly, *or* a string.
This avoids gratuitous formatting (on creation) and lookups (later).

Signed-off-by: Rusty Russell <[email protected]>
Make the common fields the first ones, and make part_and_group and
payment_hash const pointers.

Signed-off-by: Rusty Russell <[email protected]>
Prefix MVT_ to them, for clarity.

Signed-off-by: Rusty Russell <[email protected]>
This isn't a robust assumption, so sort them before comparison.

Signed-off-by: Rusty Russell <[email protected]>
Undocumented, but the first tag in the coin_movement notification is
considered the primary tag, and the others are optional.  The
bookkeeper plugin relies on this!

Enforce that this is true, and in the process document in the code which
is the primary tag.

Signed-off-by: Rusty Russell <[email protected]>
Rather than open-coding in json_parse.

Signed-off-by: Rusty Russell <[email protected]>
We're going to store them in the db this way, so I thought I'd see what it looks like if
we lift that interface all the way through.

We use a struct, so that types are checked strictly.

Signed-off-by: Rusty Russell <[email protected]>
…d `extra_tags`.

Signed-off-by: Rusty Russell <[email protected]>
Changelog-Deprecated: JSON-RPC: `coin_movement` notification `tags` array (use `primary_tag` and `extra_tags`).
Changelog-Added: JSON-RPC: `coin_movement` notification `primary_tag` and `extra_tags`.
This is not particularly relevant now (it's always the current time) but will be
useful when we implement the list commands.

Note that timestamp is set to be "u32" in various schemas.  This will
only become a problem on Sun 07 Feb 2106 06:28:15 UTC.  I apologize to
my grandchildren in advance.

Signed-off-by: Rusty Russell <[email protected]>
It's always set, and in fact we assume it is (journal entries are not
internal to lightningd, so we won't see them in
lightningd/notification.c: that comment is misleading).

Signed-off-by: Rusty Russell <[email protected]>
…st functions.

Signed-off-by: Rusty Russell <[email protected]>
Changelog-Deprecated: JSON-RPC: `coin_movement` notification `utxo_txid`, `vout` and `txid` fields (use `utxo` and `spending_txid`).
Changelog-Added: JSON-RPC: `coin_movement` notification `utxo` field.
Changelog-Added: JSON-RPC: `coin_movement` notification `spending_txid` field.
…common/coin_mvt.h

They're scattered and reproduced in many places: unify them.

Signed-off-by: Rusty Russell <[email protected]>
We're going to get rid of this concept, but the main change is that the
account_get_balance API can be drastically simplified:

account_get_credit_debit() accesses the raw fields, never fails, but
returns the a flag which tells us if the account doesn't actually have
any events.

The one place we care about the balance, calculate by hand.  Then
account_get_balance() (and struct account_balance) can simply be
moved to th test.

Subtly, without the "GROUP BY" clause, you always get one row, even if
there are no rows (but the SUM are null).

Signed-off-by: Rusty Russell <[email protected]>
We still output the fields, they're just always the currency of the node.

Signed-off-by: Rusty Russell <[email protected]>
Changelog-Changed: Plugins: `bookkeeper` now explicitly assumes every transaction is in the same currency as the node (true unless you added manually)
bookkeeper used to generate these as channel events, now lightningd does.

We also add a "journal" event, which we will need later too.

Signed-off-by: Rusty Russell <[email protected]>
@rustyrussell rustyrussell disabled auto-merge August 14, 2025 10:19
@rustyrussell rustyrussell merged commit 22b452a into ElementsProject:master Aug 14, 2025
38 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants