Skip to content

test: create assert_not_equal util#29500

Merged
glozow merged 1 commit intobitcoin:masterfrom
kevkevinpal:testFrameworkUtilsOne
Apr 1, 2025
Merged

test: create assert_not_equal util#29500
glozow merged 1 commit intobitcoin:masterfrom
kevkevinpal:testFrameworkUtilsOne

Conversation

@kevkevinpal
Copy link
Contributor

In the functional tests there are lots of cases where we assert != which we now swap with assert_not_equal to be more readable

This is motivated/uses logic from this PR which was closed #28528
This partially helps #23119

I've broken it up to just assert_not_equal to keep the PR smaller as suggested in #28528 (comment)

I can create follow up PR's if this is wanted

@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 28, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29500.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK hodlinator, ryanofsky, janb84
Concept ACK theStack, rkrux
Approach ACK BrandonOdiwuor
Stale ACK vasild

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #32155 (miner: timelock the coinbase to the mined block's height by darosior)
  • #31492 (Execute Discover() when bind=0.0.0.0 or :: is set by andremralves)
  • #29415 (Broadcast own transactions only via short-lived Tor or I2P connections by vasild)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot DrahtBot added the Tests label Feb 28, 2024
axizqayum686

This comment was marked as spam.

@Empact
Copy link
Contributor

Empact commented Feb 28, 2024

Would be nice to structure this as a scripted diff: one commit where you add the function and requires, another where you convert the assert !=s.

That would ensure programmatically that you hit all of the cases.

@kevkevinpal
Copy link
Contributor Author

kevkevinpal commented Mar 2, 2024

added scripted diff in dc087f1 with the following script
git grep -l "assert.*!=" ./test/functional

I will have to amend the commit as I see more spots I need to modify with the assert_not_equal util

@kevkevinpal kevkevinpal force-pushed the testFrameworkUtilsOne branch 6 times, most recently from 57ecac9 to 6ee4175 Compare March 2, 2024 20:35
@theStack
Copy link
Contributor

theStack commented Mar 2, 2024

Concept ACK, thanks for picking this up.

@kevkevinpal kevkevinpal force-pushed the testFrameworkUtilsOne branch 5 times, most recently from 8d6da40 to ad1e879 Compare March 2, 2024 21:47
@brunoerg
Copy link
Contributor

brunoerg commented Mar 4, 2024

There are some places you are using assert_not_equal but not importing it and other ones you're importing it twice. Check CI.

Running Unit Tests for Test Framework Modules
Traceback (most recent call last):
  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_runner.py", line 906, in <module>
    main()
  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_runner.py", line 539, in main
    run_tests(
  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_runner.py", line 580, in run_tests
    test_framework_tests.addTest(unittest.TestLoader().loadTestsFromName("test_framework.{}".format(module)))
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/unittest/loader.py", line 137, in loadTestsFromName
    module = __import__(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/ci_container_base/test/functional/test_framework/address.py", line 14, in <module>
    from .script import (
  File "/ci_container_base/test/functional/test_framework/script.py", line 14, in <module>
    from .key import TaggedHash, tweak_add_pubkey, compute_xonly_pubkey
  File "/ci_container_base/test/functional/test_framework/key.py", line 16, in <module>
    from test_framework.crypto import secp256k1
  File "/ci_container_base/test/functional/test_framework/crypto/secp256k1.py", line 316, in <module>
    G = GE.lift_x(0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ci_container_base/test/functional/test_framework/crypto/secp256k1.py", line 257, in lift_x
    y = (FE(x)**3 + 7).sqrt()
         ^^^^^
  File "/ci_container_base/test/functional/test_framework/crypto/secp256k1.py", line 41, in __init__
    assert_not_equal(den, 0)
    ^^^^^^^^^^^^^^^^
NameError: name 'assert_not_equal' is not defined

@kevkevinpal kevkevinpal force-pushed the testFrameworkUtilsOne branch 3 times, most recently from 1c0929e to a6a6210 Compare March 5, 2024 04:21
@kevkevinpal kevkevinpal force-pushed the testFrameworkUtilsOne branch from 6b3260e to dcf9a45 Compare November 13, 2024 02:39
@kevkevinpal
Copy link
Contributor Author

yes @rkrux you are right if I want to use a scripted diff then all the changes must be done using the script

I can remove the scripted diff and just squash them into a single diff since right now the script is getting very large already if that is preferred by others

@fanquake
Copy link
Member

@kevkevinpal what's the status of this?

@vasild
Copy link
Contributor

vasild commented Feb 21, 2025

Almost ACK dcf9a45aab4c4a7f9f900d5dafcdfe0e9a598a38, modulo squash the two commits into one and rebase.

@kevkevinpal
Copy link
Contributor Author

@kevkevinpal what's the status of this?

Just rebased and addressed @vasild comment


Almost ACK dcf9a45, modulo squash the two commits into one and rebase.

Just rebased and squashed the commits, I removed the scripted-diff because it was getting more difficult to review the diff than the actual code. But now all the changes are in d9cb032


if you do grep -nr assert\ .*!= ./test/functional/ there should only be one result

assert len(ss) == 175 - (in_type == SIGHASH_ANYONECANPAY) * 49 - (out_type != SIGHASH_ALL and out_type != SIGHASH_SINGLE) * 32 + (annex is not None) * 32 + scriptpath * 37

which we don't need to add assert_not_equal to

Copy link
Contributor

@hodlinator hodlinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK d9cb032ec1aa7000fc3f5b9fa48405f269879497

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should appear after assert_equal alphabetically.

Same in test/functional/feature_dbcrash.py and others.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use f-strings:

- Use `f'{x}'` for string formatting in preference to `'{}'.format(x)` or `'%s' % x`.

nit: And possibly rephrase:

Suggested change
raise AssertionError("%s == %s %s" % (str(thing1), str(thing2), str(error_message)))
raise AssertionError(f"Both values are {thing1}{f', {error_message}' if error_message else ''}")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be more eager to ACK if code introduced by the PR followed bitcoin/test/functional/README.md as noted above. If you don't want to change the phrasing towards what's in the nit-part that's fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went ahead an used your suggestion in be71af3

it now looks like this

2025-03-27T23:14:10.629000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/mnt/shared_drive/DEVDIR/bitcoin/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
  File "/mnt/shared_drive/DEVDIR/bitcoin/./build_dir/test/functional/p2p_blockfilters.py", line 98, in run_test
    assert_not_equal(stale_block_hash, stale_block_hash, "node 0 chain did not reorganize")
  File "/mnt/shared_drive/DEVDIR/bitcoin/test/functional/test_framework/util.py", line 81, in assert_not_equal
    raise AssertionError(f"Both values are {thing1}, {f'{error_message}' if error_message else ''}")
AssertionError: Both values are 05ba865bb920e2ff5763411a9dd3c225ace8ff4393469bb25acffd56df1bfb6e, node 0 chain did not reorganize

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread #29500 (comment):
Thanks for taking my suggestion!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Feels a bit weird to not have assert_equal() here and other directly adjacent places. Could touch up adjacent lines like this in separate commit?

Copy link
Contributor

@hodlinator hodlinator Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In thread #29500 (comment):

Agree this is a valid argument against the PR. It is very seldom useful to print the values when they are equal.

I still prefer assert_not_equal() for:

  • Symmetry with assert_equal().
  • Distinguishing itself from assert which is skipped when running Python with -O.

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK d9cb032ec1aa7000fc3f5b9fa48405f269879497

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK d9cb032ec1aa7000fc3f5b9fa48405f269879497. Seems like a nice change, though needs to be rebased. I think this change is good because it makes tests more readable, prints more information on failures, avoids misusing the assert keyword, and makes the test framework api more consistent internally and consistent with other frameworks.

@kevkevinpal
Copy link
Contributor Author

rebased to 29fa2b9

Comment on lines 79 to 81
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the point of the error_message, given that it is unused and highly fragile:

  • It is not a named arg, nor type-safe, so someone writing assert_not_equal("a", "b", "c") or assert_not_equal(1, 2, 3) will be wrong and confusing at best.
  • It is not used anywhere else in this file for another assert

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread #29500 (comment):

It would be good if all assert_ functions supported error messages explaining why the condition should never fail, or what could be wrong if it has failed. (Built-in assert supports it too).

To avoid mixing the error message up with a value to be checked, one could do:

Suggested change
def assert_not_equal(thing1, thing2, error_message=""):
if thing1 == thing2:
raise AssertionError(f"Both values are {thing1}, {f'{error_message}' if error_message else ''}")
def assert_not_equal(thing1, thing2, *, message=None):
if thing1 == thing2:
raise AssertionError(f"Both values are {thing1}{f', {message}' if message else ''}")
  • * enforces named parameters from that point.
  • It would be good to shorten the name for callers.
  • ", " is only printed if there is a message.
  • Feels more correct to use None as default value.

The traceback in this case does print the source code line, so a comment on the same line could be sufficient. It depends on how the error is presented. Example failure:

2025-03-28T09:58:37.993000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
  File "/home/hodlinator/bitcoin/build/test/functional/p2p_blockfilters.py", line 98, in run_test
    assert_not_equal(main_block_hash, stale_block_hash, message="node 0 chain did not reorganize")
  File "/home/hodlinator/bitcoin/test/functional/test_framework/util.py", line 81, in assert_not_equal
    raise AssertionError(f"Both values are {thing1}{f', {message}' if message else ''}")
AssertionError: Both values are 1b0ec65a9941d57790524ff90cc1cb93d2f8e70680c1c7486d9edf934d96a041, node 0 chain did not reorganize

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert_not_equal(main_block_hash, stale_block_hash, message="node 0 chain did not reorganize")

I'd say it is already self-explanatory to see assert_not_equal(main_block_hash, stale_block_hash) and the message "AssertionError: Both values are (the same)". It should be clear that the chains are the same, when they should be different.

Even more so, given that the test logs before the assert: self.log.info("Reorg node 0 to a new chain.").

Unique and descriptive error messages make sense when it is the only piece of information given (for example in the context of an init error during startup of the program). Given that no other asserts in this file give the option, it would be best to defer this to a future where it is actually widely needed. However, given the extensive context in tests and test failures, I doubt it will ever be widely needed. Until then, it is just bike-shedding and overhead that has to be maintained and reviewed.

Copy link
Contributor

@hodlinator hodlinator Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"AssertionError: Both values are (the same)"

Don't know if you mean literally "Both values are (the same)" or f"Both values are {thing1}". Find the latter clearly more useful; providing clues of cause of the failure, making it easier to reproduce.


What it comes down to with custom error messages is being able to directly communicate with whoever is troubleshooting a failure. If one doesn't even need to go to the logs to see what happened before, I consider it a small win. But agree it is bikeshedding-prone, so I would be okay with removing it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know if you mean literally "Both values are (the same)" or f"Both values are {thing1}". Find the latter clearly more useful; providing clues of cause of the failure, making it easier to reproduce.

Agree. Sorry for being unclear, I meant f"Both values are {thing1}" when I said "(the same)".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: #29500 (comment)

I do like error_message parameter idea, fwiw. I could see these making tests more readable if they started being used for more important or more confusing checks. Tests are often written with some important checks directly related to the thing being tested, and other less important checks to see if other things are consistent. Having a place to indicate what it actually means when a check fails could be helpful for more important checks to distinguish them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The traceback in this case does print the source code line, so a comment on the same line could be sufficient. It depends on how the error is presented. Example failure:

This is an annoying draw back that the stack trace won't contain the line in the test that's in error, but instead the line in the assert_not_equal helper. In Rust, one can decorate the helper function with the track_caller macro. I don't see anything that is similar for C++ sadly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're in Python here though. I spent a couple of hours with an LLM in mid-February to brute force this embryo:
master...hodlinator:bitcoin:2025/02/assert_ergonomics

Output examples

Current assert_equal()-behavior:

2025-03-28T21:15:36.379000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 178, in main
    self.run_test()
  File "/home/hodlinator/bitcoin/./build/test/functional/wallet_multisig_descriptor_psbt.py", line 80, in run_test
    assert_equal(
  File "/home/hodlinator/bitcoin/test/functional/test_framework/util.py", line 77, in assert_equal
    raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(0 == 123)

With first commit, chomping off util.py traceback entry you complained about:

2025-03-28T21:16:19.814000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 178, in main
    self.run_test()
  File "/home/hodlinator/bitcoin/./build/test/functional/wallet_multisig_descriptor_psbt.py", line 80, in run_test
    assert_equal(
AssertionError: not(0 == 123)

With second commit, replacing another traceback entry with proper context:

2025-03-28T21:17:39.494000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 178, in main
    self.run_test()
AssertionError: not(0 == 123)
Failing expression: /home/hodlinator/bitcoin/./build/test/functional/wallet_multisig_descriptor_psbt.py:80 - 83
        assert_equal(
            123 if (False and
            True) else 0,
            123)

The code needs more work, should be more defensive, forgot to output function name (run_test). And I wouldn't bet on it being accepted anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinion, but I think we shouldn't spend too much time on the error message explaining the context because the traceback could contain too little context, or on the traceback containing too much (redundant) context by default. A full log of a test failure could be thousands of lines, so one additional line, even if could be redundant in a strict sense should be fine. Also, looking at the historic failures (in Python) at https://github.com/bitcoin/bitcoin/issues?q=is%3Aissue%20%20label%3A%22CI%20failed%22%20, it was never(?) an issue to figure out what failed where. The problem usually is to figure out why something failed, and how to reproduce and fix it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind having an error message being logged and I do agree with the following points:

It is not a named arg, nor type-safe, so someone writing assert_not_equal("a", "b", "c") or assert_not_equal(1, 2, 3) will be wrong and confusing at best.

Unique and descriptive error messages make sense when it is the only piece of information given (for example in the context of an init error during startup of the program).

As I see from the usages of assert_not_equal, there is only 1 case where the error message is passed out of ~77 total usages. I believe a named argument for the error message could help in avoiding unintended usages of this utility function.

Copy link
Contributor

@hodlinator hodlinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK be71af3cc0b0bcb7d917cc6f2e5fda119f1b1bd6

This change adds the missing opposite of assert_equal() and decreases usage of built-in assert for performing always-on checks during tests.

Comment on lines 79 to 81
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread #29500 (comment):

It would be good if all assert_ functions supported error messages explaining why the condition should never fail, or what could be wrong if it has failed. (Built-in assert supports it too).

To avoid mixing the error message up with a value to be checked, one could do:

Suggested change
def assert_not_equal(thing1, thing2, error_message=""):
if thing1 == thing2:
raise AssertionError(f"Both values are {thing1}, {f'{error_message}' if error_message else ''}")
def assert_not_equal(thing1, thing2, *, message=None):
if thing1 == thing2:
raise AssertionError(f"Both values are {thing1}{f', {message}' if message else ''}")
  • * enforces named parameters from that point.
  • It would be good to shorten the name for callers.
  • ", " is only printed if there is a message.
  • Feels more correct to use None as default value.

The traceback in this case does print the source code line, so a comment on the same line could be sufficient. It depends on how the error is presented. Example failure:

2025-03-28T09:58:37.993000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
  File "/home/hodlinator/bitcoin/build/test/functional/p2p_blockfilters.py", line 98, in run_test
    assert_not_equal(main_block_hash, stale_block_hash, message="node 0 chain did not reorganize")
  File "/home/hodlinator/bitcoin/test/functional/test_framework/util.py", line 81, in assert_not_equal
    raise AssertionError(f"Both values are {thing1}{f', {message}' if message else ''}")
AssertionError: Both values are 1b0ec65a9941d57790524ff90cc1cb93d2f8e70680c1c7486d9edf934d96a041, node 0 chain did not reorganize

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread #29500 (comment):
Thanks for taking my suggestion!

Copy link
Contributor

@janb84 janb84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK be71af3

  • Code review ✅
  • Build & tested ✅

I agree with @hodlinator on the import order NIT — let's keep it consistent and maintain clean, high-quality code. #29500 (comment)

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK be71af3cc0b0bcb7d917cc6f2e5fda119f1b1bd6. Just rebased and added f-string since last review.

Comment on lines 79 to 81
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: #29500 (comment)

I do like error_message parameter idea, fwiw. I could see these making tests more readable if they started being used for more important or more confusing checks. Tests are often written with some important checks directly related to the thing being tested, and other less important checks to see if other things are consistent. Having a place to indicate what it actually means when a check fails could be helpful for more important checks to distinguish them.

Copy link
Contributor

@rkrux rkrux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK be71af3cc0b0bcb7d917cc6f2e5fda119f1b1bd6

Requesting changes mainly because an additional comma is printed in case of an assertion error when the error message is not passed, which can be confusing for the reader later.

2025-03-31T07:28:44.958000Z TestFramework (INFO): Test fundrawtxn with locked wallet and hardened derivation
2025-03-31T07:28:46.455000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/Users/rkrux/projects/bitcoin/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
    ~~~~~~~~~~~~~^^
  File "/Users/rkrux/projects/bitcoin/./build/test/functional/wallet_fundrawtransaction.py", line 141, in run_test
    self.test_locked_wallet()
    ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/rkrux/projects/bitcoin/./build/test/functional/wallet_fundrawtransaction.py", line 665, in test_locked_wallet
    assert_not_equal(fundedTx["changepos"], 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rkrux/projects/bitcoin/test/functional/test_framework/util.py", line 81, in assert_not_equal
    raise AssertionError(f"Both values are {thing1}, {f'{error_message}' if error_message else ''}")
AssertionError: Both values are 1, 
2025-03-31T07:28:46.512000Z TestFramework (INFO): Not stopping nodes as test failed. The dangling processes will be cleaned up later.

Comment on lines 79 to 81
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind having an error message being logged and I do agree with the following points:

It is not a named arg, nor type-safe, so someone writing assert_not_equal("a", "b", "c") or assert_not_equal(1, 2, 3) will be wrong and confusing at best.

Unique and descriptive error messages make sense when it is the only piece of information given (for example in the context of an init error during startup of the program).

As I see from the usages of assert_not_equal, there is only 1 case where the error message is passed out of ~77 total usages. I believe a named argument for the error message could help in avoiding unintended usages of this utility function.

@kevkevinpal
Copy link
Contributor Author

Updated to f3d4b6d

Added named variable for error_message and also made the assert able to take more than two variables to compare to each other. The error will now say
Two or more values equal to <value>: <error_message> or
Two or more values equal to <value>

if no error_message provided

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with:

assert_not_equal(stale_block_hash, stale_block_hash, "node 0 chain did not reorganize")

Output was:

2025-04-01T08:15:47.847000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/home/hodlinator/b2/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
  File "/home/hodlinator/b2/build/test/functional/p2p_blockfilters.py", line 66, in run_test
    assert_not_equal(stale_block_hash, stale_block_hash, "node 0 chain did not reorganize")
  File "/home/hodlinator/b2/test/functional/test_framework/util.py", line 85, in assert_not_equal
    raise AssertionError(
AssertionError: Two or more values equal to 775ea4a5e5005376a49f2da6d20ce0449fbd627d766c8fc3108d22db0d41e116

This indicates that the message wasn't recognized as such. Naming the parameter has the desired effect:

assert_not_equal(stale_block_hash, stale_block_hash, error_message="node 0 chain did not reorganize")

A possible step forward would be to update callers to name the parameter. As I said before, I think the name should be shortened.

(Phantom space)

Suggested change
def assert_not_equal(*things, error_message=""):
def assert_not_equal(*things, error_message=""):

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved in 7bb83f6

I went back to using two params and made error_message a named parameter that cannot be used as a positional argument

Comment on lines 79 to 87
Copy link
Member

@maflcko maflcko Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still wrong (the error message is not a named arg). Also, adding support for more values is unused and dead code.

My recommendation would be to just use assert_not_equal(thing1, thing2):. Any other attempt pushed to this pull so far was wrong or brittle.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair point and thanks for the review

I've updated to 7bb83f6

  • we are now using the error message (f"Both values are {thing1}{f', {error_message}' if error_message else ''}"
  • we only take two positional arguments and a named error_message arg which is used in p2p_blockfilters

In the functional tests there are lots of cases where we assert != which
this new util will replace, we also are adding the imports and the new assertion
Copy link
Contributor

@hodlinator hodlinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK 7bb83f6

Remaining reservations from #29500 (comment) (sorry for broken record):

  • Current push still uses falsy error_message="" with the if error_message, instead of clear error_message=None
  • error_message= at call sites is quite verbose.

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 7bb83f6. Only change since last review is fixing error message formatting and passing it as a keyword argument

Copy link
Contributor

@janb84 janb84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-ACK 7bb83f6

Copy link
Contributor

@rkrux rkrux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post-merge ACK 7bb83f6

Comment on lines -671 to +672
assert ret != 0 # Exit code must indicate failure
assert_not_equal(ret, 0) # Exit code must indicate failure
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message arg can/could have been used here instead of the comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.