Skip to content

TST: Restore some tolerances for some arch/platform-specific failures#31343

Open
QuLogic wants to merge 3 commits intomatplotlib:text-overhaulfrom
QuLogic:test-tolerance
Open

TST: Restore some tolerances for some arch/platform-specific failures#31343
QuLogic wants to merge 3 commits intomatplotlib:text-overhaulfrom
QuLogic:test-tolerance

Conversation

@QuLogic
Copy link
Member

@QuLogic QuLogic commented Mar 20, 2026

PR summary

The tolerances on these were increased in various PRs, with notes to reduce/remove them, so they were removed in #30184, but should have been reverted to a smaller value:

  • test_floating_axes.py::test_curvelinear4[png] fails on macOS
  • test_grid_helper_curvelinear.py::test_axis_direction[png] fails on ARM and macOS
  • test_grid_helper_curvelinear.py::test_polar_box[png] fails on Windows, ARM, and macOS

These fail on macOS; there's tolerances on them already, they fail by just a small amount, so increase them:

  • test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[png]
  • test_patheffects.py::test_collection[png]
  • test_axes3d.py::test_scale3d_artists_log[png]

This fails only on AppVeyor (Windows), but not Azure. There is a tolerance there already, so just add it to the list:

  • test_mathtext.py::test_mathtext_rendering

AI Disclosure

None

PR checklist

@QuLogic QuLogic added this to the v3.11.0 milestone Mar 20, 2026
@github-project-automation github-project-automation bot moved this to Waiting for other PR in Font and text overhaul Mar 20, 2026
@QuLogic QuLogic moved this from Waiting for other PR to In Progress in Font and text overhaul Mar 20, 2026
@QuLogic QuLogic force-pushed the test-tolerance branch 2 times, most recently from 9f6c9e9 to 8a451fb Compare March 21, 2026 07:25
@QuLogic QuLogic force-pushed the test-tolerance branch 2 times, most recently from 34a33c2 to 4e264da Compare March 24, 2026 08:25
@QuLogic QuLogic marked this pull request as ready for review March 24, 2026 17:46
@QuLogic
Copy link
Member Author

QuLogic commented Mar 24, 2026

OK, this is now passing.

The failure in test_use14corefonts does point out how backend-specific measurement is useful. PDF/PS support for using the core 14 fonts means it overrides get_text_width_height_descent and returns builtin metrics for those when the option is enabled. Since the line-height-from-font-metrics code doesn't use a backend-specific API for it, it doesn't see that override and is dependent on whether Helvetica is installed or not. So I regenerated it with Helvetica installed, but I've had to add a tolerance for other systems without it.

@QuLogic
Copy link
Member Author

QuLogic commented Mar 25, 2026

OK, it does require a bit of internal trickery, but instead of the last commit, we can instead do #31371.

When outputting files using the "core 14 fonts" (e.g.,
`rcParams['pdf.use14corefonts'] = True`), text will be measured using
our internal AFM files instead of any external files.

While we don't have a full API decided for how to pass full `Text`
objects through backends, add in a small internal helper to allow the
PS/PDF backends to override font height metrics with the AFM files.
@QuLogic QuLogic moved this from In Progress to Ready for Review in Font and text overhaul Mar 26, 2026
@QuLogic
Copy link
Member Author

QuLogic commented Mar 26, 2026

Rebased on top of #31371 and dropped the AFM test change since that PR's approved.

@timhoffm
Copy link
Member

Test failures are a bit getting out of hand, so that I don't dare to merge. We should fix setuptools scm, likely temporarily by pinning <10.0, see #31387.

@github-actions github-actions bot added the CI: Run cygwin Run cygwin tests on a PR label Mar 26, 2026
@QuLogic
Copy link
Member Author

QuLogic commented Mar 26, 2026

OK, I've pinned setuptools-scm for this branch and I think I've got all the places. Unfortunately, since this branch hasn't been merged with main recently, it doesn't have #29281.

@QuLogic
Copy link
Member Author

QuLogic commented Mar 26, 2026

OK, looks like I didn't increase one tolerance enough, and test_stairs_options is failing everywhere. I don't quite understand the latter yet as if I apply that image locally, it doesn't pass for me.

@QuLogic
Copy link
Member Author

QuLogic commented Mar 26, 2026

and test_stairs_options is failing everywhere.

So before the AFM change, the minimum ascent was 15.1953125, and after it's 15.195312499999996 which causes only the A in the legend to shift slightly. I don't think that's too important, so I've just updated the final figure in the other branch and restarted the CI here.

@QuLogic
Copy link
Member Author

QuLogic commented Mar 27, 2026

Trying to re-trigger Azure, which seems to have not run lately.

@QuLogic QuLogic closed this Mar 27, 2026
@github-project-automation github-project-automation bot moved this from Ready for Review to Done in Font and text overhaul Mar 27, 2026
@QuLogic QuLogic reopened this Mar 27, 2026
@github-project-automation github-project-automation bot moved this from Done to In Progress in Font and text overhaul Mar 27, 2026
@QuLogic QuLogic moved this from In Progress to Ready for Review in Font and text overhaul Mar 27, 2026
QuLogic added 2 commits March 27, 2026 00:42
The tolerances on these were increased in various PRs, with notes to
reduce/remove them, so they were removed in matplotlib#30184, but should have been
reverted to a smaller value:

- `test_floating_axes.py::test_curvelinear4[png]` fails on macOS
- `test_grid_helper_curvelinear.py::test_axis_direction[png]` fails on
  ARM and macOS
- `test_grid_helper_curvelinear.py::test_polar_box[png]` fails on
  Windows, ARM, and macOS

These fail on macOS; there's tolerances on them already, they fail by
just a small amount, so increase them:

- `test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[png]`
- `test_patheffects.py::test_collection[png]`
- `test_axes3d.py::test_scale3d_artists_log[png]`

This fails only on AppVeyor (Windows), but not Azure. There is a
tolerance there already, so just add it to the list:

- `test_mathtext.py::test_mathtext_rendering`
This is currently causing warnings at runtime in the editable install,
which breaks almost all tests.
@QuLogic
Copy link
Member Author

QuLogic commented Mar 27, 2026

Increased the tolerance for the test failure on AppVeyor, though it doesn't show on Azure. There's already a tolerance there, so just made it a bit bigger.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Ready for Review

Development

Successfully merging this pull request may close these issues.

3 participants