Skip to content

Enable goldenFileComparator fix for on-device integration tests.#163157

Merged
auto-submit[bot] merged 2 commits intoflutter:masterfrom
matanlurey:enable-matchesGoldenFile-fix
Feb 13, 2025
Merged

Enable goldenFileComparator fix for on-device integration tests.#163157
auto-submit[bot] merged 2 commits intoflutter:masterfrom
matanlurey:enable-matchesGoldenFile-fix

Conversation

@matanlurey
Copy link
Contributor

@matanlurey matanlurey commented Feb 12, 2025

Closes #143299.
Closes #160043.

A website update is work-in-progress here to add more context for users: flutter/website#11716.

@github-actions github-actions bot added a: tests "flutter test", flutter_test, or one of our tests framework flutter/packages/flutter repository. See also f: labels. f: integration_test The flutter/packages/integration_test plugin labels Feb 12, 2025
jonahwilliams
jonahwilliams approved these changes Feb 12, 2025
View reviewed changes
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matanlurey
Copy link
Contributor Author

matanlurey commented Feb 12, 2025

@Piinks Added you as a reviewer more as an FYI since this is a quasi-breaking change.

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Feb 12, 2025
@auto-submit
Copy link
Contributor

auto-submit bot commented Feb 13, 2025

autosubmit label was removed for flutter/flutter/163157, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@matanlurey
Copy link
Contributor Author

matanlurey commented Feb 13, 2025

This is blocked by google3, which I believe has its own custom implementation that is now being overwritten.

Investigating.

@matanlurey
Copy link
Contributor Author

matanlurey commented Feb 13, 2025

This is correctly implemented externally; note that:

flutter/packages/flutter_tools/lib/src/test/flutter_platform.dart

Lines 206 to 210 in c4d53f1

if (integrationTest) {
buffer.write('''
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
''');
}

Binding.ensureInitialized() happens quite early:

flutter/packages/flutter_tools/lib/src/test/flutter_platform.dart

Line 250 in c4d53f1

return () => test_config.testExecutable(_testMain);

... and if a test_config is specified, it takes precedence.

For example:

flutter/examples/api/test/flutter_test_config.dart

Lines 9 to 12 in c4d53f1

Future<void> testExecutable(FutureOr<void> Function() testMain) {
// Enable golden file testing using Skia Gold.
return flutter_goldens.testExecutable(testMain, namePrefix: 'api');
}

... which uses:

flutter/packages/flutter_goldens/lib/flutter_goldens.dart

Line 111 in c4d53f1

await testMain();

Note again that (this is confusing) testMain() is called last, after the comparator is implicitly set by Binding.ensureInitialized(), so it's kosher.

@github-actions github-actions bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Feb 13, 2025
jonahwilliams
jonahwilliams approved these changes Feb 13, 2025
View reviewed changes
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@auto-submit
Copy link
Contributor

auto-submit bot commented Feb 13, 2025

autosubmit label was removed for flutter/flutter/163157, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@auto-submit auto-submit bot added this pull request to the merge queue Feb 13, 2025
Merged via the queue into flutter:master with commit 4ff2dea Feb 13, 2025
143 checks passed
@flutter-dashboard flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 13, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
6e309ef 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 13, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
ef8df8c 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 14, 2025
Closed
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 14, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
e0c9186 
…ests. (flutter/flutter#163157)
@Piinks Piinks mentioned this pull request Feb 14, 2025
Closed
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 15, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
db500fe 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 15, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
d94a5e9 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 16, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
d5516f7 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 16, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
a967d0c 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 17, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
74dcec1 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 17, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
4411020 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 17, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
734f228 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
ca9f8ef 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
a3002c6 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
6ef152a 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
0e75b38 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
7c4205b 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
564570a 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
217e702 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
f9d0988 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
681d0e4 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
d00dd19 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
2c048cf 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
cafd96e 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
08b4de3 
…ests. (flutter/flutter#163157)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Piinks Piinks Awaiting requested review from Piinks

1 more reviewer

@jonahwilliams jonahwilliams jonahwilliams approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

a: tests "flutter test", flutter_test, or one of our tests f: integration_test The flutter/packages/integration_test plugin framework flutter/packages/flutter repository. See also f: labels. tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Milestone

No milestone

2 participants

@matanlurey @jonahwilliams