Update tests to use Xcode 16.2 and iOS 18.2 Simulator by vashworth · Pull Request #165318 · flutter/flutter

vashworth · 2025-03-17T16:08:56Z

Runs tests on Xcode 16.2 and iOS 18.2. Also updates engine scenario golden files to iOS 18.2 and removes non-impeller (Skia) test files that we no longer use.

All framework tests passing: #148899 (comment)
All engine tests passing: #148906 (comment)

Fixes #148907 and #148957.

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
I signed the CLA.
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or this PR is test-exempt.
I followed the breaking change policy and added Data Driven Fixes where supported.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

vashworth · 2025-03-17T16:10:04Z

dev/devicelab/lib/framework/ios.dart

  final String? iosKey =
      decodeResult.keys.where((String key) => key.contains('iphoneos')).firstOrNull;
  final String? runtimeBuildForSelectedXcode = switch (decodeResult[iosKey]) {
+    {'userOverriddenBuild': final String build} => build,


Xcode 16.2 is weird in that it originally shipped with iOS 18.2 (22C150), but later the "preferredBuild" became 18.3.1 (22D8075). mac_toolchain is still using iOS 18.2 and sets it as the override build. This will make Flutter tests using a simulator use the override version if set.

This was causing a flaky error: https://ci.chromium.org/ui/p/flutter/builders/try/Mac_arm64%20module_test_ios/3351/overview

vashworth · 2025-03-17T16:10:25Z

dev/devicelab/lib/tasks/plugin_tests.dart

          destination: 'platform=macOS',
          configuration: 'Debug',
          testName: 'native_plugin_unit_tests_macos',
+          extraOptions: <String>['-parallel-testing-enabled', 'NO'],


This is potential fix for #164902:

Tests indicate it seem to be effective:
https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20plugin_test_macos/27007/overview
https://ci.chromium.org/ui/p/flutter/builders/try.shadow/Mac%20plugin_test_macos/7/overview
https://ci.chromium.org/ui/p/flutter/builders/try.shadow/Mac%20plugin_test_macos/6/overview
https://ci.chromium.org/ui/p/flutter/builders/try.shadow/Mac%20plugin_test_macos/8/overview

…r/flutter#165318)

Runs tests on Xcode 16.2 and iOS 18.2. Also updates engine scenario golden files to iOS 18.2 and removes non-impeller (Skia) test files that we no longer use. All framework tests passing: flutter#148899 (comment) All engine tests passing: flutter#148906 (comment) Fixes flutter#148907 and flutter#148957. - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord].  [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md

…r/flutter#165318)

Runs tests on Xcode 16.2 and iOS 18.2. Also updates engine scenario golden files to iOS 18.2 and removes non-impeller (Skia) test files that we no longer use. All framework tests passing: flutter#148899 (comment) All engine tests passing: flutter#148906 (comment) Fixes flutter#148907 and flutter#148957. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord].  [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md

Update tests to use Xcode 16.2 and iOS 18.2 Simulator

c8ce060

github-actions bot added a: text input Entering text in a text field or keyboard related problems engine flutter/engine related. See also e: labels. platform-web Web applications specifically labels Mar 17, 2025

vashworth mentioned this pull request Mar 17, 2025

Update tests to use Xcode 16.2 and iOS 18.2 Simulator #164786

Closed

9 tasks

vashworth commented Mar 17, 2025

View reviewed changes

remove TODO

6824779

vashworth marked this pull request as ready for review March 17, 2025 20:16

vashworth requested a review from matanlurey as a code owner March 17, 2025 20:16

vashworth requested a review from loic-sharma March 17, 2025 20:16

matanlurey approved these changes Mar 17, 2025

View reviewed changes

loic-sharma approved these changes Mar 17, 2025

View reviewed changes

vashworth added the autosubmit Merge PR when tree becomes green via auto submit App label Mar 18, 2025

auto-submit bot added this pull request to the merge queue Mar 18, 2025

Merged via the queue into flutter:master with commit 425d1bd Mar 18, 2025
171 checks passed

flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Mar 18, 2025

vashworth mentioned this pull request Mar 18, 2025

[Xcode 16] Mac plugin_test_macos timing out #164902

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 25, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

f408507

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 25, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

3ae9098

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

972be64

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

02d1cf0

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

479d71f

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

aee7b92

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

4fc42ec

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

e2c65e8

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

3050b36

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 26, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

a242ae5

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 27, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

715febd

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 27, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

d46d404

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 27, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

b0c4f9a

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 27, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

f9c8ac2

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 27, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

def2231

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 27, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

0d1a9de

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 28, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

2b0eced

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 28, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

23fb17e

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 28, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

d0ab7f3

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 28, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

ae29f9e

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 29, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

c2c4067

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 29, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

ab4a064

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 30, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

1c02dfa

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 30, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

1f6a174

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 31, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

269a12e

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 31, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

9516636

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 31, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

ff48ed8

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 31, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

d5adf6d

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 31, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

7d06e9f

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 31, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

1b05051

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 31, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

8e59423

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 1, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

293f521

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 1, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

bcc3a0a

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 1, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

26aa127

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 1, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

867df23

…r/flutter#165318)

vashworth mentioned this pull request Apr 2, 2025

Use a different device for iOS engine golden tests #166482

Open

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

1e755d3

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

63e9c65

…r/flutter#165318)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025

425d1bd Update tests to use Xcode 16.2 and iOS 18.2 Simulator (flutte…

2d263cc

…r/flutter#165318)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update tests to use Xcode 16.2 and iOS 18.2 Simulator#165318

Update tests to use Xcode 16.2 and iOS 18.2 Simulator#165318
auto-submit[bot] merged 2 commits intoflutter:masterfrom
vashworth:xcode_16_v2

vashworth commented Mar 17, 2025

Uh oh!

vashworth Mar 17, 2025

Uh oh!

vashworth Mar 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

vashworth commented Mar 17, 2025

Pre-launch Checklist

Uh oh!

vashworth Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

vashworth Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants