Remove `scenario_app/android` and rename to `ios_scenario_app`. by matanlurey · Pull Request #160992 · flutter/flutter

matanlurey · 2024-12-30T20:47:27Z

This PR eliminates (1) scenario_app/android, and all references to an Android scenario_app, including documentation, CI configuration, build rules, test runner, Android-side test application, and (2), to verify and disambiguate the remains, renames the folder ios_scenario_app, which is now accurate.

It also eliminates elements that were only used in the Android-side scenario_app, such as Firebase Test Lab uploading.

I would be open to doing this in phases if we thought it was better to do so, but given its mostly a mechanical change (and by renaming the directory, references can be checked merely by looking for /\bscenario_app/, I believe this is safe to iterate on and eventually merge after the holidays.

Background

As of the merged mono-repo, there is no longer a requirement for the engine to be testable as a standalone unit.

As an example, %ENGINE%/testing/scenario_app, which was intended to emulate the Flutter framework (and some of flutter_tools tooling), load the iOS and Android embedder, and run various "scenarios" (which ran a combination of dart:ui code and Android Java/iOS Obj-C) verifying golden-file screenshots.

Instead, it is now possible to write and run real (full) Flutter apps the same way that an end-user (or our own tests) would. One such example is dev/native_driver_test, which is a full-fledged Flutter app, which uses standard tooling (i.e. flutter), to test most of the same elements that previously were only tested in the Android version of scenario_app.

jmagman · 2024-12-30T21:02:33Z

Instead, it is now possible to write and run real (full) Flutter apps the same way that an end-user (or our own tests) would.

#160800

zanderso

Removing these on Android makes sense if we already have similar test coverage that pre-exists on emulators in the framework tests.

I see this is a draft, but still want to note that removing the entries from the ci.yaml file will cause all the builds/tests in the json file to not run, including the non-scenario-app tests. I suspect this might not be your intention.

matanlurey · 2025-01-06T17:39:35Z

I see this is a draft, but still want to note that removing the entries from the ci.yaml file will cause all the builds/tests in the json file to not run, including the non-scenario-app tests. I suspect this might not be your intention.

Yeah that's not the intent, thanks for the catch. Will clean that up before a formal review.

zanderso · 2025-01-07T19:17:42Z

Is there any diff between the test coverage of scenario_app and native_driver_test? If so, do any new tests need to be added to native_driver_test?

matanlurey · 2025-01-07T20:35:17Z

Is there any diff between the test coverage of scenario_app and native_driver_test? If so, do any new tests need to be added to native_driver_test?

@jonahwilliams and I looked briefly.

There is a different configuration of platform views Jonah would like covered, but everything else is either covered or the scenario_app tests were superfluous (not covering something we find important). Filed #161261 and will do that first.

chinmaygarde · 2025-01-13T18:59:14Z

Triage: The presub check is hung and it seems like the only way to fix it is to push an empty commit.

matanlurey · 2025-01-13T19:16:47Z

Moving back to draft. When android_engine_test feels good (~1 more week), I'll send this out for final review.

See #161534.

zanderso

Thanks!

jonahwilliams

LGTM

auto-submit · 2025-01-27T21:42:07Z

autosubmit label was removed for flutter/flutter/160992, because - The status or check suite Mac mac_unopt has failed. Please fix the issues identified (or deflake) before re-applying this label.

auto-submit · 2025-01-28T00:02:04Z

autosubmit label was removed for flutter/flutter/160992, because - The status or check suite Mac_arm64 build_tests_4_4 has failed. Please fix the issues identified (or deflake) before re-applying this label.

…`. (flutter/flutter#160992)

Roll Flutter from c1ffaa9 to b007899 (43 revisions) flutter/flutter@c1ffaa9...b007899 2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315) 2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309) 2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582) 2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903) 2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880) 2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345) 2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272) 2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336) 2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330) 2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337) 2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332) 2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289) 2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335) 2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327) 2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123) 2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049) 2025-01-28 [email protected] Refactor code inside flutter.groovy (flutter/flutter#160250) 2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282) 2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911) 2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279) 2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313) 2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992) 2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294) 2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270) 2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170) 2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205) 2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277) 2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271) 2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037) 2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996) 2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423) 2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258) 2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198) 2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254) 2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048) 2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118) 2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518) 2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164) 2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089) 2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171) 2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038) 2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113) 2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: ...

…`. (flutter/flutter#160992)

Roll Flutter from c1ffaa9 to b007899 (43 revisions) flutter/flutter@c1ffaa9...b007899 2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315) 2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309) 2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582) 2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903) 2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880) 2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345) 2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272) 2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336) 2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330) 2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337) 2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332) 2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289) 2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335) 2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327) 2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123) 2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049) 2025-01-28 [email protected] Refactor code inside flutter.groovy (flutter/flutter#160250) 2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282) 2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911) 2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279) 2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313) 2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992) 2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294) 2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270) 2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170) 2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205) 2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277) 2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271) 2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037) 2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996) 2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423) 2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258) 2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198) 2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254) 2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048) 2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118) 2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518) 2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164) 2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089) 2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171) 2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038) 2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113) 2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: ...

matanlurey added 2 commits December 30, 2024 12:37

Remove android, and rename to ios_scenario_app.

5bbedf2

Run et format.

f00c96b

matanlurey requested review from jmagman, jonahwilliams, jtmcdole, reidbaker and zanderso December 30, 2024 20:47

github-actions bot added a: text input Entering text in a text field or keyboard related problems engine flutter/engine related. See also e: labels. f: routes Navigator, Router, and related APIs. d: docs/ flutter/flutter/docs, for contributors labels Dec 30, 2024

matanlurey added 3 commits December 30, 2024 13:12

ios_scenario_app has no Dart unit tests, so remove from run_tests.py.

697da8e

Remove stale tests.

00ffc0f

Restore missing file.

36576c2

zanderso reviewed Jan 6, 2025

View reviewed changes

Revert some removals in .ci.yaml.

fcfdb55

matanlurey marked this pull request as ready for review January 6, 2025 21:09

matanlurey requested a review from zanderso January 6, 2025 21:09

gmackall mentioned this pull request Jan 6, 2025

[Android] Add .cxx directories to app template .gitignore #161069

Merged

9 tasks

matanlurey mentioned this pull request Jan 7, 2025

Add additional platform-view Android screenshot tests #161261

Closed

matanlurey added the blocked Issue is blocked by another issue label Jan 7, 2025

reidbaker approved these changes Jan 10, 2025

View reviewed changes

matanlurey mentioned this pull request Jan 13, 2025

Refactor android_engine_test, make it easier to debug/deflake locally. #161534

Merged

matanlurey marked this pull request as draft January 13, 2025 19:16

matanlurey marked this pull request as ready for review January 27, 2025 19:49

zanderso approved these changes Jan 27, 2025

View reviewed changes

jonahwilliams approved these changes Jan 27, 2025

View reviewed changes

matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 27, 2025

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 27, 2025

++

a4fadd8

matanlurey added autosubmit Merge PR when tree becomes green via auto submit App and removed blocked Issue is blocked by another issue labels Jan 27, 2025

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 28, 2025

matanlurey added this pull request to the merge queue Jan 28, 2025

Merged via the queue into flutter:master with commit 9e273d5 Jan 28, 2025
175 checks passed

matanlurey deleted the remove-android-scenario-app branch January 28, 2025 04:18

engine-flutter-autoroll mentioned this pull request Jan 28, 2025

Roll Flutter from c1ffaa9d9deb to 9e273d5e6e2e (22 revisions) flutter/packages#8520

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 28, 2025

9e273d5 Remove scenario_app/android and rename to `ios_scenario_app…

eb4fabe

…`. (flutter/flutter#160992)

engine-flutter-autoroll mentioned this pull request Jan 28, 2025

Roll Flutter from c1ffaa9d9deb to 21f7283f5dfb (23 revisions) flutter/packages#8522

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 28, 2025

9e273d5 Remove scenario_app/android and rename to `ios_scenario_app…

4be69ec

…`. (flutter/flutter#160992)

engine-flutter-autoroll mentioned this pull request Jan 28, 2025

Roll Flutter from c1ffaa9d9deb to a9c50335c736 (24 revisions) flutter/packages#8523

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 28, 2025

9e273d5 Remove scenario_app/android and rename to `ios_scenario_app…

4b099ec

…`. (flutter/flutter#160992)

engine-flutter-autoroll mentioned this pull request Jan 29, 2025

Roll Flutter from c1ffaa9d9deb to b007899d3a7a (43 revisions) flutter/packages#8527

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 29, 2025

9e273d5 Remove scenario_app/android and rename to `ios_scenario_app…

d0789c4

…`. (flutter/flutter#160992)

matanlurey mentioned this pull request Feb 20, 2025

Android emulator tests (engine) are not running on CI #163742

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025

9e273d5 Remove scenario_app/android and rename to `ios_scenario_app…

dcb5956

…`. (flutter/flutter#160992)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025

9e273d5 Remove scenario_app/android and rename to `ios_scenario_app…

c19a4c5

…`. (flutter/flutter#160992)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025

9e273d5 Remove scenario_app/android and rename to `ios_scenario_app…

4579d95

…`. (flutter/flutter#160992)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `scenario_app/android` and rename to `ios_scenario_app`.#160992

Remove `scenario_app/android` and rename to `ios_scenario_app`.#160992
matanlurey merged 8 commits intoflutter:masterfrom
matanlurey:remove-android-scenario-app

matanlurey commented Dec 30, 2024 •

edited

Loading

Uh oh!

jmagman commented Dec 30, 2024

Uh oh!

zanderso left a comment

Uh oh!

matanlurey commented Jan 6, 2025

Uh oh!

zanderso commented Jan 7, 2025

Uh oh!

matanlurey commented Jan 7, 2025

Uh oh!

chinmaygarde commented Jan 13, 2025

Uh oh!

matanlurey commented Jan 13, 2025

Uh oh!

zanderso left a comment

Uh oh!

jonahwilliams left a comment

Uh oh!

auto-submit bot commented Jan 27, 2025

Uh oh!

auto-submit bot commented Jan 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

matanlurey commented Dec 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Uh oh!

jmagman commented Dec 30, 2024

Uh oh!

zanderso left a comment

Choose a reason for hiding this comment

Uh oh!

matanlurey commented Jan 6, 2025

Uh oh!

zanderso commented Jan 7, 2025

Uh oh!

matanlurey commented Jan 7, 2025

Uh oh!

chinmaygarde commented Jan 13, 2025

Uh oh!

matanlurey commented Jan 13, 2025

Uh oh!

zanderso left a comment

Choose a reason for hiding this comment

Uh oh!

jonahwilliams left a comment

Choose a reason for hiding this comment

Uh oh!

auto-submit bot commented Jan 27, 2025

Uh oh!

auto-submit bot commented Jan 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

matanlurey commented Dec 30, 2024 •

edited

Loading