Using headless=new for Chrome macrobenchmarks by Markzipan · Pull Request #168256 · flutter/flutter

Markzipan · 2025-05-03T02:28:20Z

My newly added DDC macrobenchmarks tests have been timing out on the test infra. It turns out that headless Chrome is significantly slower than headful Chrome (reasons unknown, but perhaps related to hardware acceleration). Using --headless=new instead of flutter run's headless flag (which uses the old headless mode) makes my local runs significantly (10x+) faster.

This time for sure

eyebrowsoffire

LGTM

In general, we need to rethink our web benchmarks strategy. I am soon going to be looking into reworking our benchmarks to actually run on more representative real-world hardware, rather than cloud instances with no GPU.

Markzipan · 2025-05-05T17:31:06Z

LGTM

In general, we need to rethink our web benchmarks strategy. I am soon going to be looking into reworking our benchmarks to actually run on more representative real-world hardware, rather than cloud instances with no GPU.

Sounds good! The Dart Web team's interested in allowing DDC to integrate with those benchmarks, so please keep us in the loop when those details emerge.

…r#168256)

My newly added DDC macrobenchmarks tests have been timing out on the test infra. It turns out that `headless` Chrome is significantly slower than headful Chrome (reasons unknown, but perhaps related to hardware acceleration). Using `--headless=new` instead of `flutter run`'s headless flag (which uses the old headless mode) makes my local runs significantly (10x+) faster. *This time for sure*

…r#168256)

Roll Flutter from cfb887c to b0f5c8c (281 revisions) flutter/flutter@cfb887c...b0f5c8c 2025-05-07 [email protected] Feat: Animate fill for material app bar (flutter/flutter#163913) 2025-05-07 [email protected] Roll Packages from e800da7 to b2ce3b0 (3 revisions) (flutter/flutter#168483) 2025-05-07 [email protected] Allow iOS devicelab tests to run on macOS 15.1 (flutter/flutter#168234) 2025-05-07 [email protected] Roll Skia from c8f54c1bc565 to 14f3e3209b73 (1 revision) (flutter/flutter#168477) 2025-05-07 [email protected] Roll Skia from 55d98def951d to c8f54c1bc565 (1 revision) (flutter/flutter#168452) 2025-05-07 [email protected] Roll Fuchsia Test Scripts from TcALRG6cjvR6RVe4L... to xobrcSN07udKdXOq_... (flutter/flutter#168444) 2025-05-07 [email protected] Roll Skia from 32591be9cd3b to 55d98def951d (2 revisions) (flutter/flutter#168440) 2025-05-07 [email protected] macOS: trailing closures for FlutterRunLoop.perform* (flutter/flutter#168415) 2025-05-07 [email protected] Roll Fuchsia Linux SDK from SOlRUoAtzxSsarx-6... to _GJje-N2Jk51qtBs6... (flutter/flutter#168434) 2025-05-07 [email protected] iOS/macOS: Consistent Flutter copyright headers (flutter/flutter#168390) 2025-05-06 [email protected] Capture errors when running flutter_tools in the entrypoint_dart_registrant devicelab test (flutter/flutter#168411) 2025-05-06 [email protected] Extending timeouts for DDC macrobenchmarks (flutter/flutter#168409) 2025-05-06 [email protected] Roll Skia from 36d1445ad00b to 32591be9cd3b (1 revision) (flutter/flutter#168421) 2025-05-06 [email protected] Ignore pointer on the outgoing route (flutter/flutter#168425) 2025-05-06 [email protected] [skwasm] Dispose underlying picture recorder when ending recording. (flutter/flutter#168384) 2025-05-06 [email protected] [Impeller] libImpeller: Expose access to conservative path bounds. (flutter/flutter#168402) 2025-05-06 [email protected] Roll Skia from 5cf5d0ff46d9 to 36d1445ad00b (1 revision) (flutter/flutter#168401) 2025-05-06 [email protected] Add height and width aspects to MediaQuery. (flutter/flutter#167829) 2025-05-06 [email protected] Roll Packages from e064b04 to e800da7 (2 revisions) (flutter/flutter#168392) 2025-05-06 [email protected] Add `bin/internal/last_engine_commit.sh` and tests. (flutter/flutter#168387) 2025-05-06 [email protected] Update `mac_ios` tests to run on either arm64 or x86 (flutter/flutter#168341) 2025-05-06 [email protected] Roll Skia from 889c5b52227c to 5cf5d0ff46d9 (3 revisions) (flutter/flutter#168378) 2025-05-06 [email protected] Roll Skia from 646573154de6 to 889c5b52227c (1 revision) (flutter/flutter#168363) 2025-05-06 [email protected] Roll Skia from 65fd10118ec2 to 646573154de6 (3 revisions) (flutter/flutter#168360) 2025-05-06 [email protected] feat(SwitchListTile): ensure that 'isThreeLine' can be configured through the theme. (flutter/flutter#166820) 2025-05-06 [email protected] [tool] Refactor WebTemplate to be immutable (flutter/flutter#168201) 2025-05-06 [email protected] Upload `android-arm64-release` treemap to known location (flutter/flutter#168349) 2025-05-06 [email protected] feat(RadioListTile):ensure that 'isThreeLine' can be configured through the (flutter/flutter#166964) 2025-05-06 [email protected] Roll Fuchsia Linux SDK from hjXORaFgiSd6QjK3D... to SOlRUoAtzxSsarx-6... (flutter/flutter#168347) 2025-05-05 [email protected] Revert "Wire up `MenuAnchor`, `MenuBar`, `MenuItem`-related widgets to aria roles (#165596)" (flutter/flutter#168339) 2025-05-05 [email protected] Bumping framework Template Defaults to Android 16 (API 36) (flutter/flutter#166464) 2025-05-05 [email protected] Roll Skia from c0404ac7ea67 to 65fd10118ec2 (3 revisions) (flutter/flutter#168346) 2025-05-05 [email protected] macOS: Support writing Swift in the macOS embedder (flutter/flutter#168174) 2025-05-05 [email protected] Copy Dart SDK _internal/vm/bin sources into the sky_engine package (flutter/flutter#168238) 2025-05-05 [email protected] Using headless=new for Chrome macrobenchmarks (flutter/flutter#168256) 2025-05-05 [email protected] Roll Dart SDK from a1e1e2770c1e to 8dc0ecee5adc (1 revision) (flutter/flutter#168338) 2025-05-05 [email protected] Check for duplicate archive paths in the engine build configuration JSON files (flutter/flutter#168248) 2025-05-05 [email protected] [Widget Inspector] Add on-device button for changing default "tap" behavior (flutter/flutter#167677) 2025-05-05 [email protected] Adding utf-8 encoding to hello world's html template (flutter/flutter#168162) 2025-05-05 [email protected] Update goldens.dart error to point to moved file (flutter/flutter#168319) 2025-05-05 [email protected] Skwasm heavy (flutter/flutter#166619) 2025-05-05 [email protected] Manual roll of Dart from 5c535a368799 to a1e1e2770c1e (flutter/flutter#168317) 2025-05-05 [email protected] Roll Skia from 9e23a55870e9 to c0404ac7ea67 (1 revision) (flutter/flutter#168321) 2025-05-05 [email protected] Cupertino sheet MediaQuery values (flutter/flutter#168041) 2025-05-05 [email protected] Change FGP unit test `expect` to match on process result instead of exit code (flutter/flutter#168278) 2025-05-05 [email protected] [Impeller] Use StrokeParameters anywhere stroke is described (flutter/flutter#168276) ...

…r#168256)

Using headless=new for Chrome benchmarks

fdb7559

Markzipan requested a review from matanlurey as a code owner May 3, 2025 02:28

Merge branch 'master' into ddc_benchmarks_fix3

c027eaa

yjbanov requested a review from eyebrowsoffire May 5, 2025 16:54

eyebrowsoffire approved these changes May 5, 2025

View reviewed changes

yjbanov approved these changes May 5, 2025

View reviewed changes

yjbanov added the autosubmit Merge PR when tree becomes green via auto submit App label May 5, 2025

auto-submit bot added this pull request to the merge queue May 5, 2025

Merged via the queue into flutter:master with commit f794a58 May 5, 2025
151 checks passed

flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label May 5, 2025

engine-flutter-autoroll mentioned this pull request May 6, 2025

Roll Flutter from cfb887cd5f41 to a3a41226e5af (259 revisions) flutter/packages#9214

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 6, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

33948f9

…r#168256)

engine-flutter-autoroll mentioned this pull request May 6, 2025

Roll Flutter from cfb887cd5f41 to a3a41226e5af (259 revisions) flutter/packages#9216

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 6, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

8d4c846

…r#168256)

engine-flutter-autoroll mentioned this pull request May 6, 2025

Roll Flutter from cfb887cd5f41 to 862ce5a986f2 (260 revisions) flutter/packages#9217

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 6, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

4f6bcb2

…r#168256)

Markzipan deleted the ddc_benchmarks_fix3 branch May 6, 2025 18:20

engine-flutter-autoroll mentioned this pull request May 6, 2025

Roll Flutter from cfb887cd5f41 to 397d3e118554 (261 revisions) flutter/packages#9218

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 6, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

f477cd5

…r#168256)

engine-flutter-autoroll mentioned this pull request May 7, 2025

Roll Flutter from cfb887cd5f41 to 3a71fefdcaf1 (278 revisions) flutter/packages#9220

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 7, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

c740926

…r#168256)

engine-flutter-autoroll mentioned this pull request May 7, 2025

Roll Flutter from cfb887cd5f41 to a3e9379a87ea (279 revisions) flutter/packages#9221

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 7, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

966ddf3

…r#168256)

engine-flutter-autoroll mentioned this pull request May 7, 2025

Roll Flutter from cfb887cd5f41 to 708c0eb18555 (280 revisions) flutter/packages#9222

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 7, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

e4ba807

…r#168256)

engine-flutter-autoroll mentioned this pull request May 7, 2025

Roll Flutter from cfb887cd5f41 to b0f5c8ce03e2 (281 revisions) flutter/packages#9223

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 7, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

a698a7c

…r#168256)

engine-flutter-autoroll mentioned this pull request May 7, 2025

Roll Flutter from cfb887cd5f41 to f9fb8906c7fb (282 revisions) flutter/packages#9224

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 7, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

3a2434c

…r#168256)

engine-flutter-autoroll mentioned this pull request May 7, 2025

Roll Flutter from cfb887cd5f41 to b3f6925fda4f (283 revisions) flutter/packages#9225

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 7, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

ed8de9a

…r#168256)

engine-flutter-autoroll mentioned this pull request May 7, 2025

Roll Flutter from cfb887cd5f41 to 9d8e5e0b2f0d (284 revisions) flutter/packages#9226

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 7, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

234564d

…r#168256)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 14, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

0f0f9e0

…r#168256)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 14, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

5961bb8

…r#168256)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

e7745e1

…r#168256)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

a94923b

…r#168256)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 16, 2025

f794a58 Using headless=new for Chrome macrobenchmarks (flutter/flutte…

461d5fd

…r#168256)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using headless=new for Chrome macrobenchmarks#168256

Using headless=new for Chrome macrobenchmarks#168256
auto-submit[bot] merged 2 commits intoflutter:masterfrom
Markzipan:ddc_benchmarks_fix3

Markzipan commented May 3, 2025

Uh oh!

eyebrowsoffire left a comment

Uh oh!

Markzipan commented May 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Markzipan commented May 3, 2025

Uh oh!

eyebrowsoffire left a comment

Choose a reason for hiding this comment

Uh oh!

Markzipan commented May 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants