feat: start using the content aware hash for downloading artifacts by jtmcdole · Pull Request #171927 · flutter/flutter

jtmcdole · 2025-07-09T23:50:08Z

Use the content aware hash when downloading engine artifacts. These are currently produced when changes to DEPs, engine/, or the release file are changed in a not-seen-before way.

We can eventually remove engine.version being tracked in release branches as an optimization.

FLUTTER_PREBUILT_ENGINE_VERSION will stay for overriding the engine.version for testing. Though Cocoon does not need to actually set it for framework only PRs anymore.

jtmcdole · 2025-07-09T23:51:52Z

Note: before we remove building of git-sha artifacts, we need to validate rolls into Google.

jtmcdole · 2025-07-10T20:16:29Z

Looks like a real problem with the windows tests; I'll try it out locally.

00:13 +42 ~2 -1: C:/b/s/w/ir/x/w/flutter/dev/tools/test/update_engine_version_test.dart: resolves engine artifacts with git merge-base fallsback to origin/master [E]
  ProcessException: %1 is not a valid Win32 application.
  
    Command: C:\b\s\w\ir\x\t\update_engine_version_test.2b990013\flutter\bin\internal\content_aware_hash.ps1 
  dart:io                                      Process.runSync
  test\update_engine_version_test.dart 57:48   main.run
  test\update_engine_version_test.dart 245:50  main.gitContentHash
  test\update_engine_version_test.dart 352:34  main.<fn>.<fn>

jtmcdole · 2025-07-10T20:33:09Z

Looks like a real problem with the windows tests; I'll try it out locally.

00:13 +42 ~2 -1: C:/b/s/w/ir/x/w/flutter/dev/tools/test/update_engine_version_test.dart: resolves engine artifacts with git merge-base fallsback to origin/master [E]
  ProcessException: %1 is not a valid Win32 application.
  
    Command: C:\b\s\w\ir\x\t\update_engine_version_test.2b990013\flutter\bin\internal\content_aware_hash.ps1 
  dart:io                                      Process.runSync
  test\update_engine_version_test.dart 57:48   main.run
  test\update_engine_version_test.dart 245:50  main.gitContentHash
  test\update_engine_version_test.dart 352:34  main.<fn>.<fn>

Duh... run powershell in powershell...

jtmcdole · 2025-07-10T21:57:47Z

We should wait for stable to be cut before merging.

jtmcdole · 2025-07-14T20:39:45Z

Blocked waiting for #172132 / #171985

jtmcdole · 2025-07-15T18:24:28Z

rebased to get past ci.yaml.

jtmcdole · 2025-07-16T19:55:47Z

I need to hold off on this till we've fixed another bug: android artifacts are not uploaded correctly (right folder, wrong file). The recipes are misleading in their output on LUCI.

Converting to draft.

jtmcdole · 2025-07-18T16:05:54Z

Ah shoot, that last one is meant for another PR

…facts (flutter/flutter#171927)

…172552) reland of #171927 towards #171790 Use the content aware hash when downloading engine artifacts. These are currently produced when changes to DEPs, engine/, or the release file are changed in a not-seen-before way. We can eventually remove engine.version being tracked in release branches as an optimization. FLUTTER_PREBUILT_ENGINE_VERSION will stay for overriding the engine.version for testing. Though Cocoon does not need to actually set it for framework only PRs anymore. fxies post submit tests for content_hash download: SHARD=framework_tests SUBSHARD=misc dart --enable-asserts dev/bots/test.dart SHARD=tool_host_cross_arch_tests dart --enable-asserts dev/bots/test.dart ``` ╔═╡ERROR https://github.com/flutter/flutter/pull/1╞════════════════════════════════════════════════════════════════════ ║ Expected "Flutter Engine Version: 45b085ab2118f30441ae6e2daab4b8eb3e4a063c", but found "Flutter Engine Version: 407c051". ╚═══════════════════════════════════════════════════════════════════════════════ ``` ``` Expected: contains '45b085ab2118f30441ae6e2daab4b8eb3e4a063c' Which: does not contain '45b085ab2118f30441ae6e2daab4b8eb3e4a063c' ```

…facts (flutter/flutter#171927)

flutter/flutter@70cdc0c...ee0cc66 2025-07-22 [email protected] Licenses cpp ignore whitespace (flutter/flutter#172512) 2025-07-22 [email protected] feat: start using the content aware hash for downloading artifacts (flutter/flutter#172552) 2025-07-22 [email protected] fix: correct typo in analysis_options.yaml comment (disabeling -> disabling) (flutter/flutter#170206) 2025-07-22 [email protected] Hide `flutter assemble` unless `--verbose` (flutter/flutter#172508) 2025-07-22 [email protected] Fix PopupMenuButton crashes when being hidden immediately (flutter/flutter#171970) 2025-07-22 [email protected] Add hitTestChildren to baseline-aligned stack (flutter/flutter#172485) 2025-07-22 [email protected] Roll Skia from 4aba494b8108 to eea1c33fbe84 (11 revisions) (flutter/flutter#172515) 2025-07-22 [email protected] Replaces legacy licenses check with licenses_cpp (flutter/flutter#172486) 2025-07-22 [email protected] Omit `w` (widget tree), `t` (render tree) from `flutter run --profile` `h` (flutter/flutter#172506) 2025-07-22 [email protected] Refactor checks for `--machine` (flutter/flutter#172504) 2025-07-22 [email protected] feat(IconButton): add property to specify a states controller (flutter/flutter#169821) 2025-07-21 98614782+auto-submit[bot]@users.noreply.github.com Reverts "feat: start using the content aware hash for downloading artifacts (#171927)" (flutter/flutter#172505) 2025-07-21 [email protected] Render Flutter in its own EGL context (flutter/flutter#172330) 2025-07-21 [email protected] Feat: Add momentary varient of CupertinoSlidingSegmentedControl (flutter/flutter#164262) 2025-07-21 [email protected] Roll Packages from cb8fef6 to 9c85e5e (10 revisions) (flutter/flutter#172479) 2025-07-21 [email protected] Wrap `ShutdownHook` calls to avoid crashing the tool (flutter/flutter#171917) 2025-07-21 [email protected] fix: macOS Mojave Crash on Flutter Desktop Startup (flutter/flutter#172282) 2025-07-21 [email protected] Roll Skia from 9587301e33bc to 4aba494b8108 (41 revisions) (flutter/flutter#172471) 2025-07-21 [email protected] Add CLA note to tree hygiene (flutter/flutter#172475) 2025-07-21 [email protected] Add `radioInnerRadius` to `RadioListTile` (flutter/flutter#171851) 2025-07-21 [email protected] Use baseUri always when doing a hot reload or hot restart (flutter/flutter#172271) 2025-07-21 [email protected] feat: start using the content aware hash for downloading artifacts (flutter/flutter#171927) 2025-07-21 [email protected] Add support for `/?` to alias to `--help` (flutter/flutter#172328) 2025-07-21 [email protected] Upgrade flutter_lints in project template pubspec.yaml files (flutter/flutter#172400) 2025-07-21 [email protected] Use `$dartSdkVersion` when creating "dummy" pubspec for `create_api_docs` (flutter/flutter#172327) 2025-07-21 [email protected] Adds `(defaults to ___)` when using `flutter config --help` (flutter/flutter#172329) 2025-07-21 [email protected] Add a warning on usage of `dartPluginClass: 'none'`. (flutter/flutter#172315) 2025-07-21 [email protected] Skip [tool] crash reporting on `--local-engine` (flutter/flutter#172326) 2025-07-21 [email protected] Remove IDE validation from `flutter doctor` (flutter/flutter#171924) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

robert-ancell · 2025-07-24T01:24:16Z

This seems to have broken the Dart SDK download on Linux:

$ flutter --version
Downloading Linux x64 Dart SDK from Flutter engine 494bd6a8024b257ab1af875d28581e1fc4dc4465...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   255  100   255    0     0     45      0  0:00:05  0:00:05 --:--:--    53
[/home/bob/git/flutter/flutter/bin/cache/dart-sdk-linux-x64.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /home/bob/git/flutter/flutter/bin/cache/dart-sdk-linux-x64.zip or
        /home/bob/git/flutter/flutter/bin/cache/dart-sdk-linux-x64.zip.zip, and cannot find /home/bob/git/flutter/flutter/bin/cache/dart-sdk-linux-x64.zip.ZIP, period.

It appears that the downloaded file is corrupt; please try again.
If this problem persists, please report the problem at:
  https://github.com/flutter/flutter/issues/new?template=01_activation.yml

The hash doesn't appear on the server:

$ wget https://storage.googleapis.com/flutter_infra_release/flutter/494bd6a8024b257ab1af875d28581e1fc4dc4465/dart-sdk-linux-x64.zip
--2025-07-24 13:23:01--  https://storage.googleapis.com/flutter_infra_release/flutter/494bd6a8024b257ab1af875d28581e1fc4dc4465/dart-sdk-linux-x64.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 142.250.76.123, 142.250.204.27, 142.251.221.91, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|142.250.76.123|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2025-07-24 13:23:02 ERROR 404: Not Found.

robert-ancell · 2025-07-24T01:31:22Z

Workaround I'm currently using:

diff --git a/bin/internal/update_engine_version.sh b/bin/internal/update_engine_version.sh
index d8185cb8db5..2f058bdd658 100755
--- a/bin/internal/update_engine_version.sh
+++ b/bin/internal/update_engine_version.sh
@@ -57,7 +57,17 @@ elif [ -n "$(git -C "$FLUTTER_ROOT" ls-files bin/internal/engine.version)" ]; th
 # the current branch is forked from, which would be the last version of the
 # engine artifacts built from CI.
 else
-  ENGINE_VERSION=$("$FLUTTER_ROOT/bin/internal/content_aware_hash.sh")
+  set +e
+  # We fallback to origin/master if upstream is not detected.
+  git -C "$FLUTTER_ROOT" remote get-url upstream >/dev/null 2>&1
+  exit_code=$?
+  set -e
+
+  if [[ $exit_code -eq 0 ]]; then
+    ENGINE_VERSION=$(git -C "$FLUTTER_ROOT" merge-base HEAD upstream/master)
+  else
+    ENGINE_VERSION=$(git -C "$FLUTTER_ROOT" merge-base HEAD origin/master)
+  fi
 fi
 
 # Write the engine version out so downstream tools know what to look for.

robert-ancell · 2025-07-24T23:24:17Z

Seems to be working today, assume the server caught up.

robert-ancell · 2025-07-24T23:55:27Z

Correction - it's working on main, but any branches I have don't work and need the above patch to work.

jtmcdole · 2025-07-25T16:40:49Z

Correction - it's working on main, but any branches I have don't work and need the above patch to work.

Can you open a bug or follow up here: #171790

Assumption: you have local changes?

…lutter#171927) towards flutter#171790 Use the content aware hash when downloading engine artifacts. These are currently produced when changes to DEPs, engine/, or the release file are changed in a not-seen-before way. We can eventually remove `engine.version` being tracked in release branches as an optimization. `FLUTTER_PREBUILT_ENGINE_VERSION` will stay for overriding the engine.version for testing. Though Cocoon does not need to actually set it for framework only PRs anymore.

…ifacts (flutter#171927)" (flutter#172505)  Reverts: flutter#171927   Initiated by: matanlurey   Reason for reverting: Broke the tree, due to naughty tests but too intensive to fix forward.   Original PR Author: jtmcdole   Reviewed By: {matanlurey, chingjun}   This change reverts the following previous change: towards flutter#171790 Use the content aware hash when downloading engine artifacts. These are currently produced when changes to DEPs, engine/, or the release file are changed in a not-seen-before way. We can eventually remove `engine.version` being tracked in release branches as an optimization. `FLUTTER_PREBUILT_ENGINE_VERSION` will stay for overriding the engine.version for testing. Though Cocoon does not need to actually set it for framework only PRs anymore.  Co-authored-by: auto-submit[bot] <[email protected]>

…lutter#172552) reland of flutter#171927 towards flutter#171790 Use the content aware hash when downloading engine artifacts. These are currently produced when changes to DEPs, engine/, or the release file are changed in a not-seen-before way. We can eventually remove engine.version being tracked in release branches as an optimization. FLUTTER_PREBUILT_ENGINE_VERSION will stay for overriding the engine.version for testing. Though Cocoon does not need to actually set it for framework only PRs anymore. fxies post submit tests for content_hash download: SHARD=framework_tests SUBSHARD=misc dart --enable-asserts dev/bots/test.dart SHARD=tool_host_cross_arch_tests dart --enable-asserts dev/bots/test.dart ``` ╔═╡ERROR https://github.com/flutter/flutter/pull/1╞════════════════════════════════════════════════════════════════════ ║ Expected "Flutter Engine Version: 45b085ab2118f30441ae6e2daab4b8eb3e4a063c", but found "Flutter Engine Version: flutter@407c051". ╚═══════════════════════════════════════════════════════════════════════════════ ``` ``` Expected: contains '45b085ab2118f30441ae6e2daab4b8eb3e4a063c' Which: does not contain '45b085ab2118f30441ae6e2daab4b8eb3e4a063c' ```

robert-ancell · 2025-07-30T01:28:02Z

Yes, this is:

Get the main branch
Make changes (may or may not be committed)
Build Flutter
Run a Flutter app with flutter run --local-engine-src-path ...