Improve defaults generation with logging, stats, and token validation by guidezpl · Pull Request #128244 · flutter/flutter

guidezpl · 2023-06-05T14:28:30Z

Description

This improves defaults generation with logging, stats, and token validation.

This PR includes these changes:

introduce TokenLogger, with a verbose mode
- prints versions and tokens usage to the console
- outputs generated/used_tokens.csv, a list of all used tokens, for use by Google
find token files in data automatically
hide tokens Map
- tokens can be obtained using existing resolvers (e.g. color, shape), or directly through getToken.
- tokens can be checked for existence with tokenAvailable
remove version from template, since the tokens are aggregated and multiple versions are possible (as is the case currently), it does not make sense to attribute a single version
improve documentation

Related Issues

Fixes Provide list of used Material Design tokens #122602

Tests

Added tests for TokenLogger
Regenerated tokens, no-op except version removal

Future work

A future PR should replace or remove the following invalid tokens usages

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
I signed the CLA.
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or this PR is test-exempt.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

QuncCccccc

This looks great! Just left some questions below:)

dev/tools/gen_defaults/generated/used_tokens.csv

QuncCccccc · 2023-06-08T06:16:53Z

dev/tools/gen_defaults/lib/color_scheme_template.dart


 class ColorSchemeTemplate extends TokenTemplate {
-  ColorSchemeTemplate(super.blockName, super.fileName, super.tokens);
+  ColorSchemeTemplate(this._colorTokensLight, this._colorTokensDark, super.blockName, super.fileName, super.tokens);


Just for my education, why do we use private properties in constructor?

It's no different than the following

ColorSchemeTemplate(
Map<String, dynamic> colorTokensLight,
...
) : _colorTokensLight = colorTokensLight;

QuncCccccc · 2023-06-08T06:27:17Z

dev/tools/gen_defaults/lib/template.dart

+  bool tokenAvailable(String tokenName) => _tokens.containsKey(tokenName);
+
+  /// Resolve a token while logging its usage.
+  dynamic getToken(dynamic tokenName) {


Should these two dynamic just be String?

Yes for tokenName, no for the return type, which can be other types (e.g. double)

QuncCccccc

LGTM:)!!

auto-submit · 2023-06-09T09:13:09Z

auto label is removed for flutter/flutter, pr: 128244, due to - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

…lidation (flutter/flutter#128244)

flutter/flutter@6e254a3...da127f1 2023-06-09 [email protected] Updated material button theme tests for Material3 (flutter/flutter#128543) 2023-06-09 [email protected] Roll Flutter Engine from cb93477008d6 to 93afba901b3b (2 revisions) (flutter/flutter#128573) 2023-06-09 [email protected] Improve defaults generation with logging, stats, and token validation (flutter/flutter#128244) 2023-06-09 [email protected] [testing] Make the FLUTTER_STORAGE_BASE_URL warning non-fatal (flutter/flutter#128335) 2023-06-09 [email protected] [flutter_tools] [DAP] Don't try to restart/reload if app hasn't started yet (flutter/flutter#128267) 2023-06-09 [email protected] Roll Flutter Engine from 8f9e608d39ab to cb93477008d6 (3 revisions) (flutter/flutter#128568) 2023-06-09 [email protected] Replace `MaterialButton` from test classes (flutter/flutter#128466) 2023-06-09 [email protected] Fix `showBottomSheet` doesn't remove scrim when draggable sheet is dismissed (flutter/flutter#128455) 2023-06-09 [email protected] Manual roll Flutter Engine from a5f7d5d75ff2 to 8f9e608d39ab (31 revisions) (flutter/flutter#128554) 2023-06-09 [email protected] Revert "test owners: cyanglaz -> vashworth" (flutter/flutter#128462) 2023-06-09 [email protected] [Android] Bump integration tests using `compileSdkVersion` 31 to 33 (flutter/flutter#128072) 2023-06-09 [email protected] Remove single view assumption from MouseTracker, and unify its hit testing code flow (flutter/flutter#127060) 2023-06-09 [email protected] [flutter_tools] Precache after channel switch (flutter/flutter#118129) 2023-06-08 [email protected] Adding migration guide for Material 3 colors (flutter/flutter#128429) 2023-06-08 [email protected] Add `AppLifecycleListener`, with support for application exit handling (flutter/flutter#123274) 2023-06-08 [email protected] Sliver Main Axis Group (flutter/flutter#126596) 2023-06-08 [email protected] Reduce `_DoubleClampVisitor` false positives (flutter/flutter#128539) 2023-06-08 [email protected] Advise developers to use OverflowBar instead of ButtonBar (flutter/flutter#128437) 2023-06-08 [email protected] Reland "Migrate benchmarks to package:web" (flutter/flutter#128266) 2023-06-08 [email protected] Navigator.pop before PopupMenuItem onTap call (flutter/flutter#127446) 2023-06-08 [email protected] Fix navigation rail with long labels misplaced highlights (flutter/flutter#128324) 2023-06-08 [email protected] Update `chip.dart` to use set of `MaterialState` (flutter/flutter#128507) 2023-06-08 [email protected] Update flutter to dartdoc 6.3.0 and hide Icons implementation from doc pages (flutter/flutter#128442) 2023-06-08 [email protected] Disable blinking cursor when `EditableText.showCursor` is false (flutter/flutter#127562) 2023-06-08 [email protected] [floating_cursor_selection]add more comments on the tricky part (flutter/flutter#127227) 2023-06-08 [email protected] Move RenderObjectElement.updateChildren to Element (flutter/flutter#128458) 2023-06-08 [email protected] Fix PointerEventConverter doc (flutter/flutter#128452) 2023-06-08 [email protected] Roll Packages from a84b2c2 to e13b8c4 (9 revisions) (flutter/flutter#128508) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

…lidation (flutter/flutter#128244)

guidezpl added 8 commits June 5, 2023 15:03

update

84b0450

create TokenLogger and tests

52da031

Update README.md

4213043

move args

7da28c2

use getToken

24cb66b

add verbose flag and remove manual file listing

eabcd96

add logging

b5027ec

refactor

ec86e49

github-actions bot removed the f: material design flutter/packages/flutter/material repository. label Jun 5, 2023

flutter-dashboard bot added the f: material design flutter/packages/flutter/material repository. label Jun 5, 2023

guidezpl added 3 commits June 6, 2023 01:26

x

c429fcf

fix analysis

a152c3d

update TokenLogger docs

c1dbdaa

guidezpl requested a review from QuncCccccc June 6, 2023 00:14

guidezpl marked this pull request as ready for review June 6, 2023 08:11

QuncCccccc reviewed Jun 8, 2023

View reviewed changes

address feedback

ad7d830

QuncCccccc approved these changes Jun 8, 2023

View reviewed changes

guidezpl and others added 4 commits June 8, 2023 13:08

Merge branch 'master' into tokens-cleanup

01ce882

Merge branch 'master' into tokens-cleanup

4933049

Merge remote-tracking branch 'upstream/master' into tokens-cleanup

3ec27e3

Update pubspec.yaml

f5f8715

guidezpl added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 9, 2023

auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jun 9, 2023

guidezpl added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 9, 2023

auto-submit bot merged commit 66cda59 into flutter:master Jun 9, 2023

guidezpl deleted the tokens-cleanup branch June 9, 2023 13:39

guidezpl mentioned this pull request Jun 9, 2023

Defaults generator uses invalid tokens #128578

Closed

engine-flutter-autoroll mentioned this pull request Jun 9, 2023

Roll Flutter from 6e254a3f9fb7 to da127f15ad54 (28 revisions) flutter/packages#4170

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jun 9, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

b638a30

…lidation (flutter/flutter#128244)

HansMuller mentioned this pull request Jun 9, 2023

Flutter Framework Team: Notable Commits #121415

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 16, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

69436fe

…lidation (flutter/flutter#128244)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

c04bcf8

…lidation (flutter/flutter#128244)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

9cccb85

…lidation (flutter/flutter#128244)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

66cda59 Improve defaults generation with logging, stats, and token va…

56022bc

…lidation (flutter/flutter#128244)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve defaults generation with logging, stats, and token validation#128244

Improve defaults generation with logging, stats, and token validation#128244
auto-submit[bot] merged 16 commits intoflutter:masterfrom
guidezpl:tokens-cleanup

guidezpl commented Jun 5, 2023 •

edited

Loading

Uh oh!

QuncCccccc left a comment

Uh oh!

Uh oh!

QuncCccccc Jun 8, 2023

Uh oh!

guidezpl Jun 8, 2023

Uh oh!

QuncCccccc Jun 8, 2023

Uh oh!

guidezpl Jun 8, 2023

Uh oh!

QuncCccccc left a comment

Uh oh!

auto-submit bot commented Jun 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

guidezpl commented Jun 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Tests

Future work

Pre-launch Checklist

Uh oh!

QuncCccccc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

QuncCccccc Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

guidezpl Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

QuncCccccc Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

guidezpl Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

QuncCccccc left a comment

Choose a reason for hiding this comment

Uh oh!

auto-submit bot commented Jun 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

guidezpl commented Jun 5, 2023 •

edited

Loading