[flutter_tools] expose SKSL shader dump by jonahwilliams · Pull Request #53278 · flutter/flutter

jonahwilliams · 2020-03-25T17:22:00Z

Description

Connect getSkSls to the M key, which writes the files to shaders/. This includes instructions on what to do with them (nothing!) and what to do if none are received.

Updates the asset manifest logic to include any shaders under shaders/. I'm not completely sold on this approach, but it seems like the most automatic - assuming the shaders are cross platform? If they are platform dependent we should do something else...

The SkSl files are currently packaged in flutter_assets/shaders, for example:

unzip app.apk
...
  inflating: assets/flutter_assets/shaders/CAZAAAACAUAAACIAAAABGAABAAOAAAYABQAECAAAAAAAAAAAAAAAEAAAAAOAAVYA  
  inflating: assets/flutter_assets/shaders/CAZAAAACAYAABAAAAAABGAABAD7777777777777777776FAABYAAAAAAAAAAAAAAAIAAAABEABLQA  
  inflating: assets/flutter_assets/shaders/CAZAAAACAYAABCAAAAABGAABAD777777777777YZAAAAAFAABYAAAAAAAAAAAAAAAIAAAABEABLQA  
  inflating: assets/flutter_assets/shaders/CAZAAAACAYAABGIAAAABGAABAAOAAFAA777777YZAAAAAFAABYAAAAAAAAAAAAAAAIAAAABEABLQA

Needs tests and cleanup.

…ce_protocol_sksl

jonahwilliams · 2020-03-25T22:09:55Z

packages/flutter_tools/lib/src/asset.dart

    String packagesPath,
    bool includeDefaultFonts = true,
    bool reportLicensedPackages = false,
+    bool includeShaderFiles = false,


Note to self, need to make a google3 change first

liyuqian · 2020-03-26T05:28:24Z

packages/flutter_tools/lib/src/base/command_help.dart

+  CommandHelpOption _M;
+  CommandHelpOption get M => _M ??= _makeOption(
+    'M',
+    'Write SKSL shaders to a file.',


Write SKSL shaders to a folder or Write SKSL shaders to files?

liyuqian · 2020-03-26T05:30:00Z

packages/flutter_tools/lib/src/resident_runner.dart

    throw Exception('Canvaskit not supported by this runner.');
  }

+  /// Dump the sksl shaders to a file in the project directory.


Same here to a folder or to files.

liyuqian · 2020-03-26T05:31:58Z

packages/flutter_tools/lib/src/resident_runner.dart

+      return;
+    }
+    final Directory outputDirectory = globals.fs.currentDirectory
+      .childDirectory('shaders')


Maybe use directory shaders/sksl or just sksl to be more specific. There's a possibility that we can have other kind of shader artifacts (e.g., binary shaders).

Also, maybe put the auto-captured shaders under a folder with flutter version in it (e.g., shaders/8857c4cec89164fdb6293bef69f9d62c62733efa/sksl) because SkSL is not guaranteed to be backward compatible. A Flutter app should use the SkSLs that are captured from the same framework/engine version.

Good to know! Yeah, we can version these based on the specific engine hash that flutter is built with and refuse to include older shaders

liyuqian · 2020-03-26T05:32:53Z

packages/flutter_tools/lib/src/resident_runner.dart

        }
        return false;
+      case 'M':
+        if (residentRunner.supportsWriteSkSl) {


Skia's name convention is SkSL with capitalized SL (stands for Shader Language).

liyuqian · 2020-03-26T05:33:22Z

packages/flutter_tools/lib/src/vmservice.dart

        });
  }

+  Future<Map<String, Uint8List>> getSkSls() async {


Same here for SkSLs.

We're limited to a single character here, and s is screenshot and S is dump semantics tree. We can also explore writing a service extension that editors/IDEs can invoke on behalf of a user, perhaps when they press a button in devtools or intellij.

FYI @jacob314 @devoncarew

Oops, this was meant to be on the switch above

(unless you were just taking about the member name and not the keypress...)

@kenzieschmoll @terrylucas so they're aware of exposing Skia information to tools.

@jonahwilliams, re: the single letter overload in the CLI. Some ideas:

Change to only have single letter abbreviations for very common operations (hot reload, quit, ...). Have an overflow shortcut key which prints a menu of additional available operations - those could be numbered, or typed in by short words (skia, dump, ...)

Or, have all operations have short phrases associated with them (hot-reload, hot-restart, dump-render-tree). You'd have to type in the full text in order to select the operation, but we could offer typing completion in the console so you'd normally only have to type in a few chars manually.

@jonahwilliams : ah yes, I was just talking about renaming getSkSls() to getSkSLs() as that's the name convention from Skia.

I played around with this toy program:

import 'dart:io'; main() { stdin.echoMode = false; stdin.lineMode = false; stdin.listen((List<int> s) { print('> $s\n'); }); }

on Linux and Windows. On both it's possible to detect when ctrl is pressed while a letter key is pressed. We'd probably have to decode stdin with a custom decoder rather than AsciiDecoder though.

(On Linux alt is sometimes part of the same List<int> as the character code when it is pressed, and sometimes not. On Windows it's not detectable.)

liyuqian · 2020-03-26T05:38:09Z

packages/flutter_tools/lib/src/asset.dart

+    final List<File> newAdditionalDependencies = <File>[];
+
+    // Include the SkSl shader files in shaders/
+    if (includeShaderFiles && globals.fs.directory('shaders').existsSync()) {


What if some Flutter project has a user-created shaders directory in the root Flutter directory? Would it be safer to put such directory under build so it's clear that the files are auto-generated and not user-created?

Yeah thats a good idea - these should probably be versioned and in the build artifacts directory

liyuqian · 2020-03-26T05:43:53Z

Regarding the platform dependencies, the SkSL (Skia shader language) is by itself platform independent. It can be used for Skia CPU backend, OpenGL backend, Vulkan backend, or Metal backend. One can write an SkSL shader and run it in all platforms. However, the actual content in the SkSL file that we captured may depend on platforms (e.g., either for performance optimizations, or because some platforms have bugs in certain areas). I'll experiment how robust our captured SkSLs are.

…ce_protocol_sksl

jonahwilliams · 2020-03-28T05:11:44Z

I wonder, if its fairly stable across platforms then a developer could run the shader warmup on an Android emulator and then use that output for a released iOS application right? Would the same apply to the flutter_tester shell? I realize that is headless, or mostly headless but I assume most of the same engine bits are there...

liyuqian · 2020-03-30T18:26:29Z

@jonahwilliams : I currently don't have high hopes of having iOS use the SkSLs generated from Android emulators. But I'm certainly interested in doing such experiments and see how far it can go.

Also CC @brianosman from Skia team to see his opinion about having iOS devices use SkSLs generated from Android emulators. My guess is that it won't crash the iOS app, but it's unclear how much useful warmup those SkSLs can do.

brianosman · 2020-03-31T23:44:33Z

Sorry, closed the wrong PR. While I'm here, though - yeah, using SkSLs from a platform that different is unlikely to be helpful. There may be some overlap, but I wouldn't recommend it.

jonahwilliams · 2020-04-10T23:14:17Z

Continuing the rest of the work in #54499

[flutter_tools] expose sksl shader dump

1ec3a85

fluttergithubbot added tool Affects the "flutter" command-line tool. See also t: labels. work in progress; do not review labels Mar 25, 2020

googlebot added the cla: yes label Mar 25, 2020

jonahwilliams added 2 commits March 25, 2020 11:43

Merge branch 'master' of github.com:flutter/flutter into expose_servi…

fb32574

…ce_protocol_sksl

connect shaders to asset build system

1beb6d9

jonahwilliams commented Mar 25, 2020

View reviewed changes

liyuqian reviewed Mar 26, 2020

View reviewed changes

Merge branch 'master' of github.com:flutter/flutter into expose_servi…

7cc0d0a

…ce_protocol_sksl

more work on shader inclusion as a separate asset target

7d038ca

brianosman closed this Mar 31, 2020

brianosman reopened this Mar 31, 2020

liyuqian mentioned this pull request Apr 2, 2020

SkSL-based shader warmup #53607

Closed

8 tasks

jonahwilliams mentioned this pull request Apr 2, 2020

[flutter_tools] write SkSL file to local file #53859

Merged

jonahwilliams closed this Apr 10, 2020

jonahwilliams deleted the expose_service_protocol_sksl branch April 10, 2020 23:14

github-actions bot locked as resolved and limited conversation to collaborators Jul 31, 2021

Conversation

jonahwilliams commented Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devoncarew Mar 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liyuqian commented Mar 26, 2020

Uh oh!

jonahwilliams commented Mar 28, 2020

Uh oh!

liyuqian commented Mar 30, 2020

Uh oh!

brianosman commented Mar 31, 2020

Uh oh!

jonahwilliams commented Apr 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jonahwilliams commented Mar 25, 2020 •

edited

Loading

devoncarew Mar 27, 2020 •

edited

Loading