Skip to content

[dart] expose Dart through the Flutter SDK#53470

Closed
jonahwilliams wants to merge 1 commit intoflutter:masterfrom
jonahwilliams:add_dart_forwarders
Closed

[dart] expose Dart through the Flutter SDK#53470
jonahwilliams wants to merge 1 commit intoflutter:masterfrom
jonahwilliams:add_dart_forwarders

Conversation

@jonahwilliams
Copy link
Contributor

@jonahwilliams jonahwilliams commented Mar 28, 2020

Description

Make the dart sdk visible through flutter/bin.

We'd like to expose dart and future dartdev commands to flutter users directly without proxying things through the flutter tool. It isn't sufficient to simply but flutter/bin/cache/dart-sdk/bin on the PATH as well, since this would bypass the Dart SDK staleness check that the entrypoint script runs.

If users currently have a separate dart on their path, this may interfere depending on ordering.

#57063

In the future, this would also let us deprecate/remove flutter_tools commands that are simply pass-throughs to dart commands

@mit-mit
Copy link
Member

mit-mit commented Mar 28, 2020

Thanks Jonah. I noticed that you tagged this in progress; just to double-check, we don't want this merged until after we've cut the 1.16 stable build (we've temporarily disabled the new dart dev experience in the VM, as it's targeted for Dart 2.9, not 2.8).

@jonahwilliams
Copy link
Contributor Author

Certainly!

@jonahwilliams jonahwilliams added a: null-safety Support for Dart's null safety feature tool Affects the "flutter" command-line tool. See also t: labels. labels Apr 2, 2020
@jonahwilliams
Copy link
Contributor Author

jonahwilliams commented Apr 13, 2020

I think we're passed the cutoff point now? @mit-mit @pcsosinski

@pcsosinski
Copy link

we branched for 1.17/2.8 on 4/1, assuming that's the point you're talking about

@mit-mit
Copy link
Member

mit-mit commented Apr 14, 2020

Yup, Jonah feel free to land this now

@jonahwilliams jonahwilliams marked this pull request as ready for review April 30, 2020 00:05
@jonahwilliams
Copy link
Contributor Author

For testing: I'm not sure what the best route is. I don't want to necessarily test Dart SDK functionality, so maybe some sort of trivial --version read?


# ---------------------------------- NOTE ---------------------------------- #
#
# Please keep the logic in this file consistent with the logic in the
Copy link
Contributor

@christopherfujino christopherfujino Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how batch files work, but for bash maybe we should factor out the command logic between this and //bin/flutter into a third shell script that these two source?

I generally don't like sourcing outside scripts in shell scripts, but I feel like these dart scripts are very likely to drift in logic from their flutter counterparts.

Another option is to have both //bin/flutter and //bin/dart just be wrappers that invoke the third script, and then the common script has branching logic depending on which entrypoint it was invoked from.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the bash changes I looked at https://stackoverflow.com/questions/192292/how-best-to-include-other-scripts which scared me quite a bit.

I'm not really that familiar with shell scripting. Maybe we can chat a bit tomorrow to figure out how that would work?

I agree less duplicate is better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let's chat tomorrow. And you're right to be scared :P

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a little work on flutter's shell script here and @devoncarew asked me to poke my head in. I won't say some caution isn't warranted but a few basic best practices (adapted for Flutter's situation) should keep you from accidentally making your shell scripts any more unreadable than strictly necessary.

  1. Determine FLUTTER_ROOT first, then use that to calculate the path of any file you source. Those suggestions in the stackoverflow are kind of ad-hoc and terrifying... I can suggest some alternative reading material offline if you are interested.
  2. No file you source should do anything but declare functions -- if you want to share common variable initialization, move those to functions. In fact, do as little as you can outside of functions generally -- it makes the script much easier to follow. Put things you would ordinarily have in the top level in a main function.
  3. Use local extensively to declare variables and minimize your use of globals where you can. Globals in CAPS_WITH_UNDERSCORES, locals with lower_case_underscored. The script mostly does this already...
  4. Decide on a naming convention for your functions that make sense and minimize namespace collisions. For example: function flutter::common::retry_upgrade () {} for a function defined in the library 'common.sh'. If you use that style consistently it will scale up as far as you need and also makes it more clear when you're calling something local vs. sourced.

Like any set of best practices you can break them if you know what you're doing, but I humbly suggest starting with these in your implementation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I've been thinking about this, I can think of 3 options to keep us from duplicating code:

  1. Move shared code into functions in a seperate script and source them from the flutter and dart entrypoints, ala @jcollins-g's comment above.
  2. Move all the logic currently in //bin/flutter into //bin/launcher.sh. Create two symlinks to that script at //bin/flutter and //bin/dart. Then in //bin/launcher.sh, read which entrypoint the user used ENTRYPOINT="$(basename "$BASH_SOURCE")" and then add branching logic to the script based on the value of $ENTRYPOINT where needed.
  3. Similar to the last method, but instead of using symlinks, just have short wrapper shell scripts, and detect our entrypoint via environment variable. Proof of concept here.

Problems with these:

  1. Sourcing shell scripts are tricky and easy to get wrong.
  2. I noticed we don't use symlinks anywhere in our repo. Not sure if these links could get broken by one of the release channels.
  3. Additional overhead of one more wrapper script.

@gspencergoog WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Symlinks scare me. Option number 2 works fine until someone makes their own symlink to point to them (either to the symlink or the launcher.sh). I think I'd rather we went with option number 1, even given the pitfalls. Option 3 could also work, but I don't think that's much different from option 1, really. You're exec'ing instead of sourcing, but the problems with safely locating the script to invoke are the same.

I'd like to reiterate the principle of putting as little logic as possible into these wrappers, however. We should start Dart and use that for anything complex as soon as we've bootstrapped far enough to be able to do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, I endorse all the things @jcollins-g said above.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, sounds like we should go with option 1, following @jcollins-g's guidelines.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking we should also setup CI to shellcheck all our shell scripts.


retry_upgrade

"$DART" $FLUTTER_TOOL_ARGS --snapshot="$SNAPSHOT_PATH" --packages="$FLUTTER_TOOLS_DIR/.packages" --no-enable-mirrors "$SCRIPT_PATH"
Copy link
Contributor

@christopherfujino christopherfujino Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need $FLUTTER_TOOL_ARGS (unless this ever gets set externally?)

"$FLUTTER_ROOT/bin/internal/update_dart_sdk.sh"
VERBOSITY="--verbosity=error"

echo Building flutter tool...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how to handle this, but I expect users are going to be confused why invoking dart is re-building their flutter tool.

If we exit early here, without updating $STAMP_PATH, the next time the user invokes the flutter tool it will rebuild. Although it means disrupting the user's flow twice, I think at least that flow is more expected. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should rebuild their flutter tool unless they're actually using the flutter tool.

Why do you say this disrupts them twice? Isn't it just the once when they try to invoke the flutter tool (as it is today)? I mean, sure, it invalidates the cache earlier this way, but I don't think the user will notice that until they run flutter, and the flutter tool is going to rebuild at most once for each cache invalidation anyhow. At least this will have updated the dart SDK earlier for them.

Plus, if they don't need to run the flutter tool, then we're wasting their time/energy/battery rebuilding it.

Copy link
Contributor Author

@jonahwilliams jonahwilliams Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keep in mind that as contributors to flutter you will hit this code path approximately one million times more often than someone who upgrades stable -> stable.

Let's keep it simple and reduce conditional logic to the minimum.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TYPO: hide -> hit

@mit-mit mit-mit requested review from devoncarew and jwren April 30, 2020 06:29
@devoncarew
Copy link
Contributor

I'm going to cc in @jcollins-g, who knows quite a bit more about bash than I do, and might have some ideas here.

@christopherfujino
Copy link
Contributor

@jonahwilliams and I talked, and we agreed that I will implement the bash part of this change, applying @jcollins-g's suggestions above, and he will implement the windows part. I suggest I merge my changes into this PR, since I think we should land both halves together and this PR has the discussion for posterity.

@jonahwilliams
Copy link
Contributor Author

@mit-mit are we going to switch to shipping the full dart SDK with flutter? Something I realized based on #56573 is that we're going to remove the ability to run dart2native if we ask developers to use this dart

@jonahwilliams
Copy link
Contributor Author

staging windows changes here: https://github.com/flutter/flutter/pull/56845/files

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

a: null-safety Support for Dart's null safety feature tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants