Skip to content

Vend Flutter module App.framework as a local CocoaPod pod to be installed by a host app#36793

Merged
jmagman merged 9 commits intoflutter:masterfrom
jmagman:app-framework-pod
Jul 30, 2019
Merged

Vend Flutter module App.framework as a local CocoaPod pod to be installed by a host app#36793
jmagman merged 9 commits intoflutter:masterfrom
jmagman:app-framework-pod

Conversation

@jmagman
Copy link
Member

@jmagman jmagman commented Jul 24, 2019

Description

Previously, the add-to-app flow installed Flutter and all plugins as pods to the host app. To access the Flutter sub-app App.framework (dylib containing the compiled dart code), it added this line to the host app's xcconfig:

#include "MyApp/flutterapp/.ios/Flutter/Generated.xcconfig"

This gave the host app access to build settings like FLUTTER_ROOT, FLUTTER_APPLICATION_PATH, LOCAL_ENGINE, etc. This was required to set the right environment variables to be used in the xcode_backend script.
However, it overwrote the host app's SYMROOT to ${SOURCE_ROOT}/../build/ios which caused all kinds of horrible issues since that directory didn't exist relative to the host app's SOURCE_ROOT.
Additionally, the host app developer needed to add a script build phase to run xcode_backend.sh build and xcode_backend.sh embed.

This change:

  1. Vends the App.framework as a local pod, just like Flutter.framework. CocoaPods then handles thinning and embedding that framework into the host app, so no more need for xcode_backend.sh embed
  2. Still calls xcode_backend.sh build from a new generated script with all the necessary environment variables. Add the build script automatically to the user target (host app). No Flutter build settings are needed by or available to the host app.
  3. No more Podfile post_install hook (see Support Add2App for iOS even if the main Podfile already has post_install #32791). (Disabling bitcode in just the host app seemed to just work...?)
  4. podhelper.rb no longer assumes directories in in .ios (could be in editable ios directory).

Related Issues

Fixes #35901
Fixes #33094

Relevant to:
#33121
#28849
#24366

Will supersede:
#30966
#32791

Tests

Updated:
packages/flutter_tools/test/general.shard/project_test.dart
packages/flutter_tools/test/general.shard/commands/create_test.dart
packages/flutter_tools/test/general.shard/ios/xcodeproj_test.dart

Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I signed the [CLA].
  • I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
  • I updated/added relevant documentation (doc comments with ///).
  • All existing and new tests are passing.
  • The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
  • I am willing to follow-up on review comments in a timely manner.

Breaking Change

Does your PR require Flutter developers to manually update their apps to accommodate your change?

  • Yes, this is a breaking change (Please read [Handling breaking changes]).
  • No, this is not a breaking change.

Podfiles will need to be updated:

load File.join('my_flutter', '.ios', 'Flutter', 'podhelper.rb')
target 'MyApp' do
  install_all_flutter_pods 'my_flutter'
end

This will require an update to the add-to-app docs.
https://github.com/flutter/flutter/wiki/Add-Flutter-to-existing-apps#add-your-flutter-app-to-your-podfile

  1. Podfile instructions will change.
  2. Remove "Add a build phase" section since it's added automatically by calling install_all_flutter_pods https://github.com/flutter/flutter/wiki/Add-Flutter-to-existing-apps#add-a-build-phase-for-building-the-dart-code

@jmagman jmagman added platform-ios iOS applications specifically tool Affects the "flutter" command-line tool. See also t: labels. a: existing-apps Integration with existing apps via the add-to-app flow t: xcode "xcodebuild" on iOS and general Xcode project management labels Jul 24, 2019
@jmagman jmagman requested review from dnfield and xster July 24, 2019 01:30
@jmagman jmagman self-assigned this Jul 24, 2019
@fluttergithubbot fluttergithubbot added the c: contributor-productivity Team-specific productivity, code health, technical debt. label Jul 24, 2019
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could have called install_all_flutter_pods flutter_application_path if the application path matched the project name. I expect most of them to match, but I added this workaround in case they don't.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semi-tangential: do install_flutter_plugin_pods and install_flutter_application_pod need to be separate commands? Do people ever want to do one and not the other?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now install_flutter_plugin_pods is used in the sub-project Podfile for the stand-alone Runner app to just pull in Flutter and the plugins, but not the App.framework.
BUT this question just made me reconsider this. Why is the Runner sub-app special? Why not let CocoaPods hande embedding Flutter and App there too? Why the generated xcconfig? Why add Flutter and App to the app itself?

The sub-app works if I let the App be embedded as a pod, so why not do that? When I do I can remove:

  1. All the script phases that call xcode_build. CocoaPods handles thinning.
  2. The embed build phases.
  3. Flutter.framework and App.framework as files in the Xcode project

And if that's the case, is there any reason to not do that for all Flutter Runner apps, not just module versions?
@dnfield

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, although right now we aren't running pod install if there are no plugins, and I wouldn't want to add more CocoaPods paths...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though solving this (not embedding the frameworks explicitly if we're in CocoaPod world, and letting CocoaPod embed and thin) would fix the underlying issue of #20685, which is that two build phases want to output a Flutter.framework.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BUT this question just made me reconsider this. Why is the Runner sub-app special? Why not let CocoaPods hande embedding Flutter and App there too? Why the generated xcconfig? Why add Flutter and App to the app itself?

I think it might be for the minor shortcut where the Runner can run for people without plugins and without installing cocoapods

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CocoaPods now adds this to the host app on a pod install

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok cool. Thanks for the explanation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New script looks like

#!/bin/sh
# This is a generated file; do not edit or check into version control.
export FLUTTER_ROOT=/path/to/flutter
export FLUTTER_APPLICATION_PATH=/path/to/host/my_flutter
export FLUTTER_TARGET=lib/main.dart
export FLUTTER_BUILD_DIR=build
export SYMROOT=${SOURCE_ROOT}/../build/ios
export FLUTTER_BUILD_NAME=1.0.0
export FLUTTER_BUILD_NUMBER=1
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok. Perhaps just use a template file? That's where people would know to look at for these files. String building in a function is a bit more opaque.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, is the intent here:

whereas previously, we set these in generated.xcconfig so everything we do during the xcodebuild had these variables, we're now trying to confine them to the flutter tools session? If so, I think it's right but it's a breaking change (in case people tacked on other processes that depends on these variables). We should point this out in the breaking change announcement if so.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I didn't trace through all the call paths but do we only create this .sh file after we flutter build the module once? We should try to facilitate the case where the user flutter creates -t module, goes in their podfile, pod installs then goes in their xcode and click run right away.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it gets created when the module gets created or re-created when the module is made editable. It's available immediately.
Do you know what processes to people tack on that use these variables? I'm curious why they would need FLUTTER_ variables.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the remaining comment here is can this be in a template file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the templates are never edited after their first creation/making the module editable unless the file is missing. The only variables that are filled out in the template are things the tool knows at the time of creation, like the projectName.
The generated files like this new one and the old Generated.xcconfig are re-generated on every build. FLUTTER_FRAMEWORK_DIR gets updated to the right engine build based on the build configuration (debug, profile, release), and the build name and number are re-read from the manifest, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this comment made me realize that we can probably parse out the FLUTTER_APPLICATION_PATH in the podhelper just like it parsed FLUTTER_ROOT to find the xcode_backend script, so that path doesn't need to get passed in from the host Podfile.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh never mind, it's needed to find the podhelper script in the first place.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah interesting. I never thought about the distinction. Could be worth adding a comment here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matches the hack in xcode_backend.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😂TIL... that's wild

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually came up with an almost-identical hack independently until I saw the one in xcode_backend. CocoaPods just will not add the command to install a framework unless there's a dylib present at pod install, even before the project can be built.

@codecov
Copy link

codecov bot commented Jul 24, 2019

Codecov Report

Merging #36793 into master will increase coverage by 0.68%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #36793      +/-   ##
==========================================
+ Coverage   54.55%   55.24%   +0.68%     
==========================================
  Files         193      193              
  Lines       17849    17871      +22     
==========================================
+ Hits         9738     9872     +134     
+ Misses       8111     7999     -112
Flag Coverage Δ
#flutter_tool 55.24% <100%> (+0.68%) ⬆️
Impacted Files Coverage Δ
packages/flutter_tools/lib/src/ios/xcodeproj.dart 86.66% <100%> (+1.81%) ⬆️
packages/flutter_tools/lib/src/project.dart 82.41% <100%> (+0.23%) ⬆️
...lutter_tools/lib/src/fuchsia/fuchsia_workflow.dart 37.5% <0%> (-37.5%) ⬇️
.../flutter_tools/lib/src/android/android_studio.dart 37.12% <0%> (-14.4%) ⬇️
...ackages/flutter_tools/lib/src/commands/config.dart 71.79% <0%> (-14.11%) ⬇️
packages/flutter_tools/lib/src/cache.dart 41.83% <0%> (-0.77%) ⬇️
packages/flutter_tools/lib/src/vmservice.dart 37.3% <0%> (+0.15%) ⬆️
packages/flutter_tools/lib/src/base/logger.dart 82.44% <0%> (+0.38%) ⬆️
packages/flutter_tools/lib/src/version.dart 90.73% <0%> (+0.48%) ⬆️
packages/flutter_tools/lib/src/base/process.dart 80.46% <0%> (+0.78%) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8be195...5bb8b9c. Read the comment docs.

@jmagman jmagman force-pushed the app-framework-pod branch from 2fca97a to fbf0076 Compare July 24, 2019 17:43
Copy link
Contributor

@dnfield dnfield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overall LGTM, but we should look at seeing if we could contact Add2App customers and getting a sense for whether this would be a hard breaking change for them to work with, or whether it just doesn't matter.

@xster probably has better thoughts on that

@jmagman
Copy link
Member Author

jmagman commented Jul 24, 2019

Migration example: ios_host_app integration test.

Copy link
Member

@xster xster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I kept getting distracted so I wanted to send out half of my review first

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semi-tangential: do install_flutter_plugin_pods and install_flutter_application_pod need to be separate commands? Do people ever want to do one and not the other?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we never access any plugins via earl grey here right? Is this just a convenient packaging to go find the engine as a transient dependency by installing plugins? Perhaps we can have 3 commands, install_flutter_engine_pod, install_flutter_plugin_pods and install_flutter_application_pod for specialized things like this and users will mostly just use install_all_flutter_pods?

Copy link
Member Author

@jmagman jmagman Jul 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test target needs to embed the engine and the plugins. It's not a transative dependency, this method is explicitly installing the pod

pod 'Flutter', :path => engine_dir, :inhibit_warnings => true

I can rename. But based on my comment about embedding the App.framework as a pod in the Runner project I think I can combine the two methods into one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<I'm just curious at this point / no actions needed>what does this test do that needs the plugins?</>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I think it only needs the engine. I will make that adjustment.

Copy link
Member

@xster xster Jul 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super random question: what does CP mean? Just CocoaPods right? This isn't specifically a CocoaPods specific step right? If I just integrated using Xcode directly and I wanted to somehow click run once, I could add exactly this step in my xcode project as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These [CP*] prefixes mean CocoaPods added them automatically. So if you wanted to integrate directly, you could copy this step to get flutter build to run with the right environment variables.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok cool. Thanks for the explanation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, I see why you did the thing on top. Should this also have an optional arg when the project name isn't the same as the path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flutter application, Flutter engine.

"Flutter" is an implementation detail.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. Just "Flutter" is a confusing name. Just call it the engine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Document argument

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Describe what the assumed pwd is once this file template is written out (so future maintainers can verify your assumption)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wish the person who wrote this comment had done that, that would have saved me a lot of time!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YEEEEAAAAAAAAAHHHH!!!!!

Copy link
Member

@xster xster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general approach LGTM.

Can we create a build test for this in module_test_ios or something?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure I understood this comment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I think I get it now. This is just putting a fake stub into what pod install would put into the outer xcode project which gets fulfilled later?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😂TIL... that's wild

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume we'd have to document in the wiki that if your outer project and flutter module are source controlled separately, your other team members who check them out have to let them be in the same relative paths (or create symlinks)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The relative links here are being generated as the difference from where the outer host Xcode project is (Dir.pwd) and the current directory of this podhelper.rb file (__dir__).
Though your comment is true anyway. The Podfile needs to be checked in, which needs to pass in the path where the flutter module is located, even with the current setup: https://github.com/flutter/flutter/wiki/Add-Flutter-to-existing-apps#add-your-flutter-app-to-your-podfile

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see what you mean. The relative strings will be checked into source control since they become the inputs into the build phase. I could change this to :input_file_list_paths and pass in a generated xcfilelist, but that feature is only in Xcode 10.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, the relative-ness of the outer project and the flutter module might not be the same for all the people in a team who check these 2 gits out. We probably can't make everything 100% magical but it would be nice to account for that scenario and minimize the git dirty-ness (possibly by best practice and documentation)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we're assuming that the engine is the one coming from the cocoapods master spec repo? Or should this point to a specific local path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

podspecs don't let you specify a path. It's up to the Podfile to do that. Which is part of the reason we should take control of the Flutter CocoaPod.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work exactly? So the outer project's Podfile went to a specific directory and found the the App.framework's pod because your install_flutter_application_pod function told it where to look. Then how does the App.framework find specifically where the matching version of the Flutter.framework comes from?

It seems like it's coming from this line but wouldn't this just go to the online pod spec github repo to find the published engine?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, the Podfile is the driver, and the dependencies between the podspecs are resolved after all the explicit pods are installed. The podspec doesn't try to do anything, it just describes how the targets should be made dependent on each other in the build phases. So once all the pods are installed for a target, if there are missing pods, they will be queried and fetched from the CocoaPods' repo.
So because there's a pod 'Flutter', :path => in the Podfile, nothing additional is downloaded because the App podspec dependency is satisfied.
Screen Shot 2019-07-25 at 1 53 40 PM

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah doh, I missed that line. Thanks for explaining.

Can we add a comment here saying this acts as a requirement rather than a fulfillment mechanism and that it shouldn't do anything if podhelper does what it should do.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's just what a podspec is, CoocaPods has pretty good docs for all this.

@jmagman
Copy link
Member Author

jmagman commented Jul 24, 2019

Can we create a build test for this in module_test_ios or something?

I think it's already all being exercised in that test, it caught a few bugs.

@xster
Copy link
Member

xster commented Jul 25, 2019

This will require an update to the add-to-app docs.
https://github.com/flutter/flutter/wiki/Add-Flutter-to-existing-apps#add-your-flutter-app-to-your-podfile

We should update the wiki but also update the WIP website branch https://github.com/flutter/website/tree/staging_add-to-app when this is merged (let's chat when you're ready to start)

@jmagman jmagman force-pushed the app-framework-pod branch from ba73c24 to 74276f8 Compare July 29, 2019 20:03
@xster
Copy link
Member

xster commented Jul 29, 2019

LGTM

@BernardGatt
Copy link

This definitely makes life easier to bundle flutter & app frameworks however i'm now running into issues when adding 3rd party frameworks that have native iOS plugins, more information here: #37756

@xster
Copy link
Member

xster commented Aug 10, 2019

I just thought of something. We should probably be more strongly opinionated about whether flutter_export_environment.sh should be source controlled or gitignored (I assume more likely the latter).

@jmagman
Copy link
Member Author

jmagman commented Aug 10, 2019

I just thought of something. We should probably be more strongly opinionated about whether flutter_export_environment.sh should be source controlled or gitignored (I assume more likely the latter).

This was merged a few days later:
107f365

@xster
Copy link
Member

xster commented Aug 10, 2019

ah ok. I'm probably only seeing it because I took a previously created project and 'upgraded' it. That's probably ok.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

a: existing-apps Integration with existing apps via the add-to-app flow c: contributor-productivity Team-specific productivity, code health, technical debt. platform-ios iOS applications specifically t: xcode "xcodebuild" on iOS and general Xcode project management tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

6 participants