Currently flutter/plugins uses whatever the latest stable and master are when running tests. This means that:

• We are subject to out-of-band breakages due to Flutter changes (engine, framework, tool, and Dart have all broken us in the past).
• Re-running tests (e.g., due to flake) can actually change what is being tested, because it's not hermetic. We've had cases where we tried re-running a flaky test, only to have a flutter change break us in the meantime, causing it to be impossible to ever make that test pass for that commit (because fixing such breakage often requires a commit in flutter/plugins). This is especially problematic for our auto-publish, which can only run when all tests are green.

To prevent this, we should use pinned versions of master and stable, with auto-rollers that update them to the latest versions on a regular basis. The master version would be expected to commit more or less all the time, as with the flutter/engine->flutter/flutter roller, while the stable version would usually not need to do anything.

/cc @godofredoc Are there pointers to what's involved in setting up an auto-roller? (The read-from-a-pinned-version for CI is something I could implement pretty easily.)