Skip to content

Improve MouseTracker lifecycle: Move checks to post-frame#44631

Merged
dkwingsmt merged 63 commits intoflutter:masterfrom
dkwingsmt:postframe-mouse-tracking
Dec 2, 2019
Merged

Improve MouseTracker lifecycle: Move checks to post-frame#44631
dkwingsmt merged 63 commits intoflutter:masterfrom
dkwingsmt:postframe-mouse-tracking

Conversation

@dkwingsmt
Copy link
Contributor

@dkwingsmt dkwingsmt commented Nov 12, 2019

Description

This PR rewrites MouseTracker's lifecycle, so that mouse callbacks are all triggered in post frame, instead of the current one where some are triggered during the build phase. This change fixes a few issues of mouse events without increasing the frames it takes for a change to take effect.

This PR also changes the onExit callback to MouseRegion, RenderMouseRegion, and MouseTrackerAnnotation, so that it is no longer triggered on dispose, which is a breaking change.

Background

Currently MouseTracker's collection process happens in the following way:

  • When an annotation is attached, the collection is scheduled in the next post-frame.
  • When an annotation is detached, the collection happens immediately, which is during the build phase (with a delicate bypass to allow setState during build)
  • When a pointer event occurs, the collection happens immediately.

This process has the following issues:

  1. False-positive calls. Annotation attachments and detachments are triggered by Element.deactivate and Element.activate respectively, which can be called when the element is simply moved to a different position of the render tree without actual location change. The result is that the onExit and onEnter of the same annotation can be called back-to-back within a single frame, which is counter-intuitive. This fix is reflected by this test.
  2. Calling setState during build. Although this doesn't cause crash through the delicate bypass, it is neither an intended design, nor is this complex process beneficial, since the effect of the state change will no go live until the next frame anyway (this test passes both before and after PR). Moreover, detaching attachment can still cause crash (this test fails before PR).
  3. Not updated by new frames. When a new frame is drawn, mouse tracker's collection process is not triggered. This is reflected by this test.

Changes

MouseTracker's lifecycle

In this PR MouseTracker's collection process happens in the following way:

  • For each new frame, a post-frame callback is scheduled to collect the updates of all devices.
  • When a pointer event occurs, the collection happens immediately for the affected device.

It fixes the aforementioned issues because:

  1. Temporary attachment and detachments no longer trigger callbacks immediately; instead the collection only cares whether the annotation attachment is changed eventually.
  2. All setState are called either in a post-frame callback, or in the immediate callback of a pointer event, which is also post-frame. In fact, this PR adds an assertion to make sure the collection does not take place in the drawing phase.
  3. At every frame, all devices are marked dirty and will be collected in the upcoming post-frame.

As a result of this change, attachAnnotation and detachAnnotation no longer triggers any immediate effects, but serve as a state keeper that keeps "whether an annotation is attached" and "whether the owner object of the annotation is mounted" in sync, which is used in the next section.

The change to onExit

Since an annotation's onExit is now officially called after the annotation's removal, it's extremely easy to call setState in it and cause exceptions, because an annotation's exit can be triggered by unmounting its owner widget, which means we're essentially calling setState in Widget.dispose. However, we have to consider that it's a necessary demand and popular code pattern to call setState in onExit.

As a compromise, this PR changes onExit so that it is only called when the annotation and the pointer stops contact while the annotation is still attached. Despite the breakage, some code will not be affected by this change, while most of the others need a small degree of migration. For more information, see #44957.

TestGesture maintains "added"

This PR also changes TestGesture so that it keeps track of PointerAddedEvents and PointerRemovedEvents, so that when an event is called on a gesture that has not added pointer,

  • For a Removed event, it becomes an no-op
  • For an event of other kind (that is not an Added), an Added is inserted beforehand.

This is because many tests are written without the proper calling sequence of events, which is heavily relied upon by MouseTracker. Instead of fixing all the tests (including those of third-party), maybe it's better to make testing utilities safer.

Related Issues

Tests

I added the following tests:

  • MouseTracker
    • should correctly handle when annotation is attached or detached on the pointer
    • should correctly handle when the pointer is added or removed on the annotation
    • should correctly handle when the pointer moves in or out of the annotation
    • should correctly handle when annotation is attached or detached while not containing the pointer
  • MouseRegion
    • Regions mounted under the pointer should should take effect in the next postframe
    • Regions unmounted under the pointer should not trigger state change
    • Regions moved into the mouse should take effect in the next postframe

Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I signed the CLA.
  • I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
  • I updated/added relevant documentation (doc comments with ///).
  • All existing and new tests are passing.
  • The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
  • I am willing to follow-up on review comments in a timely manner.

Breaking Change

Does your PR require Flutter developers to manually update their apps to accommodate your change?

@fluttergithubbot fluttergithubbot added f: material design flutter/packages/flutter/material repository. framework flutter/packages/flutter repository. See also f: labels. labels Nov 12, 2019
@dkwingsmt dkwingsmt changed the title Improve MouseTracker algorithm: Move checks to post-frame Improve MouseTracker lifecycle: Move checks to post-frame Nov 12, 2019
@gspencergoog gspencergoog self-requested a review November 12, 2019 22:16
@dkwingsmt dkwingsmt added a: desktop Running on desktop f: gestures flutter/packages/flutter/gestures repository. and removed f: material design flutter/packages/flutter/material repository. labels Nov 12, 2019
@gspencergoog
Copy link
Contributor

Why can't there just be onExit and onDispose, and you listen to both if you want to know about exit and dispose both? That way only onDispose requires special handling, and we don't combine the onExit meaning with the onDispose meaning.

Also, when would I want to listen to onExit as opposed to onExitAndDispose? Wouldn't I always want to listen to onExitAndDispose? Otherwise, I'm not guaranteed to get all the exit modalities, right? So why not just change the meaning of onExit to what you're calling onExitAndDispose? I think it's a breaking change either way, and might be less confusing.

@goderbauer
Copy link
Member

Looks like cirrus is unhappy on this PR.

//
// The old states are collected on `oldWidget` if it is provided, or the
// current widget (before doing `task`) otherwise.
void _mayTriggerCallback({VoidCallback task, FocusableActionDetector oldWidget}) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think task here can just be a positional argument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _mayTriggerCallback called in didUpdateWidget does not have a task, but an oldWidget instead.

final bool didShowHoverHighlight = _shouldShowHoverHighlight(target: oldTarget);
final bool didShowFocusHighlight = _shouldShowFocusHighlight(target: oldTarget);
if (task != null)
task();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the task is calling setState, isn't it possible that it won't actually execute the contents of the setState function arg right away? Although, I'm not sure anything needs to call setState in the task anymore, so it may be moot.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think according to the doc of setState the content is run synchronously.

_handleShowFocusHighlight();
widget.onFocusChange?.call(_focused);
_mayTriggerCallback(task: () {
setState(() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I look at this, I'm not sure that it even needs to call setState, since it doesn't use the value of _focused in its build function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about this too, although setState does mark its children as dirty, so you might want this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if none of the properties are states, is there really a reason to make FocusableActionDetector a widget any more?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I think that it definitely is a good convenience widget to combine several concepts for desktop support, so it still has value.

@dkwingsmt dkwingsmt merged commit afb8f38 into flutter:master Dec 2, 2019
@dkwingsmt dkwingsmt deleted the postframe-mouse-tracking branch December 2, 2019 21:00
@Piinks Piinks added the c: API break Backwards-incompatible API changes label Dec 2, 2019
@Hixie
Copy link
Contributor

Hixie commented Dec 3, 2019

Looks like microbenchmarks stock_layout_iteration regressed on this commit.

@dkwingsmt
Copy link
Contributor Author

@Hixie I'm taking a look

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

a: desktop Running on desktop a: tests "flutter test", flutter_test, or one of our tests c: API break Backwards-incompatible API changes f: cupertino flutter/packages/flutter/cupertino repository f: gestures flutter/packages/flutter/gestures repository. f: material design flutter/packages/flutter/material repository. framework flutter/packages/flutter repository. See also f: labels.

Projects

None yet

7 participants