@markrian The screen readers all tested well and announced the alert as expected. That said, I'm still pondering if there's a wrinkle we're not seeing because we're testing in isolation.

The alert has a role="alert" which makes it assertively announce itself to screen readers. Setting focus and disrupting the user for an important announcement feels correct.

Where this gets a bit tricky is if we're using the focus prop on a regular alert that has the role="status". This one will not announce itself to Chrome + JAWS correctly; there's an issue with setting focus on containers instead of a text node. When Chrome + JAWS receives focus on a status alert, nothing is announced, which is not great. It's an edge case but a case nonetheless.

Maybe the right answer is documenting guidance to only set focus when it's worth an interruption to the user (IE, using role="alert").

I'm at my EOD so I'll pick this up in the morning after you've had a chance to review and comment.

The politeness prop should be rendering string "assertive" with a role="alert"] but it's still showing "polite". Might be outside the scope of this MR, I'm okay making it a follow on item if so.

---

Sure thing @jaycooney. I think it'd be a good opportunity for us to reach out to the customer who reported the issue, either through their company support or social media, to let them know it's been fixed.

@mrincon Great question. I went with the full month printed out because screen readers do a really spotty job announcing month abbreviations. Of the big three I test with (VoiceOver, NVDA, JAWS) only one announces "Jan" as "January" somewhat consistently. Spelling out the full date felt like a better approach than trying to bolt together spans to show the abbreviation visually and the full month non-visually.

Trevor Pierce (8f1f5d61) at 17 Mar 20:14

@seggenberger Would you mind giving this one a look when you've got some free time?

Thanks for the heads up @mrincon. I thought I had that one bracketed but seems not. gitlab-org/gitlab!227686 (merged) is in for a long-term quarantine. I'll take another run at getting those tests up to par.

What does this MR do and why?

MR adds long-term quarantine to collapsed left nav mouse and keyboard specs in spec/features/nav/pinned_nav_items_spec.rb. MR is a follow on to the fast quarantine while test flakiness root cause is remediated.

References

• #593872
• gitlab-org/quality/engineering-productivity/fast-quarantine!510

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #593872

Trevor Pierce (c9aecf54) at 17 Mar 15:22

@ddieulivol gitlab-org/gitlab#593872 is filed and I'm adding the information in a branch for long-term quarantine. MR should be up in the next couple of hours.

@ddieulivol No objections. I'll use the discussion here and on the incident report to guide refactoring the tests in a follow on MR to shore up the wait behaviors.

Fast-quarantine process

• Edit the rspec/fast_quarantine-gitlab.txt file and add a test entity identifier, in one of the following formats:
  • An example id, e.g. spec/tasks/gitlab/usage_data_rake_spec.rb[1:5:2:1]
  • A test file with a line number, e.g. ee/spec/features/boards/swimlanes/epics_swimlanes_sidebar_spec.rb:42
  • A test file, e.g. ee/spec/features/boards/swimlanes/epics_swimlanes_sidebar_spec.rb
• Commit the change to a new branch and open a merge request with a description on why the test is fast-quarantined, with a link to the flaky test issue.
• You can merge the merge request yourself or ask for review/approval/merge, as you see fit.
• Once merged, to immediately unblock a pipeline:
  • For RSpec tests (unit/integration/system): First re-run the retrieve-tests-metadata job and wait for it to complete, then retry the failed test job
  • For E2E tests: Simply retry the failed job (E2E tests automatically download the latest fast quarantine file each time a job is run)
  • Alternative: Run a new pipeline
  • See the README for more details
• BEFORE 10:00 AM UTC SUNDAY: Create a long-term quarantine merge request for the flaky test you just fast-quarantined.
  • ⚠️ Fast quarantines are automatically cleared every Sunday at 10:00 AM UTC
  • You will receive an automated comment after merging with instructions for creating the permanent quarantine
  • This is critical to ensure the flaky test doesn't start failing again after the fast quarantine is cleared

Why is this test fast-quarantined?

8 tests in spec/features/nav/pinned_nav_items_spec.rb from two describe blocks ("keyboard behavior with collapsed sidebar" and "mouse behavior with collapsed sidebar") are fast-quarantined. These account for all high-volume failures (17-32 blocked pipelines/day each). The remaining tests in the file have 0-1 blocked pipelines/day.

• Incident: gitlab-org/quality/analytics/master-broken-incidents#732
• Failing since Feb 4 (6+ weeks). 178 pipelines blocked in the last 90 days.
• Fix attempt gitlab-org/gitlab!226740 merged today but did not resolve the failures.
• Root cause: section_button.hover does not reliably trigger flyout mouseenter in headless Chrome, causing 30s Capybara timeouts.

Quarantined tests

keyboard behavior with collapsed sidebar (line 200):

• :208 — opens and closes flyout menu with Enter key
• :215 — opens and closes flyout menu with Space key
• :222 — returns focus to section button after closing flyout with Escape
• :232 — pins item from flyout menu using Enter key
• :261 — removes pinned item from pinned section using Space key

mouse behavior with collapsed sidebar (line 301):

• :309 — allows pinning items from flyout menu with mouse hover and click
• :341 — allows unpinning items from pinned section flyout with mouse hover and click
• :378 — allows unpinning items from their original section flyout with mouse hover and click

cc @trevorpierce

Trevor Pierce (094ace86) at 16 Mar 19:13