Given a test file like:

import 'package:test/test.dart';

void main() {
  test('1', () => expect(true, false));
}

If you:

• Run with the Dart: Run All Tests command
• Rename the test to 2
• Re-run with the Dart: Run All Tests command

The old result for 1 remains in the test tree. Yet if you run the file with F5 or the Run CodeLens from the main function, it correctly is removed.

Possibly this is due to the "is new test run" flag (or the potentially deleted flags) not being set by the Run All Tests command?