Reproduction steps:
1. Open the attached “repro_IN-89373“ project
2. Open the “Assets/Scenes/SampleScene.unity“ Scene
3. Open the Test Runner window (“Window → General → Test Runner“)
4. In the Test Runner window, select the “PlayMode“ tab
5. Press the “Rerun Failed“ button in the bottom-right

Expected result: Only failed tests are rerun
Actual result: All tests are rerun

Reproducible with: 2021.3.46f1, 2022.3.54f1, 6000.0.31f1, 6000.1.0a8

Reproducible on: Windows 10 (22H2), macOS 14.1 (user’s)
Not reproducible on: No other environments tested