Steps to reproduce:

1. Open attached project "784006.zip"
2. Open scene "repro"
3. In hierarchy, notice "Cube"
4. Expand it and select both of its children
5. In scene view, while holding down ctrl, use area select tool to deselect Capsule and Sphere. Notice that they get deselected as expected
6. Now drag "Cube" to project tab to create a prefab
7. In hierarchy, select Sphere and Capsule once again
8. In scene view, while holding ctrl, use area select tool to deselect these objects again. Notice that this time they aren't deselected

Note: Bug is also present when prefab is deleted

Reproduced with: 5.2.4f1, 5.3.4p1, 5.4.0b12