Reproduction steps(GIF attached):
1. Open attached project.
2. Open "1" scene.
3. Select "Main cameraa" game object in the hierarchy.
4. Go to "GameObject > Align View To Selected".
5. Double click "Cubeb".
6. Double click "Cubea".
7. Select "Hand" Editor manipulation tool.
8. Click and hold left mouse button on the scene view.
9. Slowly drag it to the left side.
10. Notice the twitching.

Note:
* Editor camera twitches more on higher distances.
* Skip step 6 to increase the distance 5 times.

Reproduced with:
5.3.6f1
5.4.0f2,
5.5.0a4