Steps to reproduce:
1. Create a new project under Unity 5.3.4f1
2. Move out all floating panes from main window
3. Resize main window to a minimal size (You can watch my repro video which demonstrates what to do from this point and shows result)
4. Close project
5. Open project under Unity Beta 5.4.0b17
6. Notice main window now has a minimal height that is unusually high and if you resize it, artifacts appear
7. Close project
8. Open project under Unity 5.3.4f1 again
-- Notice main window still has a minimal height that is unusually high and if you resize it, artifacts appear

reproduced with: 5.4.0b18; 5.4.0b11
Not reproducible with: 5.3.4f1; 5.3.5f1