Steps to reproduce:

1. Create new project
2. Maximize editor/make it fullscreen
3. Reset Layout to Default
4. Add empty game object to heararchy
5. Add child game object to it
6. Select parent GO create in step 4
7. Change it's layer
8. When asked if layer of child should also be changed click Yes
9. Notice invalid operation exception in console:
InvalidOperationException: Operation is not valid due to the current state of the object
System.Collections.Generic.Stack`1[UnityEditor.PropertyGUIData].Pop ()
UnityEditor.EditorGUI.EndProperty () (at /Users/builduser/buildslave/unity/build/Editor/Mono/EditorGUI.cs:4885)
UnityEditor.GameObjectInspector.DoLayerField (UnityEngine.GameObject go) (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/GameObjectInspector.cs:439)
UnityEditor.GameObjectInspector.DrawInspector () (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/GameObjectInspector.cs:215)
UnityEditor.GameObjectInspector.OnHeaderGUI () (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/GameObjectInspector.cs:138)
UnityEditor.Editor.DrawHeader () (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/Editor.cs:355)
UnityEditor.InspectorWindow.DrawEditor (UnityEditor.Editor editor, Int32 editorIndex, Boolean rebuildOptimizedGUIBlock, System.Boolean& showImportedObjectBarNext, UnityEngine.Rect& importedObjectBarRect) (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/InspectorWindow.cs:1110)
UnityEditor.InspectorWindow.DrawEditors (UnityEditor.Editor[] editors) (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/InspectorWindow.cs:1024)
UnityEditor.InspectorWindow.OnGUI () (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/InspectorWindow.cs:359)
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
UnityEditor.PopupCallbackInfo:SetEnumValueDelegate(Object, String[], Int32)

Tested on OSX. Not reproducible when editor isn't maximized.

Reproduced with: 5.6.0b5
Not reproducible with: 5.5.1f1