Steps to reproduce:
1. Open attached project "reference_bug.zip"
2. Open "0" scene
3. Select "Main Camera" in the hierarchy and make sure that "test" script component has "object 1" references
4. Open "test" script
5. Change "public GameObject object2" to "public BoxCollider2D object2" and save script
6. Select "Main Camera" in the hierarchy
Result: "test" script component is showing 2 GameObject references even though one of them is BoxCollider2D

Note: this happens if the GameObject type reference is changed to any of the component type it has

Reproduced with: 5.3.1f1, 5.4.1p4, 5.5.0b9

Workaround: drag "object 1" reference once again after recompilation process