How to reproduce:
1. Create a repository with the “ReproProjectGit“ project
2. Open the “ReproProjectGit“ project
3. Open the “SampleScene“
4. Create an empty GameObject
5. Delete that GameObject
6. Save the Scene
7. Observe the changes in the “Assets/Scenes/SampleScene.unity“ asset
8. Commit the “Assets/Scenes/SampleScene.unity“ asset
9. Switch from “SampleScene“ to “New Scene“
10. Switch back to the “SampleScene“
11. Repeat steps 1 through 8

Expected result: No changes occur in the “Assets/Scenes/SampleScene.unity“ asset
Actual result: Different types of changes occur in the “Assets/Scenes/SampleScene.unity“ asset

Reproducible with: 2021.3.32f1, 2022.3.12f1, 2023.1.19f1, 2023.2.0b16, 2023.3.0a12

Reproducible on: macOS 13.5.2 (Intel)
Not reproducible on: No other environment tested

Notes:
- More changes occur in the “Assets/Scenes/SampleScene.unity“ asset when the “ShadowCaster2D” Prefab is nested in another “ShadowCaster2D” Prefab
- If the “ShadowCaster2D” Prefab is the only GameObject in the Scene, the changes seem to eventually work themselves out after several loops through steps 1 through 11
- The problem is not reproducible with a Scene that does not contain the “ShadowCaster2D“ Prefab
- Reproducible with Unity Version Control and GitHub Desktop