To reproduce:
1. Delete the "NewUnlitShader X.shader" files from the assets folder
2. Delete everything from Assets/StreamingAssets/Test1 and Assets/StreamingAssets/Test2 folders.
3. Delete the library folder
4. Open the project
5. Press Tools > TestBinaryReproducibility
6. Press "Run Test 1"
7. Close Unity
8. Delete library folder
9. Delete the "NewUnlitShader X.shader" files from the assets folder (again!)
10. Press "Run Test 2"
11. Press "Compare Built Bundles"
12. The message in the console should now indicate that the bundles are different.

Notice that this doesn't happen if you untick Static Batching in Player Settings.