Reproduction steps:
1. Open the attached project with 5.5.2f1 (case_895380-texLoading.7z or fix texture references in the original zip).
2. Reimport textures located in Assets\Stuff\textures
3. Build and run non development build
4. Press left mouse button to load between empty scene and one that has textures. First load will take more time than others - this is expected. Consecutive loads will take less. Take a note of loading times for scene with textures. On my machine that's 250-260ms
5. Open project with 5.5.2p1 or newer version
6. Reimport textures located in Assets\Stuff\textures
7. Build and run non development build again
8. Press left mouse button to load scene with textures for few times. First load will take more time than others - this is expected. Consecutive loads will take less. Take a note of loading times. On my machine I get around 370ms

Reproduced with: 5.5.2p1, 5.5.2p4, 5.6.0f3
Regression introduced in 5.5.2p1 (9360c5517afe)
Not reproducible: 5.5.2f1
Note: It seems to be relevant to RGB textures and ONLY if you reimport them.