How to reproduce:

1. Open the attached project and "main" scene in it.
2. Build Windows executable using Development Build settings.
3. Make sure to include "assets" folder (with small "a") from New Unity Project/Assets to build Data folder.
4. Run executable, pres "p" to pause and try to change quality settings.

Expected behavior: No error when changing quality settings.
Actual behavior: Error "Trying to reload asset from disk that is not stored on disk" appears when changing quality settings.

Reproducible: 5.3.3p3, 5.3.5f1, 5.3.5p8, 5.4.0b25