Steps to reproduce:
1. Create a new project with the HDRP template
2. Import any cubemap asset with ".exr" or ".hdr"
3. Change "Texture Shape" in the Inspector window to "Cube" and press "Apply"
4. Observe the imported cubemap

Expected result: Cubemap has a preview icon of how the asset looks like
Actual result: Cubemap has a default icon, providing no information about how the asset looks like

Reproducible with: 2019.2.18f1, 2019.3.0f4, 2020.1.0a19 (HDRP package versions previe.5.0.0 - 7.2.1)
Could not test with: 2017.4, 2018.4 (No asset preview icons)

Note: This bug seems to only occur when the project is created with the HDRP template. Afterwards, you can uninstall HDRP, the issue will still occur. Testing showed that it has no connection to the HDRP package or its version.