Reproduction steps:
1. Open attached "HDRP-Unused-RenderTexture-Allocation.zip" project and Scenes/rtt-shadows scene
2. Enter the Play mode
3. In the Hierarchy window, select RTT Camera
4. In the Inspector, enable Lighting/Shadow Maps
5. Open Profiler and select Memory
6. Select Take Sample Playmode and observe the memory usage of Not Saved/Render texture
7. In the Hierarchy window, select RTT Camera and disable Shadow Maps
8. Select Take Sample Playmode and observe the memory usage of Not Saved/Render texture

Expected result: RenderTexture memory decreases or stays the same
Actual result: RenderTexture memory usage increases

Reproducible with: 10.2.1 - 11.0.0(2020.2.0f1, 2021.1.0a10)
Not reproducible with: 7.5.3 - 10.1.0(2019.4.16f1, 2020.1.17f1, 2020.2.0f1)