Reproduction steps:
1. Open the user's attached project
2. Select Window/Analysis/Profiler and enable Deep Profile
3. Set Profiler mode to Editor and start recording
4. Select Window/Asset Management/Addressables/Groups and Build/Clean Build/All
5. Select Build/New Build/Default Build Script (observe the build time)
6. Stop recording with Profiler and select the frame which shows AddressablesBuild step (should be the biggest spike)
7. In the Addressables Groups window, select Build/Clean Build/All
8. Select Build/New Build/Build Default Script (observe the build time)

Expected result: The build time is the same as previous
Actual result: The build takes longer (Calculate Asset Dependency Data is very slow)

Reproducible with: 2018.4.28f1, 2018.4.32f1, 2019.4.3f1, 2019.4.20f1, 2020.1.17f1, 2020.2.0b2, 2020.2.5f1, 2021.1.0b8, 2021.2.0a6
Not reproducible with: 2018.4.27f1, 2019.1.0a10, 2019.4.2f1
Could not test with: 2019.1.0a1, 2019.1.0a9, 2020.2.0a1, 2020.2.0b1 (SBP errors)

Notes:
1. With 2018.4.28f1 - 2018.4.32f1 and 2020.2.0b2 - 2021.2.0a6 more data in heap memory is needed to reproduce the issue (that's why Step 6 is needed)
2. With 2019.4.3f1 - 2020.1.17f1 less data in heap memory is needed to reproduce the issue (so selecting AddressablesBuild step/biggest spike is not necessary)