How to reproduce:
1. Open the user's attached project "AnimatorOverrideTest2019.zip"
2. From the Project tab, open Assets/TestScene.unity
3. Make sure that Inspector (Window > General > Inspector) and Animator (Window > Animation > Animator) windows are visible, as well as Game View
4. Enter Play mode
5. In the Hierarchy window, click on TestObject
6. In the Animator window, change TestParameter to a different number than is currently set
7. From the Project tab, drag DefaultAnimatorOverrideController and drop it in the Inspector tab, as TestObject's Animator's Controller
8. In the Animator window, check what the value of TestParameter is

Expected results: TestParameter has the same value as was set in step 6
Actual results: TestParameter reverts to the previous value

Reproducible with: 2019.3.0b1, 2019.3.16f1, 2019.4.27f1, 2020.3.8f1, 2021.1.8f1, 2021.2.0a17
Not reproducible with: 2018.4.35f1, 2019.3.0a11
Could not test with: 2019.3.0a12 due to license errors when trying to open Editor

Notes:
- The value doesn't revert to its previous state if steps 5-8 are executed when Play mode is not entered
- Documentation (https://docs.unity3d.com/2019.4/Documentation/ScriptReference/AnimatorOverrideController.html) says that "Swapping Animator.runtimeAnimatorController with an AnimatorOverrideController based on the same AnimatorController at runtime doesn't reset state machine's current state."