1. What happened

After adding the XR Plugin Management package to an HoloLens 2 project, the holograms appear in the wrong position, they drift when moving the head and also they seem to render as if the device was not correctly calibrated.
On this last point, what happens is I can see the objects rendered on the right eye at a slightly different height than the left eye. This can be "solved" by tilting the headset. Other apps render fine and I've also calibrated the device multiple times, but this issue only appears with the XRPM.

I first noticed this issue in a project that had the MRTK and Vuforia in it. After importing the XRPM, all the holograms had the issues described above. I tried different setups in empty project and was able to trace the issue to the XRPM. I've also added Vuforia because the displacement is most noticeable when using Vuforia's Image Targets. I also noticed that the displacement increases the further the target is from the head position on scene load.

It's worth noting that Vuforia works without issues on HL2 in a project with the MRTK and the old XR system.

2. How we can reproduce it using the example you attached

With the attached project, simply deploy the app on an HoloLens 2.
In an empty project:
- Import the XRPM package
- Import Vuforia
- In a scene remove the Main Camera, add an AR Camera from GameObject > Vuforia Engine
- Add the Tracked Pose Driver to the new Camera
- Set the Clear Flags to Solid Color and the Background to (0, 0, 0, 0)
- Add an Image Target from GameObject > Vuforia Engine
- In the ImageTargetBehaviour, set the Type to From Database and import the default database
- Remove the Turn Off Behaviour from the Image Target
- In the Project Settings enable the Windows Mixed Reality plugin and tick "Initialize XR on Startup"
- In the WIndows Mixed Reality settings set the Depth Buffer to 16 Bit and tick Shared Depth Buffer
- In Player > Other > Capabilities select at least WebCam and InternetClient
- In Supported Device Families select Holographic
- Add the scene to the Build Settings, switch platform to UWP, then build and deploy the app for HL2 from Visual Studio 2019
- In the app, look at the Vuforia Astronaut target and see that the card augmentation is displaced.

You can find the images in Assets/Editor/ForPrint/ImageTargets/target_images_USLetter.pdf. You can either print the pdf or just have it open on the screen without scaling it.