Reproduction steps:
1. Open the attached project "CustomColliderOffset.zip"
2. Open the "SampleScene" scene
3. Select the "CustomShape" GameObject
4. Adjust the "Offset" property on the "CustomCollider2D" component

Expected result: All the shapes in both the Game/Scene views moves with the offset
Actual result: The shapes do not move with the offset

NOTE: There is an additional minor and cosmetic issue that is also fixed under this bug report because it didn't really justify the additional overhead of a full bug report. If you notice in the screenshot attached, the Polygon Shape isn't the correct color (it's purple and not green). This is due to some specific issue when drawing the previous shape in the test project (Edge shape). This has also been fixed as can be verified by seeing all shapes are drawn in the gizmo colour (green).

Reproducible with: 2021.2.0b1, 2022.1.0a1
Not reproducible with: Feature landed in 2021.2.0b1

Reproducible with: All Editors