Complaints from a user:

PrefabUtility.SetPropertyModifications() should have better documentation since you need an understanding of how prefabs are internally referenced/stored in the context of nested prefabs and has a lot of ways you can use it that won't work at all with no warning, or will work only sometimes in some cases.

The API along with its counterpart GetPropertyModifications take a UnityEngine.Object reference, it seems like in modern unity in practice this must actually be a GameObject reference to return what you expect most of the time. This API seems to have existed prior to the nested prefab workflow so I assume it inherited some changes with the new workflow. If you supply a Component, Unity will actually return all modifications on all Components on the same GameObject, they may actually be the modifications on the outermost prefab instance but I am not certain. Moreover, it must be a GameObject reference for the outermost prefab instance root obtained by PrefabUtility.GetOutermostPrefabInstanceRoot(), otherwise modifications will apply sometimes but not always, depending on the target object, which seemed to be because applying modifications GameObject-wise when not targeting the outermost prefab instance overwrite the modifications on other children gameobjects of a given instance.

The target field of each PropertyModification must target the source object of a component to modify obtained via PrefabUtility.GetCorrespondingObjectFromSource(), otherwise it will not apply any changes. This has the easy way to shoot yourself in the foot by assuming either it is targeting the component instance on the current prefab being modified, or that it's targeting the rootmost component obtained by PrefabUtility.GetCorrespondingObjectFromOriginalSource(). In the former no modifications will be applied, in the latter modifications will be applied only if you are not working with multiple levels of nested prefabs.

I'm also not sure if the value field of PropertyModification is intended to be set in the case that you're working with UnityEngine.Object references, I have set it to an empty string as that's what GetPropertyModifications returns in such cases. I'm not entirely sure on the exact rules for the stringly typed data either, I assume it's similar to or the same as the textual representation of each given type you want to modify. I didn't need to work with this beyond string values and a UnityEngine.Object array so I'm not sure if there is more that's worth documenting.

I am not 100% certain my understanding is comprehensive but this seemed to be enough for the use cases I needed to cover.
--------------------------

See slack thread: [https://unity.slack.com/archives/C0A9NNY9W/p1669992287506099]

Note that the new documentation should have links to the improved overrides APIs:

GetObjectOverrides(GameObject prefabInstance)
GetAddedComponents(GameObject prefabInstance)
GetRemovedComponents(GameObject prefabInstance)
GetAddedGameObjects(GameObject prefabInstance)
GetRemovedGameObjects(GameObject prefabInstance)