Thank you for reporting a bug to Unity.

This behavior is actually as designed.
The 'OnValidate' method should primarily be used to validate the data and not to edit it.
Doing the edit at this point will not be taken into account and the dirty flag will not be set on the scene because it is still in the loading process.

If you want to edit a scene from a script on a GameObject in your scene you need to add the [ExecuteInEditMode] attribute on your MonoBehaviour class and register yourself to the EditorSceneManager.sceneOpened event.

Here is an updated example of your script to do so.
Please note that due to the nature of Events, you might need to do a domain reload to prevent accessing an old instance of the 'Test' component.

[ExecuteInEditMode]
public class Test : MonoBehaviour
{
[SerializeField] private List<BoxCollider2D> boxColliders;

void Awake()
{
EditorSceneManager.sceneOpened += OnSceneLoaded;
}

void OnSceneLoaded(Scene scene, OpenSceneMode mode)
{
EditorSceneManager.sceneOpened -= OnSceneLoaded;
TryToModify();
}

[ContextMenu("Modify")]
public void TryToModify()
{
// Same code here
}
}

Today we will be closing this case as "As Designed". Thank you again for taking the time to report this issue, and please let us know if there is anything else that changes the context or severity of this issue.

Thank you for reporting a bug to Unity.

This behavior is actually as designed.
The 'OnValidate' method should primarily be used to validate the data and not to edit it.
Doing the edit at this point will not be taken into account and the dirty flag will not be set on the scene because it is still in the loading process.

If you want to edit a scene from a script on a GameObject in your scene you need to add the [ExecuteInEditMode] attribute on your MonoBehaviour class and register yourself to the EditorSceneManager.sceneOpened event.

Here is an updated example of your script to do so.
Please note that due to the nature of Events, you might need to do a domain reload to prevent accessing an old instance of the 'Test' component.

[ExecuteInEditMode]
public class Test : MonoBehaviour
{
[SerializeField] private List<BoxCollider2D> boxColliders;

void Awake()
{
EditorSceneManager.sceneOpened += OnSceneLoaded;
}

void OnSceneLoaded(Scene scene, OpenSceneMode mode)
{
EditorSceneManager.sceneOpened -= OnSceneLoaded;
TryToModify();
}

[ContextMenu("Modify")]
public void TryToModify()
{
// Same code here
}
}

Today we will be closing this case as "As Designed". Thank you again for taking the time to report this issue, and please let us know if there is anything else that changes the context or severity of this issue.