Repro steps:
1. Create a new project and switch build target to Android
2. Make sure that only "arm64" architecture is selected under Player Settings for Android
3. Make sure scripting backend is set to IL2CPP and that "Development Build", "Script Debugging" and "Wait for Managed Debugger" options are checked
4. Build and Run on an Android device
5. In Visual Studio go to Debug -> Attach Unity Debugger

Notice that the device shows up in the list of targets to debug

6. Select all of the target architectures under Player Settings for Android
7. Repeat steps 4-5

Actual: The device doesn't show up in the list of targets to debug

Reproducible with: 2017.4.30f1, 2018.4.4f1, 2019.1.11f1, 2019.2.0b10, 2019.3.0a9

Tested and reproduced on these devices:
VLNQA00218, Samsung Galaxy Note9 (SM-N960U), Android 8.1.0, CPU: Snapdragon 845 SDM845, GPU: Adreno (TM) 630
VLNQA00230, Oneplus OnePlus6T (ONEPLUS A6013), Android 9, CPU: Snapdragon 845 SDM845, GPU: Adreno (TM) 630
VLNQA00096, Samsung Galaxy S8 (SM-G950F), Android 8.0.0, CPU: Exynos 9 Octa 8895, GPU: Mali-G71
VLNQA00009, Samsung Galaxy Note8 (SM-N950F), Android 9, CPU: Exynos 9 Octa 8895, GPU: Mali-G71