Steps to reproduce:

1. Open attached project
2. Open scene "LightmapTest2"
3. Open Lighting window and build lightmap
4. Notice that bottom branches of the tress are unaffected by ambient lighting (ambient lighting is intentionally overexposed to show trangles that are affected and those that aren't)

Additional steps:
5. Disable Baked GI
6. Build Lightmap
7. Notice that whole tree is overexposed, which means that all branches are affected by Ambient light

Reproduced with: 5.1.4f1, 5.2.4f1, 5.3.4p1, 5.4.0b12 (a little different result compared to previous versions)