Search Issue Tracker
Won't Fix
Votes
2
Found in
5.5.0b10
Issue ID
848959
Regression
No
"AlphaToMask On" in a cutout surface shader removes shadow casting using DX11
How to reproduce:
1. Open attached project "case_848959-ColorMask.zip"
2. Open "m" scene
3. Observe the Scene view
Expected behavior: GameObject using a cutout shader with "AlphaToMask On" can cast shadows
Actual behavior: GameObject using a cutout shader with "AlphaToMask On" can't cast shadows
Note: Works as expected using Direct3D9 and OpenGLCore
Reproducible: 5.6.0a2, 5.5.0b11, 5.4.2p4, 5.3.7f1
Add comment
All about bugs
View bugs we have successfully reproduced, and vote for the bugs you want to see fixed most urgently.
Latest issues
- Articulation Body with 'Revolute' Joint Type has erratic behavior when Upper Limit is set to above 360
- WebGL Player fails to render Scene when Terrain with Detail Mesh is added and WebGPU Graphics API is used
- Inconsistent errors are logged when different types are passed into the Query "Q<>" method in UIToolkit and the ancestor VisualElement is null
- Crash on GetMaterialPropertyByIndex when opening a specific Scene
- Discrepancies in the styling are present when using a TSS file instead of a USS file in custom EditorWindow
Resolution Note:
The problem is in Unity's shadow caster pass.
AlphaToCoverage expects a valid non-zero value returned from the pixel shader, but Unity's shadow pass returns 0 (and gets ALL the pixels "killed" because "coverage" is 0 >>> no shadows in dx11).
Easy workaround:
Shader "Custom/AlphaToMaskCutout" {
Properties {
_Color ("Color", Color) = (1,1,1,1)
_MainTex ("Albedo (RGB)", 2D) = "white" {}
_Cutout ("Cutout value", Range(0, 1)) = 0.5
}
SubShader
{
Tags { "RenderType"="Opaque" }
LOD 200
AlphaToMask On // This line kills shadows
CGPROGRAM
#pragma surface surf Standard fullforwardshadows alphatest:_Cutout
#pragma target 3.0
sampler2D _MainTex;
struct Input {
float2 uv_MainTex;
};
fixed4 _Color;
void surf (Input IN, inout SurfaceOutputStandard o) {
fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
o.Albedo = c.rgb;
o.Alpha = c.a;
}
ENDCG
// Pass to render object as a shadow caster
Pass
{
Name "Caster"
Tags { "LightMode" = "ShadowCaster" }
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma target 2.0
#pragma multi_compile_shadowcaster
#include "UnityCG.cginc"
struct v2f {
V2F_SHADOW_CASTER;
float2 uv : TEXCOORD1;
};
uniform float4 _MainTex_ST;
v2f vert( appdata_base v )
{
v2f o;
TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
return o;
}
uniform sampler2D _MainTex;
uniform fixed _Cutout;
uniform fixed4 _Color;
float4 frag( v2f i ) : SV_Target
{
fixed4 texcol = tex2D( _MainTex, i.uv );
clip( texcol.a * _Color.a - _Cutout );
// this is the key, returning something nonzero for Alpha2Coverage.
return texcol.a; // or a constant like 1.0 for stability
}
ENDCG
}
}
}