To reproduce this bug:

1. Open the attached project
2. Open 'test' scene
3. There is a script attached on toggle gameobject which:
-Sets toggle isOn to false
-Calls SetColor on the CanvasRenderer components of the toggle's child objects
4. Play the scene
5. The toggle inspector displays isOn as false, the Checkmark child object displays the enabled toggle

By Design:

"When we have a toggle the child element (the checkmark) the visibility is controlled by the canvasrenderer alpha. What you are doing is making the checkmark visible by fading the alpha in. What you should do is add a CanvasGroup to the root of the toggle and fade the alpha of that instead of the alpha on the individual CanvasRenderers."