This behavior has changed since Unity 4.2.0f4. Previously pressing the space bar did not trigger focused button.

Reproduced with:
Unity Version 4.2.0f4 (38efbd14869d)
Unity Version 4.2.2f1 (54f0fc09defb)
Unity Version 4.3.3f1 (c8ca9b6b9936)

This is a desired behavior for UI systems in general.
There's a way to filter events, keypresses etc. and that should be used instead.

For filtering user hitting space bar:

if (Event.current.keyCode == KeyCode.Space || Event.current.character == ' ')
Event.current.Use();