Just a thought, but if the joystick is pressed in any direction, or the fire button is pressed down when you switch the atari on, the keyboard goes a bit silly then.
Perhaps the joystick or ports are buggered?
I found this out when I had AUTO-FIRE switched on!