I am trying to move an object in straight lines as in the image below but the Bezier handles won't make this possible. How can I get rid of them and move my shape in straight lines with no curves whatsoever? Is there a way that they (bezier handles) can be turned off completely also?
I assume you're talking about the bezier path created by position keyframes.
Something like this:
Select the keyframes and alt + click any of the points with the Pen Tool and you'll get this:
You can alt + drag the points/handles to control the bezier just as you can in Illustrator, so you can—for example—do something like this:
You can also right click the keyframes and select Keyframe Interpolation for a few more options. Spacial Interpolation is what controls the positional movement ("Linear" will give you no curve), but you can also change the Temporal Interpolation, which controls the timing: