I am doing kerning manually in GIMP, i.e. trying to move letters closer to each other, but my hand movements are not exact enough on my mouse so whenever I move a layer to the left or right, it also goes a little in the vertical direction. I wonder if there is some feature that enables me to move a layer left or right (alternatively only up and down) so that I don't accidentally sway it a little down or a little up. Does anyone know? I am a beginner when it comes to GIMP.
You can use the arrow keys with the move tool as in Paolo's answer - note that if you hold shift as you press the arrow keys, the movement is made in larger steps instead of one pixel at a time.
Another way of doing it is select the move tool, start moving your layer in the direction wanted and press and hold the
CTRL key: movements in the other direction (i.e. horizontal if you started moving it on the vertical) will only happen after a large threshold in that direction - so it becomes easy to move only in one direction.
Still another option is to use image guides (click on the rulers on the sides and drag horizontal or vertical guides onto your image), position the guide along the top or bottom border of the layer you are moving, then make sure
View->Snap to Guidesis enabled , and move the layer - it will be easy to keep on track.