Using Gimp, I want to make a small part of an image transparent. However, when I use colour to alpha, it turns the whole image partially transparent by removing the colour from all pixels.
Is there a way to stop this?
I think what you are looking for is Layer mask:
A Layer Mask is linked to a layer and hides part of the layer from the picture. What is painted black on the adjustment layer will not be visible in the final picture. What is grey will be more or less transparent depending on the shade of grey. As the layer mask can be both edited and moved around independently of both the background layer and the layer it applies to, it gives the user the ability to test a lot of different combinations of overlay.
Or, if you want to get more technical, you can read more about Layer masks on GIMP docs.
There are many videos about the actual process of creating Layer mask in GIMP.