The main trick, in my experience, to adding smooth transparency to an image in GIMP is using the Layer → Transparency → Color to Alpha... tool. Of course, you have to know how to use it to good effect — on its own, all it does is make your images look all funny and translucent.
If I take the image you posted above, and just run Color to Alpha on it (picking white for the transparent color, of course), what I get is this:
As you can see, this image looks fine on a light background. Unfortunately, putting it on a dark background makes the gray elements vanish completely, and the blue parts don't look so great either:
The problem is that the Color to Alpha tool did what it was supposed to do: it converted all the white in the original image into transparency. This means that the gray lines became semi-transparent black lines, and the light blue became semi-transparent dark blue.
What we really want, however, is presumably that the basic colors of the text and the other elements of the logo should stay opaque, and only the anti-aliased pixels around their edges should become semi-transparent. To fix this, we need to add some white back into the colors.
One way to achieve that, for an image like this where the opaque areas consist mostly of single colors, is this:
Duplicate the layer.
On the lower layer, convert the transparency to a mask by doing Layer → Mask → Add Layer Mask... and selecting "Transfer layer's alpha channel".
After transferring the transparency to a mask, make the lower layer completely white (e.g. using the Bucket Fill tool in "Fill whole selection" mode).
You've now added some white to all the colors of the image, but the interiors of the letters and figures still aren't completely opaque. To make them so, we need to normalize the mask on the lower layer — but, since the different parts of the image have different colors and lightnesses, we need to do it separately for each part.
To do that, click on the mask of the lower layer in the Layers dialog to edit it, use the Rectangle Select tool to select each distinct part of the image (the "B", the "NANO" and the drawing above them) in turn, and run Colors → Auto → Normalize on each selection.
After doing all that (and optionally merging the layers), the result should look like this:
If you compare this with the first image above, there's almost no difference to be seen. But see what happens when we put it on a black background:
Now the colors look opaque, but the background is still transparent and the edges are smooth.
Unfrotunately, you can also see some gray fringing around the letters, especially the "B". I suspect this is mostly because the original JPEG image already suffered from chroma loss in those areas due to lossy compression, it just wasn't so apparent on a white background. There are two ways (that I know of) to try to fix that: you can either manually adjust the mask color levels on the white layer to reduce the fringing, or you can try sampling the solid colors from the letters in the original image and replacing the white on the lower layer white those solid colors (Rectangle Select, Bucket Fill). Or you can even try both.
However, all this is really something you should only try if you have no other choice. A much better solution is to try and find the original vector (AI, SVG, EPS, PDF, etc.) files from which these logos were surely rendered — they should have full transparency information, be free of compression artifacts, and be scalable too! Only if it's really impossible to obtain the originals should you even consider working from low-resolution JPEG files like these.
Also, even if you do end up using the bitmaps, you could still get a much cleaner result by redrawing some of the elements — in particular the text, which seems to be simple Copperplate.