Ok, let's try to write an answer...
0. Some technical background
Many image file formats can actually preserve the color values of transparent pixels. PNG, TIFF, TGA, BMP are among them, as well as the working formats of most (or more likely all) serious image manipulation applications.
You can think of pixels as being colored and transparent according to a RGBA quadruple, with Red, Green, Blue and Alpha in a range of e.g. 0-255, 0.0-1.0, 00-FF or whatever is suitable (advanced user will notice that these examples are somewhat 8-bpc-centric)).
The A value being 0, usually meaning fully transparent, does not make the RGB values of the pixels useless per se, you might just not see them at the moment..
1. The building block
So let's take a basic PNG image with some colors and some transparency. The one here has been created by some solid noise clouds, mapped to a pastel full saturation gradient, and had the original solid noise clouds applied as it layer mask:
If you load this image in GIMP or a comparable application, and check the color values of the transparent (in your browser: background color, white for me) parts, you will notice that the RGB color values are intact. You can use the color picker tool for this.
As long as you stick to only one layer, you can apply tools like the eraser or procedures like layer masks to it - the colors or transparent pixels will be preserved. If you lock the transparency in the layers dialog, you can even use paint tools to change the color, without affecting the transparency.
What you can't do is use paint tools with opacity set to 0 - this won't have any effect.
2. How to fail
So this is nice and fine as long as you stick to one layer. But what if you want to do more, for example add more layers, e.g. some text? Well, usev the text told to do just this, export as PNG and get:
Get the image above loaded in GIMP and examine the transparent parts, and you'll see what happened. Looks ok, but this is one of the easiest ways to fail.
Add a layer, and the transparent parts will be invisible black if you merge the layers into one - something that is done in the file export plug-ins, for example. A somewhat arbitrary technical decision, but so far no one has changed it yet. And changing it in a way that make the results predictable might be some kind of challenge.
3. How to succeed
There is a workaround for this, however. The one method to preserve the image as-is is copy/cut&paste - somewhat logical, because you do not want to copy or cut a part of an image, paste it somewhere else and find some arbitrary image data altered, except the parts where you cut from and where you paste, of course.
So, if you have added a text layer, you can cut it from the image, paste it immediately, and then anchor it to the one layer you image is supposed to have. Examine the resulting image below, you will find that the transparent parts have retained their color data:
This workflow requires some planning and time, of course - you might find yourself setting the image up as a proper XCF file with multiple layers first and save it so, and then copy the layers over into a new, temporary file one by one, anchoring them in the proper order after each paste. But you will preserve the color data of transparent pixels this way.