The selection is part of the image, so it is saved with the XCF (there are many cases where it is useful...). If you don't want it, do Select > None
before saving (or after reloading).
The border is easy to remove if you draw it on its own, initially transparent, layer (which in your case could be just adding the layer before doing Edit > Stroke selection
) Then as long as you keep the layers distinct (in other words if you don't "flatten" the image, but you normally don't need to do this, it is flattened implicitly when you export to JPG/PNG), you can remove the border by removing the layer on which it was painted (or just making that layer invisible).
In other words, to obtain this:
You do this: