If it's something that can be a vector graphic, then an SVG would be best. I can't imagine there being any worthwhile argument against that. I can see from comments you suggest Photoshop wouldn't open it well anyway, but Photoshop isn't for vector graphics, someone looking to modify an SVG would favor a vector editor like Inkscape.
If you have a raster or bitmap image, then it depends on what takes priority.
I recommend reading the question and answers here:
What are the best open image formats that support layers?
OpenRaster
For accessibility OpenRaster (.ora) is best because it has an open-spec and can be opened with free software. However, software like Photoshop or Clip Studio Paint don't support it. If one was keen on using Photoshop for example, they'd have to open it in GIMP, Krita or some other free software, and export it as a PSD first.
OpenRaster also has limited support for tool-specific features, a lot of things get rasterized, including text. They are rasterized to their own layers though, so it's easy to delete and re-add different text.
PSD
If you're looking to retain as much information as possible, i.e. clipping masks, text boxes, etc, then the proprietary format of the editor you're working with is probably best. Presumably Photoshop in your case since you named PSD. However, PSD is a closed-format and not all software has great support for it.
For example, if you have clipping masks in your PSD file, those that use GIMP, Krita, or ImageMagick won't be able to open it properly.