In the realm of graphic design, no such format. TIF is indeed problematic due to its nature and its not just about photoshop extensions but the way extensions are implemented. Not usually a problem in graphic design. In graphic design the real horse is PDF but while pdf may be open to standard its not exactly open and riddled with problems in your usecase.
Your requirements "looks" detailed, but they are not detailed enough. Let me explain.
PSD files don't open well on software like GIMP and ImageMagick due to the issues with clipping masks
A clipping mask is a vector shape that contains stuff inside, either another vector shape or a raster image.
In this case, I would say SVG is your best ...