Some ideas:
Your requirements "looks" detailed, but they are not detailed enough. Let me explain.
PSD files don't open well on software like GIMP and ImageMagick due to the issues with clipping masks
A clipping mask is a vector shape that contains stuff inside, either another vector shape or a raster image.
In this case, I would say SVG is your best option. But I really doubt you actually need a clipping mask. You probably need a layered mask.
But you probably do not really need a layered mask either. Probably simply the alpha channel will work.
You need to define if you need, for example blending modes. Some blending modes are easier to implement than others, even a browser can handle them.
But things do not look soo good if you need a special filter to be used "on the fly" (not already rendered into a layer), this we probably need a specific file format used on an application. In this case, the original requirements should have been using Gimp in the first place.
I am not sure if Image Magic can open Gimp files. Probably converting from PSD to Gimp will be a good option, flattening some specific layers with PSD only features.
Another option is simply PNG... Remember that Fireworks used PNG as a file format. Yup. Those PNG files had layers, and masks and texts, and whatever strange things inside and they were PNG files. I even asked Greg Roelofs if the file was, and he told me that it was. Of course, you would need to program the features you want and cry when no other program recognizes that feature.
If you only need alpha channels, you probably can implement some kind of 7zip file with simply some PNG files inside. and prepare a script to layer them on your project declaring the blending mode or something. Which sends me back to SVG.
What I mean, define what specific features you expect with the term "layer".