My theory is that it is inheriting real-life design elements.
A lot of software interfaces, which we now call UI, has a real-life counterpart. Desk, Mouse, File, etc.
A real-life gadget needs a tactile input so you know where the button is even if you are blocking the view with the same finger you are using to press a button.
Sometimes you had a big fat crank, or you have a membrane button https://www.google.com/search?q=membrane+button which are upwards.
But sometimes having a button that goes beyond the level of the surrounding area might be pushed by accident. So you need to make a valley for it. (They feel nicer, by the way)
So a good industrial design interface is a "sunk" button, not really pushed, but sunk.
A lot of gadgets use that concept. So if your interface wants to look gadget-ish, you have a style to explore.
It is also interesting that the way you could potentially mark that button as active, is not by the shadows, but changing color, probably making it glow... gadget-style again.
One thing that potentially makes the design fail is a reference.
If the button is the only one present, it has no context on how the button looks when not active.
If you see more buttons next to each other and one is active by default, the difference is clear.
An additional comment. As UI designer you need to differentiate the terms pressed, depression, active, default, "up and down". They are all different.
Another really interesting concept is that a pressed button could be the real default "un-active" or "off" state.
If you have some switches, lever-like, on a wall (imagine an old nuclear reactor panel) the off status would be the lever down.
A series of buttons could be also down as default position. So you only activate some of them to be on the up position.
Going a bit deep on a real case scenario, this could have some safety reasons, For example, the safest status could be the off switch, if something hit the panel, the helmet of a worker, it is safer than the object falling turn off a switch instead of turning on the self destruct one (something similar to the gadget reason)