I noticed that the GIMP layer mode "subtract" only subtracts part of the value in the MASK layer. Subtracting 10 from 16 results in 7. Subtracting 10 from 32 yields 28. Subtracting 10 from 64 yields 62, etc. So the higher the value in the IMAGE layer, the less is subtracted. Not only is GIMP not really subtracting absolute values, it seems to subtract inverse proportional values. Am I doing something wrong here? I use GIMP version 2.10.18.
This is because values 0..255 are gamma-corrected. Under the hood Gimp converts them to linear values, does the subtraction, and gamma-corrects them again.
In this image, the top layer is a linear black to white gradient, with RGB(127,127,127) in the middle of the image. This top laye is in
Subtract mode This works as you describe, the result of the subtraction is not RGB(128,128,128).
You can ask Gimp to work directly on the values without gamma-correction by switching to "legacy" blend modes (small selector at the end of the mode selector). In legacy mode (Wilber's head) you get what you expect: