I was following along a tutorial for painting and for some reason, when I subtracted the selection from a layer, the paint goes beyond it. I have used the default shortcuts such as control clicking the layer that I wanted to subtract from and ctrl alt to subtract from that layer. Here's a gif explaining my problem.
You made fuzzy selecion. The fuzzy layer was subtracted from the selection only as strongly as the color in the layer had opacity.
But the question was useful. For ex. I had never noticed that this makes fuzzy selections, I have used it only with solid opaque colors with sharp edges.
If you use the fuzzy selection to generate a layer mask to a new layer, you get something like this:
Of course you can make it more steep with curves tool or tresholding:
Here some green strokes are masked with the new steeper mask, but I bet manually painted mask or eraser is better.
It's the way you make the selection.
You got a layer with 100% opacity (
A). Then you make another with like 30% opacity (
B). You make a selection of A and subtract that with B and what do you get? The subtracted part of the selection will be at 70% opacity.
So anything you brush in that area will be at most 70% visible.
A quick way to see what the selection looks like exactly is to toggle on Quickmask (
q). Double click the Quickmask icon in the toolbar to edit its settings.
A quick approximation of how your selection should look like with the quick mask enabled.