I was following along a tutorial for painting and for some reason, when I subtracted the selection from a layer, the paint goes beyond it. I have used the default shortcuts such as control clicking the layer that I wanted to subtract from and ctrl alt to subtract from that layer. Here's a gif explaining my problem.
I tried doing another test and I found out that if the pixels are darker at the edge, then it's fine.– ArchnilaSep 2, 2018 at 13:40
You made fuzzy selecion. The fuzzy layer was subtracted from the selection only as strongly as the color in the layer had opacity.
But the question was useful. For ex. I had never noticed that this makes fuzzy selections, I have used it only with solid opaque colors with sharp edges.
If you use the fuzzy selection to generate a layer mask to a new layer, you get something like this:
Of course you can make it more steep with curves tool or tresholding:
Here some green strokes are masked with the new steeper mask, but I bet manually painted mask or eraser is better.
You can use image adjustments on a quick mask too. No one ever gives quick mask the love it deserves :'( — I wouldn't recommend the levels or curves adjustment or any other, because it affects the selection as a whole, potentially screwing up the edges in this scenario. Would make more sense to just subtract with 100% opacity in the first place. Sep 1, 2018 at 17:03
@Joonas people who use selections sometimes like to tweak it. Painting into quick mask would be the easiest way to do the tweak, but there's something between the ears that prevents one to take a brush when he has started to make a selection.– user82991Sep 2, 2018 at 10:19
Yea, I think in general people kinda steer away from it. You know, I always thought it interesting that PS Elements has a selection brush. I think it may be there to kinda lower the bar... Quick Mask has a little more going on. — For anyone who doesn't know, the selection brush does the same thing as brushing in the
QM, but you work in the marching ants mode. Plus like I mentioned
QMlets you do more. Anything you can do in a layer mask, you can do there. For example, you can free transform a selection if you enter
QMmode. Or add filters to the selection. Sep 2, 2018 at 11:13
Theoretically, the paint shouldn't go beyond the area that was subtracted in the gif since the "ao" layer was "deselected" in the new layer, so when I apply paint in the new layer, it shouldn't go beyond the deselected "ao" layer. Sep 2, 2018 at 13:37
@user287001 I tried my method again, but with slightly opaque pixels and it seems to be better. I think it has got to do with the opacity of the pixels. Sep 2, 2018 at 13:46
It's the way you make the selection.
You got a layer with 100% opacity (
A). Then you make another with like 30% opacity (
B). You make a selection of A and subtract that with B and what do you get? The subtracted part of the selection will be at 70% opacity.
So anything you brush in that area will be at most 70% visible.
A quick way to see what the selection looks like exactly is to toggle on Quickmask (
q). Double click the Quickmask icon in the toolbar to edit its settings.
A quick approximation of how your selection should look like with the quick mask enabled.
That makes sense. It seems to be fine when both layers are opaque. Eg layer A is an opaque star at 100% and layer B is an opaque circle at 100%, so if I subtract layer B from layer A, layer B is deselected from layer A. All pixels from layer B is deselected from layer A. Sep 2, 2018 at 7:28
@Archnila, I tend to make the selection layer all the way exactly as I need it so I don’t have to take multiple selections. In this case, I might've duplicated layer A and just used eraser to get rid of what I don't need. Or possibly use another layer to make a selection and delete the excess from layer A duplicate. — I do sometimes use Quick mask too. Take selection from A, enable Quick mask and brush away (black or white), disable Quick mask. At that point I could make that into a layer by filling the selection so I can easily use it again or just brush in the new color in the selection. Sep 2, 2018 at 7:49