I've got a simple hair image and want to create a b&w mask for it. I've selected the black color via Select-->Color Range and then inverted selection. Finally I've painted the selection with pure white color. enter image description here

When I put the mask under the color image I noticed, that it's edges are slightly visible. I've tried to duplicate the image and desaturate/bright it to create a mask, but after layering it under the color image the result was the same. enter image description here

How to make a mask, which'll be identical with an image (the mask's edges won't be visible)? It's important to me since I am making the 3d hair texture in 3d software and after applying transparency to it the final result looks really ugly. enter image description here

P.s. Please go easy on me guys, since this is my first question here :).

  • 1
    Possible duplicate of Techniques for cutting out hair accurately - I'm sure you can figure it out from Ryan's answer. It's a complete tutorial on how to cut out hair (even the messy ones).
    – Alin
    Commented Sep 18, 2016 at 19:57

2 Answers 2


I have done some 3D renders and I know the frustration.

The theory behind it

Let us make a zoom on this circle and see what is happening. This is what you want. A red shape transparent on the borders. (If you put this image "A" on diferent background it will blend properly the rest are totally opaque.)

If we see the real image and the mask separately we will notice something. The transparent red pixels are actually red and we have a mask where we expect it to be.

But when we have a pre rendered image, with anti alias on, thoose pixels are no more red, but diferent grades of pink.

Most of the masking techniques rely on reducing the mask, so the transparency is "eating" more where you have not the modified color, so you do not see this halo.

A method of using your exact image as a mask

1) Open your image

2) Go to the channels panel and select the most contrasted one. In this case Red.

3) Duplicate it. Name it for example "MASK". Now select it and turn the eye on.

4) Image > Adjust > Levels (Ctrl+L) and pick the white droper on the correct area, the darkest hair fill. Press Ok.

  • Depening on the image you could need to move the slider a bit more to the left.

Now you have an alpha channel that uses the exact same antialiased the original image used. A beauty.

5) You can load this channel Selection > Load Selection and choose "MASK" Channel.

Let us work with your image

6) Invert the selection, we will work with the black area.

7) Now we need to expand this selection.

  • Selection > Modify > Expand > And use 2 pixels.

From this selection make a new layer (Copy Paste) Name it "Halo1" and you will have a layer like this.

Note that we now have a slight brown halo of two pixels.

8) Duplicate this layer. Move this layer with the keyboard one pixel to the right and one down.

And move the other "Halo" Layer one pixel to the left and one up.

On this both layers use the blending mode "Lighter Color"

Remove any selection and flaten the layers.

You have now a slightly "fatter" image.

Let us compare it versus the original (on the right):

9) Now go back to your Alpha Cannel MASK where you saved the original selection.

  • Selection > Load Selection > And choose your alpha channel.

10) Copy + Paste and now you have a neat image with no halos.


There are still a thin dark part on the tips of the hairs. To remove them try to use the "halo" layers, instead of moving them 1 pixel, move them 2 or 3.

An automatic quicker method (in recent versions of photoshop)

But I have not compared it to the other result.

  • Using your selection "MASK" make a new layer

  • Menu > Layers > Halos > Remove Halo > and try 1 or two pixels.

  • Ok this is what happens you have premultiplied alpha and unmultipled alpha (called straight alpha). Photoshop uses straight alpha IN anycase, there is a function in the menuset to deal with these its Layer -> Matting. Note that straight alpha does not cause a halo in photoshop if you combine it with the alpha but premultipled will. So your description is not really accurate. If your confused about my answer as a question and i will explain.
    – joojaa
    Commented Sep 19, 2016 at 6:53
  • @Rafael +1 for efforts, but your suggestions didn't give me a desired, clean result.
    – Paul Gonet
    Commented Sep 19, 2016 at 16:09
  • @PaulGonet I modified the process for your specific case.
    – Rafael
    Commented Sep 19, 2016 at 19:32
  • @joojaa, I will take a look at the matting and premultiply.
    – Rafael
    Commented Sep 19, 2016 at 19:33
  • @Rafael Thanks for the solution. Works good. The only disadvantage is the jagged image edges, but after bluring the mask it looks decent. I guess to make it look perfect I'd have to adjust the mask by hand.
    – Paul Gonet
    Commented Sep 20, 2016 at 18:50

Here the hair is synthetic. There's no noise, color contamination and radical irregular brightness & contrast variations. It should be far easier than a natural one.

Unfortunately all easy to use automatic (=do not draw it by yourself) masking tools in Photoshop still fail miserably. Long sharp corners and 1 pixel wide low contrast curves simply are not in their zone of comfort.

If the user has substantial underhood knowledge, he still can bypass the troubles and get usable results by using rare functions with surprising parameter combinations. That is not the way for us, the ordinaries. We need something easy. Here is one:

  • select your photo and copy it to the clipboard
  • paste it onto empty Inkscape artboard
  • goto Path > Trace bitmap
  • select mode = brightness cutoff and "smooth"
  • set a low treshold brightness, say 0,01. This is possible due the solid black background and very low noise. Play a little to find a good setting
  • on the Options tab deselect "smooth corners" see other options from the screenshot
  • press OK

enter image description here

Move the original aside. It can be exported as PNG if you want to insert it in Photoshop to a layer mask. For that

  • create a layer mask
  • open the PNG as another photo and copy it into the clipboard
  • open the layer mask by Alt+Click onto the mask icon in the layers panel
  • paste
  • see the result by clicking the photo icon in the layers panel

Here is a high zoom screenshot. There's adde a red underlay to test how accurate the mask is

enter image description here

The black areas are small and a 1 pixel shift brings the red visible. This gives a good reason to assume the mask acceptably good.

The mask in Inkscape is SVG vector drawing. It should be possible to have it as vector mask in Photoshop. I do not know that procedure, usability nor benefits well enough to show an unwavering example.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.