Take the 2-minute tour ×
Graphic Design Stack Exchange is a question and answer site for Graphic Design professionals, students, and enthusiasts. It's 100% free, no registration required.

I have two opaque Smart Objects. Object A is black-and-white and I want to use it as a transparency mask for object B.

In other words, for the following layers:

example

How can I get a Red X on a transparent background without rasterizing object A? Please note, that layer B is a smart object and can be fancy (i.e. not just red). I want to be able to edit both of these smart objects and see the effects instantly.

result

share|improve this question

2 Answers 2

up vote 1 down vote accepted

This is not possible.

Smart objects can only be used as clipping masks (Option/Alt-clicking between layers). Smart objects can not be used as any other form of mask. Masks must consist of pixels or paths and nothing more complex than that.

However.. if some raster content is okay.....

Assumptions:

  • The X layer Smart Object can not be edited in any way.
  • raster masks are okay
  • Image is RGB

    1. Move the red layer above the X.
    2. Set blend mode of red layer to "Lighten"
    3. Highlight the X layer
    4. Layer > Adjustment Layer > Invert..
    5. Highlight the X layer and the Adjustment layer
    6. Right/control-click on highlighted layers > Convert to Smart Object
    7. Option/alt click between the two layers you now have.
    8. Highlight the X layer
    9. Command/Ctrl-click the Blue Channel
    10. Layer > Layer Mask > Hide Selection

Final image:
Final

Zoomed to show edges:
zoom

Steps 3 through 6 could be replaced by simply editing the X smart object so it's inverted.

Conversely, you could simply highlight the X layer, Command/Ctrl-click the Blue Channel. Inverse the selection and apply a mask to the red layer. Then turn off the X Smart Object visibility. This provides a much easier file to work with.

I'm not clear as to why you need to use the smart object to mask anything. If you need a red X why not simply create a red x?

share|improve this answer
    
Red X is just an example. Both of my layers are complex and both are to be editable. Your answer would work only for that particular example. Moreover, it implicitly uses rasterization (through ctrl-clicking channels) which I wanted to avoid. –  wrygiel Jun 4 '12 at 23:17
    
BTW, if I wanted to use rasterization, then there is a better way to go: copy the X layer, switch to quick mask mode, paste, switch back, apply current selection as a mask to the Red layer. I know this trick, but I am looking for something better. –  wrygiel Jun 4 '12 at 23:25
1  
Then I'm going to have to state what you want to do is not possible. Smart Objects can not be used as masks, other than clipping masks. And with clipping maskes, there is no way to designate a transparent color. Loading channel as a mask is raster.. but it is in NO WAY rasterizing any of the smart objects. –  Scott Jun 4 '12 at 23:32
    
Okay, my +1. Are you sure about that? Just clipping masks? –  wrygiel Jun 4 '12 at 23:33
    
I'm sure.. Smart Objects can only be used as clipping masks and not any other form of mask. Masks can only consists of pixels or paths... nothing more complex than that. –  Scott Jun 4 '12 at 23:34

As Scott stated, this seems to be impossible (or, was, in CS5). The fastest way I know (which involves rasterization though) is:

  • Select Layer A (the one with an X).
  • Ctrl+A, Ctrl+C, Q, Ctrl+V, Q (copy the contents of Layer A, switch to quick mask mode, paste, switch back).
  • Apply current selection as a mask to Layer B and hide Layer A.

These steps have to be repeated every time Layer A is changed. Compared to Scott's answer, this will work with any kind of layers (not just plain red).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.