"Transparency Shapes Layer" checkbox in Blending Options

Question

I am creating a photo filter program and trying to do some actions in Photoshop CS5 in order to repeat them in code.

One action I stuck with is Vignette effect. This is typically gradient fill layer with "Vivid Light" blending mode. However, it is looking nice only when "Transparency Shapes Layer" checkbox is turned off.

What I see in Photoshop manual:

"Select Transparency Shapes Layers to restrict layer effects and knockouts to opaque areas of the layer. Deselecting this option, which is always selected by default, applies these effects throughout the layer."

However, it does not looks true because when checking / unchecking this checkbox, the image becomes completely different.

Just wanted to know if someone knows any insights on how this checkbox works.

Thanks,

Andrey.

Answer 1

"Transparency Shapes Layer" is one of a group of checkboxes that modifies how layer effects affect the layer they are applied to and layers below. It is not something you would use unless you needed to restrict an applied layer effect to the non-transparent parts of the layer. You're using it for something it is not intended for, so the result is a kind of super-black overlay effect.

If you're using a layer with gradients to create the vignette, use one of the Multiply/Color Burn/Linear Burn mode group for better control.

Alternatively, if you want for that heavily-saturated color look, try using a neutral gray layer in Overlay mode and adding your vignetting gradient to that using a black brush, black-to-transparent gradients or similar.