Graphic Design Stack Exchange is a question and answer site for Graphic Design professionals, students, and enthusiasts. It only takes a minute to sign up.
Sign up to join this community
There used to be a way to do masking tricks to create rounded polygon corners in Photoshop, but I can't seem to find anything for accomplishing the same trick in Photoshop CS6. Does anyone out there have a solution?
Help me Obi-Wan Kenobi, you're my only hope.
It can also be done with a layer mask over the vector shape.
Create your polygon, then load the layers selection by Command-Clicking (Ctrl-click on pc) on the vector mask (the thumbnail of the pentagon in the layers palette below).
Then go to Select>Modify>Smooth. Enter the smooth amount.
After that it's just a matter of creating a layer mask based on the now rounded selection.
Now you have your rounded polygon fairly quickly and easily. You could also manually edit the shape with by adding points and using the convert point tool, but that would involve a great deal more work.
It's a trick, no more, but it gets the job done: give your polygon a stroke layer effect and align the stroke outside. With thicker strokes, the effect is rounded. No control over the roundness, though...
there is an option for polygon (radius) click on the small arrow behind the shape button (: 
