Vector Brush Gets Pixelated in Photoshop

Question

I'm trying to create a brush the makes little squares in p-shop. I tried several times to do the whole thing in p-shop but whenever I change the size of my brush after the fact, the edges of my square are all pixelated. I've even tried creating a vector object in illustrator and copy pasting it in p-shop to create my brush. Is there a way to create the kind of brush I'm attempting and still have crisp edges on my squares?

This is the shape I'm trying to create

For further clarity, this is the type of brush effect I'm going for, but as you can see, it's not clean.

Answer 1

Brushes in Photoshop are always raster. There's no such thing as a "vector brush" in Photoshop. And as with all raster content, scaling, especially enlarging, can cause degradation.

If you wish to use a brush and prevent any loss of quality, create the brush at the largest size you need. Reducing the brush size won't show the pixel issues as much. But it will still be raster in nature.

The only real way to get around this is to use Custom Shapes or Shape Layers, not brushes.

Answer 2

I was thinking about this question because I think I've bumped into this before... So, I don't think it's a recent bug. If it is a bug, it has probably always existed.

Either way, the issue seems to be that Angle jitter on a square brush does weird things to anti-aliasing on the outside edges. Occasionally anti-aliasing disappears completely and generally just looks terrible. Solution: Rotate the square 45 degrees before making it into a brush.

Road to discovery

This is a pretty lengthy explaining how I got to my conclusion, which is at the bottom of the answer after "Conclusion"

1. I realized that if I start with a rectangle with straight angles and no anti-aliasing, the brush engine has to figure out the anti-aliasing and it seems like it is pretty much botching it for whatever reason.

2. So I thought: "What if the source image already has anti-aliasing?"

3. I rotated my square 45 degrees before making it a brush and that seems to have given me a pretty good result. The straight angle square looks terrible in comparison (check the image below).

4. I also tested a blurred square with smaller brush size and that seemed to work better than no anti-aliasing, but it still failed quite badly when the brush was scaled to a really small size using Size jitter.

5. After that it occurred to me that in the bottom right result of the last image, the anti-aliasing looks pretty damn good on the inside edges of every single square.

6. When you make a brush, all whitespace gets trimmed out, so similar to my blur attempt, I tried adding some extra whitespace to the outside edge of the source image using a thick outside stroke with 1% opacity. With this method the angle jittered result looks pretty good as well. It wasn't as reliable and more hacky than just rotating the source image.

7. What these tests lead me to believe is that when you make the brush, there is a square(or a rectangle) area surrounding the brush, sort of like a smart object or a canvas and that area is somehow perhaps cropping away part of the outer edge when angle jitter rotates the brush. To be more specific, I believe the graphic inside the square "canvas" is shifting a bit and the edge of the "canvas" is cropping part of it away. Or actually, I'm not sure if it's getting cropped, but it definitely seems like the outer edge is getting degraded somehow when the graphic meets the outer perimeter of the brush.

Conclusion

So it's probably not that much to do about anti-aliasing after all, but rather the edges of the source image are hitting the edges of the brush "canvas" or "outer perimeter", which is chipping away at the edge in some way. That's my suspicion anyway.

It could just be the brush engine can't create anti-aliasing at the edge of the brush "canvas".

So the reason why rotating the square 45 before making it a brush works is because that makes it so only the corners of the brush are hitting the brush "canvas" and the square edges are as far away from the "canvas" edges as possible. So when this suspected shifting happens with Angle jitter, there's nothing noteworthy at the very edges to degrade.

A little diagram on what I think is happening:

This might not be what is actually going on...

Brush tests:

The image is scaled by the browser, so you should open the image in a new tab to see the edge quality better with 100% zoom.

Answer 3

Start with a high res document. 2500 ox square @ 300 dpi. You’ll get a high quality brush from that.