I was thinking about this question because I think I've bumped into this before... So, I don't think it's a recent bug. If it is a bug, it'sit has probably always existed... I think.
Either way, the issue seems to be that Angle jitter
on a square brush does weird things to anti-aliasing on the outside edges. Occasionally anti-aliasing disappears completely and generally just looks terrible.
I had a moment of clarity and I realized that if I start with a rectangle with straight angles, the brush engine (or whatever) has to figure out the anti-aliasing and it seems like it is pretty much botching it. So I got an idea... what if you start with anti-aliasing?
I rotated my square 45 degrees before making it a brush and that seems to have given me a pretty good result. The straight angle square looks terrible in comparison.Road to discovery
I also testedThis is a blurred square with smaller brush size and that seemedpretty lengthy explaining how I got to work better than no anti-aliasingmy conclusion, but it still failed quite badly in smaller sizes.which is at the bottom of the answer after "Conclusion"
I realized that if I start with a rectangle with straight angles and no anti-aliasing, the brush engine has to figure out the anti-aliasing and it seems like it is pretty much botching it for whatever reason.
So I thought: "What if the source image already has anti-aliasing?"
I rotated my square 45 degrees before making it a brush and that seems to have given me a pretty good result. The straight angle square looks terrible in comparison (check the image below).
I also tested a blurred square with smaller brush size and that seemed to work better than no anti-aliasing, but it still failed quite badly when the brush was scaled to a really small size using Size jitter
.
After that it occurred to me that in the bottom right result of the last image, the anti-aliasing looks pretty damn good on the inside edges of every single square.
When you make a brush, all whitespace gets trimmed out, so similar to my blur attempt, I tried adding some extra whitespace to the outside edge of the source image using a thick outside stroke with 1% opacity. With this method the angle jittered result looks pretty good as well. It wasn't as reliable and more hacky than just rotating the source image.
What these tests lead me to believe is that when you make the brush, there is a square(or a rectangle) area surrounding the brush, sort of like a smart object or a canvas and that area is somehow perhaps cropping away part of the outer edge when angle jitter
rotates the brush. To be more specific, I believe the graphic inside the square "canvas" is shifting a bit and the edge of the "canvas" is cropping part of it away. Or actually, I'm not sure if it's getting cropped, but it definitely seems like the outer edge is getting degraded somehow when the graphic meets the outer perimeter of the brush.
After that it occurred to me that in the bottom right result, the anti-aliasing looks pretty damn good on the inside in every single squareConclusion.
When you make a brush,So it's probably not that much to do about anti-aliasing after all whitespace gets trimmed out, so when I added some padding tobut rather the outsideedges of the source image using a thick outside stroke with 1% opacityare hitting the edges of the brush "canvas" or "outer perimeter", which is chipping away at the angle jittered result also looks pretty goodedge in some way. That's my suspicion anyway.
What that leads me to believeIt could just be the brush engine can't create anti-aliasing at the edge of the brush "canvas".
So the reason why rotating the square 45 before making it a brush works is because that when you makemakes it so only the corners of the brush, there is a square area surrounding are hitting the brush "canvas" and that area is somehow croppingthe square edges are as far away part offrom the outer edge,"canvas" edges as possible. So when this suspected shifting happens with angleAngle jitter
rotates the brush. I'm not sure if cropping is actually what is happening, but it definitely seems likethere's nothing noteworthy at the outer edge is getting chipped offvery edges to degrade.
So it's probably not that much to do about anti-aliasing after all, but rather when the square is rotated 45 degrees before making it a brush, the sides are as far away from the sides and nothing gets cropped when the brushA little diagram on what I think is rotated using angle jitter
.happening:
This might not be what is actually going on...
With 100% zoom:
Brush tests:
The image is scaled by the browser, so you should open the image in a new tab to see the edge quality better with 100% zoom.