As the title says: How can I create a custom brush that allows me to use "hardness" settings? As far as I've come searching the net, I haven't found any tutorial or how-to.
The closest you can get is to use a soft-edged image to create the brush. As you've seen, they are basically vector shapes, but a custom brush retains opacity information as a function of the saturation of the original pixel image used to create it. If you start with a soft-edge shape, you'll have a soft-edge custom brush.
I dont know whether it is useful or not for you but its response to your comment.
You can do a little softening of the edge of a custom brush if you take a regular Soft round brush and then go to brushes window (F5) and set
Dual brush and there select your custom brush. ( ..and possibly adjust some settings there.. )
At best it's sort of like the effect you're looking for.
I think the easiest way to look at this is that custom brushes are much like raster images.
So, when you make a brush, you save the image in the state that it is in.
The bottomline is however that photoshop doesn't have a brush setting that would blur the edge of a custom brush.
I'd like to add that best way to getting around this issue is to make both soft edges version and hard edges version of the brush and of course as big as it needs to be because its better to resize down that up ( much like with your regular raster images )
Having both versions will more or less give you the same exact result as if you were using hardness except theres only those hardness levels that you are willing to make...