As far as I know you can't change the default behaviour of the brush size scrubber.
What you can do is use your keyboard up/down keys to change the size in single increments and shift+up/down to change the size in increments of 10. Dragging the scrubber automatically selects the value so theres no need to select the input box again, just drag the scrubber then refine with your keyboard.
Once you get close to the max 300 size with the scrubber it should still be easy enough to get within 5-10 of where you want anyway, so dragging the slider relatively close to the value you want then using your keyboard to refine the value should be easy enough.
Without using the scrubber you can also ctrl + alt + Click/drag (OS X) or Alt + Right click/drag (Win) to change brush size, which may be easier, although I think the behaviour is the same. Left and right is for size, up and down is for hardness.