I have googled this and found a number of answers and videos which all say that this can be done by setting the brush size slider in the brushes panel or brush editor panel.
I'm either clicking on the wrong thing or doing something strange as I don't seem to have the same options available to me. At least setting all the sliders to their minimum value still leaves me with a 3 pixel wide square brush:
I'm using Gimp 2.8.4. I'm obviously making a basic error, can anyone help?