As Scott mentioned, you could use the pen tool & convert the path to a selection. Or make a vector mask.
These are effective and quick. There is also the option of using the polygonal lasso tool (L) to select the 2,1 and 5 and the ellipse tool (holding SHIFT to make a circular selection) for the 0.
Each time you have a selection of a number I suggest right-clicking it and layering via copy. That way you will wind up with a layer of each number as opposed to trying to clean out the black and not having a layer of your source image.
Also it sounds like the issue you are having at the moment is because either
a) You are using either the quick selection tool or the magic wand and have not adjusted the tolerance appropriately.
b) you are selecting by colour and therefor all of the black tones in the image are selected as opposed to just the background.
As there are many options to achieve your selection here you may not need to worry about tolerance for this project in particular however I strongly suggest that you consider toggling tolerance in the future to get a feel for it. Tolerance is discussed in the links above for the quick selection and magic wand tools.