When I choose a font size for use in a graphic I usually opt for font size menu dropdown defaults like 6, 8, 9, 10, 11, 12, 14, 18, 24, 30, 36, 48, 60, or 72. If I'm working on something large (like an A3 poster) I need much larger fonts. I'm always tempted to keep these as powers of 2 (e.g. 128, 256, or 512). But why? Am I just being superstitious! Does font size have any technical implications? Is choosing 512 in any sense better than 513 or am I actually free to choose whatever looks best?
In the analog days, typefaces came in specific sizes for the simple reason that manufacturing of type in metal and wood required it.
In the digital age, you can pick any sizes you want. You can set your fonts to 512.34492 points if you'd like.
The size is really a visual design issue. Use the size appropriate for the design. The actual numeric size is irrelevant.
Short answer: you're just being superstitious.
The reasons for old type being available only in set sizes is historical. Modern fonts can reproduce typefaces in any size you like. You can even have 15.65 point fonts if you like.
The main thing is to be consistent with your own font sizes. Don't randomly change size, and don't change size for no reason.
Another point to consider is that it can be aesthetically pleasing to adhere to a baseline grid, and doing so many affect the font sizes you choose and guide you in the direction of at least making your large heading type a multiple or near-multiple of your body type. It makes sense to combine, for example, 12, 24, 36 and 48pt fonts because they are all multiples of 12 (assuming 12pt is your body type size). But there is nothing conceptually wrong with choosing, say, 13pt or 11.2pt as your body text size and sizing your headings based on those instead. Also, font size doesn't necessarily have to remain in a fixed relationship with baseline-to-baseline spacing - with increased line spacing (leading), you can use a smaller font in a larger space.
The benefit of that particular set of numbers, and so presumably why those particular numbers became the standard for physical type and persevered as the standard even when it was no longer needed, is that mathematically they share lots of common factors. Choose numbers from this set, and there will be numbers they are all divisible by.
DA01 is right that what looks right and what works on the page should be what drives font size choices, but this can be handy convenience if you are setting up a baseline grid on a complex layout in something like InDesign with various sizes, columns and side bars or blocks. For example, with two levels of header at 24 and 18, lead-in paragraph text at 12 and regular paragraph text at 9, those sizes are all divisible by 3. So you could have a baseline grid at 3 points, and still line everything up quite neatly without too much difficulty, despite the many very varying sizes. More on grid sizes set to increments in this question.
That said, this a) applies more to leading (line spacing) than it does to point size, and b) is only a convenience, not an absolute benefit. Even in a case like the example above, don't be restrained by those arbitrary numbers - if you like those relative proportions but find the whole thing looks better scaled down by 6.4% so the main headers are 23.424 points and the baseline grid snaps text every 2.808 points, then do so.
The only other consideration I'm aware of is (and it's a very small one), if you are designing for a screen and find the font looks bad at a particular size in particular software, it's possible (but unlikely) that there's some hinting threshold or similar set in the font or the software that the decimal puts you just on the wrong side of. So, if you find when testing that a font set to 10.2 is rendered badly in some browser or operating system, the first thing to try is to see if it still looks awful at 10 or 11 (or 10.5), before looking for a replacement font. It's more likely that something about the software or operating system just doesn't look good with that font around that size, but it's worth a try.
So basically, choose whatever size looks best, but remember that a useful set of numbers with lots of useful common factors exists which might be handy if you're struggling to get a grid to work for a complex layout, and that these are the points that the pixel rendering of fonts are most likely to be set to revolve around, which very rarely might be help avoid bad rendering.