I think there's two popular ways to approach this: device-dependent breakpoints and design-dependent ones. While I do try to build my CSS from mobile on up towards large screens, I don't pre-define breakpoints.
I start with as many intrinsically responsive elements as possible anyway, making use of the inherent property of block-level elements to try to fit in their containers and using min-width and max-widths and percentages (and recently viewport units) on top of that.
Only then will I start resizing the browser from small to large, and as soon as I hit a point where an element starts to look like it shouldn't, I introduce a breakpoint. I might group a few if they're within like 20 pixels of each other, but after that I'll happily create a new breakpoint.
Most projects I work on end up with 3-6 breakpoints, each with a small handful of selectors. Sometimes even just one selector in a breakpoint to adjust that one element after a certain width. I never predefine them though.
In short: I let the design decide the breakpoints, not the device. Each design is different.
For further reading I recommend this blog post.