I had a micro version of this issue throughout this year when making custom wedding stationary. Brides and grooms change their mind a lot. This leads to very many samples.
Do you keep physical samples? If so, how many samples of each project do you keep?
Yes. While the client's account is active I kept every example. I sent/showed them a copy of each, had a physical copy in their file and had a digital copy (obviously the psds etc.) but also a picture of the printed out sample, front, back, inside, outside. Within that business client's would often come back for matching stationary such as thank you cards months later. I tended to keep a copy of course of the final design that they had chosen but also a physical copy of any samples that had major changes.
For clients from over a year and a half ago, I would keep the final physical samples and only earlier samples that I felt were worthy of the range that we were showing.
How long do you keep them? Do you keep all project samples or merely the ones you feel are "best"?
Final products, forever. Almost. If they are considered good enough to make the range they will be a part of it until something 'better' comes along. We would also keep a physical final product in the clients file. The samples unless deemed worthy or useful in another way would be trashed.
How do you store physical samples? Are you concerned about degradation over time? If so, how do you combat that?
Because of the nature of what I was doing, we just kept the stationary in plastic pockets within a file. It does not need to last for generations to come, we keep the digital copy of anything needed to recreate that piece if needed. However if degradation over time is something that you would be worried about there are several means of storage that you could consider, and choose which is most appropriate for your pieces.
I had considered a good photo shoot then maintaining the photos rather than the pieces. Is it imperative to have the actual physical piece so it can be held and felt?
This depends on what it is. A graphic and some text on a poster for example, going onto white 150gsm non textured paper, you will not need to have a physical example of. But a curious collection virtual pearl, 300gsm, possibly textured invitation that folds at each corner, has a pearl sitting on some ribbon to seal it and vomits confetti in a fantastical romantic experience will need to be kept in physical form for a client to understand that it is so much 'more' than what is on screen. People who are not familiar with design have a particularly hard time visualizing what the final product might be. If it is for your own reference, then keeping a photographic log with comprehensive notes and the appropriate files to recreate the piece is no problem but if you want a client to make a decision, a final sample of what you are suggesting is always best.
If you haven't already, I would strongly suggest a photo shoot and some serious notes on each piece. Even you are impartial to the design it is worth the hard drive to have a copy of every major change to avoid issues either of damage to the piece or a client who has decided they picked ocean blue, not sky blue.
These are my thoughts, I hope they prove helpful. Regardless though, photograph everything - starting with active clients as it is most important to have several means of reference and recreation for their things. As regards your own personal projects and how you deemfit to store your samples that you feel are worthy, it is entirely up to you :)