At the risk of being frowned upon for not answering your question, I would say this: don't worry about file size. Instead, worry about load time. They're certainly directly correlated, but the difference is that load time is universally applicable. A 2MB file might load instantly on a powerful hosting site such as imgur but not on an inexpensive shared hosting service.
To determine your answer, you can run load time benchmarks before you begin designing. Upload a simple page with a few images of varying file sizes to the future site host and record the network performance. You can do this yourself using browser developer tools, or you could use an external service.
Try pushing the limit to see what you can get away with. Keep in mind that downloading a 5MB file is not the same as downloading 10 512KB files because of the overhead involved. If the load time for the entire page is taking a few seconds, it's probably a good idea to reconsider an image-heavy site design.
Your question focuses on images, but there is so much more to optimize beyond that. Google has a great resource available for this kind of thing: Web Performance Best Practices. Not all of the advice there will necessarily apply to your situation. Depending on your expected traffic some of the optimizations might be lead to insignificant performance improvements, but it's a great guide to know what to look out for.
