Look, you're a designer just starting out, and it sounds like you've shown enough talent to be noticed by a reputable client. You're off to a good start.
Let's look at your career path from a long term perspective.
As a free-lance designer, you're going to start out not making very much cash, and working hard to advance to bigger and better (and higher paying) projects.
The source of your business comes from your ability to show an impressive portfolio, and that impression will be increased exponentially by the reputation of those who the work was done for.
You want to advance to bigger clients, who will provide you with a steady flow of well-paid work.
So the answer to your question is clear:
Your request was not unfair, in fact it was the only proper move for you to make. Without portfolio rights to work completed for this client, you will not be advancing in your design career as you should.
You should not back down or anything of the sort. Chances are, your client doesn't care one way or the other. They want to make money, and you simply need to convince them that you can help them do that, without any issue or complication.
Have a lawyer draft an edited version of the contract (don't expect the client to do it), provide the client with a formal notice of the changes, and have your lawyer forward the signed, altered document to your client.