This is super frustrating feature that I can't overcome. I often want to align one object to another, without moving the one I'm aligning to. I've tried selection, key object but it keeps moving both elements.
Once you've selected the objects you want to align, click on the object you want to align relative to again. You'll notice that object gets a thicker outline—it is now the "Key Object". You'll notice the "Align to" button in the Align panel also changed to a key icon.
You can also just change the "Align To" in the Align panel to "Align to Key Object" and it will automatically set the top object to be the key, although you can click another object in the selection to change.
As long as you see that key icon, whichever object is the key object will not move.
You can read more here: Illustrator Help — Align or distribute relative to a key object