Assuming that there is a background, you should be able to select and remove it. you can achieve that with many different techniques, one problem with SVGs is that some times they have many complicated layers and groups.
First try an see if you can select it using Direct Selection tool, if it's in a group then double clicking on the object would isolate it, making it possible for you to select it individually.
I don't understand why magic wand tool didn't work for you, if it is the only white color object it should have worked...
Alternatively you can try and find that object from your objects, look in the layer window.
Since it's a royalty free SVG, it would be helpful if you give us a link.