The CC0 licence clearly states that any waiver is permanent and irrevocable. Excerpt from CC0 1.0 Universal (emphasis added on relevant text):
To the greatest extent permitted by, but not in contravention of, applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and unconditionally waives, abandons, and surrenders all of Affirmer's Copyright and Related Rights and associated claims and causes of action, whether now known or unknown (including existing as well as future claims and causes of action), in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each member of the public at large and to the detriment of Affirmer's heirs and successors, fully intending that such Waiver shall not be subject to revocation, rescission, cancellation, termination, or any other legal or equitable action to disrupt the quiet enjoyment of the Work by the public as contemplated by Affirmer's express Statement of Purpose.
Wether there is any legal condition that would allow you to go against this is something that would be better asked on law.stackexchange.
It's worth noting that it's possible for someone to "restrict free downloads" without going against the CC0 license. There is nothing stopping anyone from commercially selling those images, restricting downloads from their website, etc.