That's hard to see what exactly is against your morality in this case.
How bad it is? The "evils" of some are the "gods" of others...
No one can say they'll never do any design that is helping a bad cause, otherwise that would imply they fully researched already every single topic and companies they've worked for. Which is usually not the case.
Same goes with industrial farming, religion, sugar, fast food, politic, alternative medicine, porn, new age, etc. All the negative/positive views on these things could be considered opinions; and for others, these views are based on rational thinking. That's where you need to decide if your views are opinion based OR based on facts. For my part, as an example, I think sugar is a poison yet I don't care doing candy ads; I'm not the mom feeding this to her kids. She's the one doing the moral choice there and not informing herself; my choice is to stay out of this only if I want to feel like I didn't gain any benefit from others' people lack of knowledge about nutrition but I also live with my time, and my time is full of sugar :) To me that's a "lesser evil" if I do design for that kind of thing that goes against my principles.
My take on this is: How bad is it and how much do you get to sleep better at night? Is it abusive towards innocent people like kids, victims or old people? Because no, you won't save the world by your choice; another great designer will simply do the work for you. If you enjoy the payment and if the product is based on a personal choice of someone else, then simply ask for a payment that will make your decision worth it at least (unless you can simply refuse the project because you really don't need that money.)
People who get into "bad stuff" don't do it because of a nice flyer's layout but because of the "prophet" who did intense neuro-programming on them. Your layout could be made in Microsoft Word, these people will still buy the product/ideology or believe the guy selling it to them. Some people just want to believe in miracles; if you want to save them, find another job, go study neuro-science or go at a law school!
As someone else said in the other answers, maybe what you'll get from this will in fact be used for a better cause. If the project you'll work on makes you sick and mad, and doesn't even pay well for that discomfort, then don't do it. The reason why money is part of this choice is because you need to keep your creativity up, and that's very hard if you don't feel good with yourself or get angry because you need to work on things you despise. If the projects makes you mad and slows down your other "good" projects than don't do it!
If it's really not ethical or is even illegal, you also need to report it to the authorities.
Already, if you wonder if you should do it or not and spend time
questioning yourself about the ethic... you got a big clue of the
decision you should take!