Here are two techniques you can use in Photoshop, commonly used for exactly this kind of situation. They give different results, so pick the one that achieves what you're looking for.
A basic thing to keep in mind is that marker strokes (including the texture of the paper than is usually visible) are really just a form of noise. In this type of situation, though, the "noise" pixel areas are usually beyond the scope of
Filter > Noise > Reduce Noise or
Filter > Noise > Dust and Scratches.
Here is a detail, zoomed to 250%, from a scan I received from a client. (The artist did a terrible job of the scan using some kind of homeowner-grade Mac imaging software, but the marker strokes, which is what we're interested in, are clearly visible.)
Filter > Noise > Median merges the colors in adjacent pixels. Isolate the drawing from the background first and keep it selected so that the filter doesn't blur the edges. Adjust the radius to get the effect you're looking for. It will be small -- 2-5 pixels, depending on the size of the image, is usually plenty. You can apply this more than once to build the effect gradually.
Filter > Blur > Surface Blur is often more useful. It blurs regions where the brightness differences among adjacent pixels are below a threshold setting. (No need to make a selection, because it doesn't affect hard edges.) It's a matter of playing with the values until you see the effect you want. The higher the threshold value, the more pixels will be affected. Again, you can apply this one more than once. Sometimes applying the Median filter first, then Surface Blur, gives the best result.
Both of these can be applied to the composite RGB image or to individual channels. In some cases, changing the mode to Lab (Image > Mode > Lab), applying Surface Blur only to the Lightness (L) channel, then switching back to RGB and applying again to the RGB image will give you "better" (depending on the effect you want) results.
Sometimes one of these will get you all the way. Usually you'll still have to do a little bit of touch-up with the Spot Healing Brush, Patch Tool or Clone Stamp. Experiment a bit, and you'll soon get the hang of which one, or which combination, works best in different situations.