I made a sketch on a piece of paper, photographed it and used the Image Trace tool. Now I have a vector image with a lot of detail.

traced photograph

I'd like to clean it up by removing those white and gray-ish chunks, and keep the dark strokes. Most of the advice I'm finding covers the SelectSameFill color, which selects only identical colours, and therefore doesn't really do it for me.

Do you think there's a way to automatically select all colours darker/lighter than a given threshold?