The option to select it is on the same place as the option to replace it.
Go to searchFind and replace. (I agree, It should read search or replace)
Menu > Edit > SearchFind and Replace.
Select search.
Choose Fill as the search method.
Choose the specific color, you can use the eyedropper.
Finish that search, and the floating bar has a three buttons.