It sounds like your document is in CMYK mode. To change it to RGB, choose File → Document Color Mode → RGB Color
.

Are you doing print work? Or designing for the web? If you're designing for the web, you'll want to have Illustrator and Photoshop set up for web and UI design work — The correct way to set up Illustrator and Photoshop for web and UI design is with all RGB color management disabled (at the very, very least, ensure Convert to sRGB
is dialled in Save For Web for both apps).
If you're doing RGB photo retouching, then a different strategy is required.
If you're doing print work, then you'll want to use CMYK color mode and may need to talk to your printer about the correct color profile to use, and may also want to calibrate your display.