With a rectangle shape I'd like to remove/hide one side of the rectangle so it looks like one side is open. I've seen online where the focus is on removing/cutting/deleting the stroke but it leaves me to think that then I can't fill the shape as it will have open paths. Is there a way to do this?
You can select a segment with the Direct Selection tool A, then hit Delete. Open paths can still have a fill attribute.
For a circle or a more complex shape, another approach would be needed if you want to retain a fill in the shape of the original.
Create a circle with a fill and no stroke
Copy and Paste in Place
Set a stroke, and no fill
Delete one segment of the circle