First Baseline Offset
The First Baseline Offset option of the text area does exactly what it says, affects only the first baseline of the text frame. It essentially shifts the entire contents of the text frame by whatever offset you set.
...which aligns correctly, cap height to top of baseline.
That isn't what's happening, it is aligning to the top of the frame. Nothing to do with the baseline.
You can see below, I have a document set up with a baseline grid, a text frame with text of various sizes (without align to grid set). If I set the first baseline offset to Cap Height, it appears as if it is aligned to the baseline grid because the frame is aligned with the grid. If I move the frame, the text is no longer aligned with the baseline grid.
Align to baseline grid
I'd like all of these to align to the baseline the same way
As I said above, the text area first baseline offset option doesn't really align anything to a baseline. If you want to align all your text to a baseline you need to set that in the paragraph style. You do this under
Indents and Spacing -> Align to Grid -> All Lines.
The document I have has a baseline grid set for the whole document but you can set a custom baseline grid for a specific text frame in the text frame options. As long as the paragraph style for the text has Align to Grid set then the text will align to that baseline grid.
Based on your comments that you have multiple text frames set with a Cap Height first line offset. If your first text frame begins with an 11pt font and the second begins with a 9pt font, the offset will be different as the offset is based off the Cap Height of the font on the first line of that text frame, not the first line of the story. If you want the same offset for each frame, set a fixed offset. Since you said everything has the same leading and you want everything aligned to that, set the offset to leading and everything will align. You can see the difference illustrated in the following images:
You said the text becomes misaligned even in the same text frame. If that is the case and everything has the same leading, there must be some other spacing set somewhere, probably paragraph before or after spacing. You need to set these either to 0 or a multiple of your leading. You can see the difference in the following images: