Yes, it's possible.
Method
Enter Object > Captions > Caption Setup and set it up as you please. Optionally enable Group Caption with Image.
Place the image somewhere in your document, select it and use Object > Captions > Generate Live Caption to generate the caption.
If you have enabled Group Caption with Image the image and text frame is now a group. Otherwise group them manually.
Simply cut the group and paste it inside your main text frame.
The Live Captions will update even when nested inside a group which is anchored inside a text frame.
Tips
I wouldn't use line breaks to add space around the images. Instead make a Paragraph Style for the paragraph containing the anchored object and add Space Before and Space After.
To make things more dynamic and enable you to make changes later in the process, also make three Object Styles: One for the image, one for the text frame with the caption and one for the group. Apply them after you generate the Live Caption.
That way you can later change the dimensions of the images, add a frame, change the distance from the image to the caption etc.
Major Drawback
Live Captions have one major drawback though (which is one of the reasons I never use them myself): The caption text must be able to fit within the width of the image. Multiple lines are not supported! If the text is too long, the letters will simply be crammed together.
Adobe have added this issue as a feature request in 2017 (I would call it a bug rather than a missing feature). Here, five years later, they still haven't fixed it.
Only way to work around it is to convert the Live Captions to Static Captions, but it's kind of against the whole idea of working dynamically.
If you do your whole layout with Live Captions, set the text frames with the captions to auto-size and then convert all the captions to Static Captions, all the frames will suddenly increase in height and ruin the layout.