However, as you can see, it is leaving gaps between shapes, thus totally undoing the whole point of the tool, if I have to manually line them up anyway. How do I fix this?
I've a totally different response here: your snap settings are basically Layout snap-settings - for organising a whole composition, spread or document; they are unequivocally NOT drawing snap settings at all!
Note that you are snapping to object bounding boxes, but not to gaps or sizes, and most crucially you don't have snap to Shape Key Points or Object Geometry enabled - this means your snapping candidates were the spread, the various object bounding boxes and no actual geometry or node snaps at all.
I'm quite impressed you came as close as you did with your snapping set like this.
Hope this helps.
This is said already in the comments, but here's some images:
In the left the red shape refuses to sit perfectly when I move it with the normal selection tool, no matter do I have snapping ON or OFF, only one corner can be in the right place at a time.
In the middle I succeeded to place with the node tool three of the four corners of the red shape when I put snap=ON. The 4th corner was a problem because there's nothing to snap.
In the right a horizontal and a vertical help line were drawn. The problematic corner snapped easily to the crossing with the node tool
Not asked: Exact seam isn't always good, because it's partially transparent due anti-aliasing. You can see the background as faint stripes through them. Overlapping prevents it.