Sorry, it's not quite clear to me what exactly the desired result is. If you just want to make one of the segments a straight line, first make sure 'Show bezier handles of selected nodes' () in the toolbar is enabled, then Ctrl-click the round bezier handles on either side of the segment to remove them.
If you actually want to completely remove the segment, then there is the problem that you can't use the 'Delete segment [...]' button () in the toolbar directly, since Inkscape can't tell which one of the segments you want to remove based on which nodes are selected. Instead, double-click the superfluous segment to temporarily add a new node there, select the two new split segments and delete them with the button above, one after the other.
Alternatively, it might be faster to just create a new path with the bezier tool and don't close it this time by right-clicking or pressing Enter instead of clicking the start of the path.