I agree with others that vector drawing quickly gets too limited in Photoshop and that Illustrator is a better alternative.
But I can't reproduce the problem. In Photoshop 2020 I can draw Bezier curves which are not closed. And if they are closed I can select an edge with Direct Selection Tool and delete it.
Edit: Perhaps your problem lies in drawing a Path instead of a Shape. While having the Pen Tool selected, in the top tool bar try selecting Shape instead of Path.