When you place or draw a vector object in Photoshop, it's edges are anti-aliased to the underlying artwork (layers). This means, Photoshop interpolates pixels to create a smooth edge appearance. And Photoshop interpolates at the resolution of the document (PPI).
You monitor, any monitor, needs to use pixels to display anything. Most monitors can't display vector content as vector. Monitors display everything as pixels. The greater the pixel density of your monitor, the less noticeable these pixel may be. i.e. a 4k or "retina' monitor will show the actual pixels to a lesser degree, but they are still ever-present. So, in addition to Photoshop anti-aliasing the edges for its use, your monitor also introduces some pixels to display anything.
Zooming in on any vector object will always show pixel edges, especially in a native raster image editor such as Photoshop. If you were to zoom in using a vector editing application, the edges are anti-aliased again based upon your monitor pixel density, whereas in Photoshop anti-aliasing remains set at the document PPI. So in Photoshop.. the greater the zoom, the more pixel edges you'll see.
The important factor when considering vector smoothness is that it's smooth upon output not always upon display. When a vector object is output it is anti-aliased to the highest possible setting of the output device. Therefore, even if a vector object appears to have pixel edges in Photoshop, when you output the document, the vector edges are anti-aliased again for the specific output settings or device. This often smooths them compared to onscreen display.
In short, seeing pixel edges on vector content when zoomed in is perfectly normal in Photoshop.