This is due to anti-aliasing on non-perpendicular edges. Anti-aliasing is the processes designed to make odd edges appear a bit smoother without being "stair-stepped" and jagged.
In order to create an angle or curve along a pixel grid, Photoshop must choose which pixels to grab and which to ignore. In some cases, and based upon selection settings, Photoshop may grab a pixel at a lower opacity in order to create the visual appearance of a soft or smooth edge.
The thin white line you are seeing is a result of having a white background behind the anti-aliased edges. If you were to paint a dark color behind the pasted layer, or change the documents background color to a dark color, you would not notice the white edge as much if at all.
Another way to remove that white edge would be to duplicate the pasted layer several times. This will stack the partially transparent pixels until they appear solid. However, this will make those odd-angle edges appear more jagged.
Using layer masks rather than cutting and pasting would also assist in lessening the anti-alias edges.
In the end, it often requires some care to create edges which blend smoothly. High quality work is rarely as simply as copy and paste. It's generally copy, paste, blend a bit.