I want to convert an image to this style using Photoshop:
How do I add the border made of clouds? Does this style have a name?
The effect is called "a vignette." This can be created by using a layer mask where the mask knocks out parts of the layer and allows what is beneath it to be seen.
The clouds themselves are probably a brush that looks like clouds, but it can be a black and white image of clouds to make the mask, and an oval gradient to soften the effect.
If I were to do this, I might make a layer filled with white, create a layer mask with an oval gradient to describe the basic vignette, and then use a photo of clouds to roughen the edges of the oval.
I usualy do this with the brush tool, you might need to download a good 'cloud' brush package. I don't know any other way.