Something similar can be achieved entirely within Photoshop.

Create a rounded rectangle and add a stroke effect, and choose a pattern overlay. Although this will only work for small stroke sizes of around 2px.

[![enter image description here][1]][1]

You could also use an outer glow effect, and apply some noise to get a thicker irregular line. You could even combine both stroke and outer glow effects.

Screenshot showing an outer glow effect

[![enter image description here][2]][2]


  [1]: https://i.sstatic.net/o6JDa.png
  [2]: https://i.sstatic.net/1gCno.png