how to create a UV mapping (?) of a texture onto a product photograph based on a calibration pattern

Question

UPDATE: Here you can download high quality images of calibration patterns as well as them printed on socks if anyone wants to give this a go

Pardon my lack of vocabulary, this is not my area of expertise and why I'm looking for some guidance on how to create this product preview. I will most likely end up hiring someone

I'd like to create a system that can map/apply new designs onto an image of a sock worn by a model, with the proper transformations and deformations in order to visualize said designs.

The mapping below can be created by printing a matrix "calibration" design onto the product. How the actual mapping/deformation is then created in photoshop or other software is what I would like to learn to do. Ideally I'd like to turn it into a process that can be run in a free/open source software that can be run headless in a linux server (especially as a python script) so I can run it as an internal service for a bunch of designers. As an initial proof of concept a repeatable photoshop or other commercial software action/script would be ok, though!

Things I have:

• a calibration (matrix) design with dot patterns
• socks printed with the matrix dot pattern
• pictures of models wearing the socks

Here is an example of the results I'm trying to achieve:

The pattern to be visualized on the socks

The pattern as it would be printed on the socks. Note that it's not just a simple overlay with blending, but actual deformation of the image to take into account the 3D shape of the socks being worn.

The socks are printed on a cylinder, so the mapping should reflect the bending of the socks at a 45 degree angle.

Attached are a sample of white socks to do the overlay over:

Answer 1

To my knowledge there's no way to use UV data in Photoshop and I'm not sure it would help with this example because the deformation is quite strong: it'd be a job for vector displacement filter (which Photoshop also doesn't have). If I had a task like this I'd go 3d and since you have Blender as a tag, here's a little breakdown of how I'd approach this:

1. Since a sock is a simple object and it doesn't require any complex folds, I'd start with a model from Daz3d (free) as a base

   

2. in Blender I'd cut the feet and run it through voxel remesher (to get rid of toes and details) and quad remesher (to get a nice topology)

   

3. then a simple UV-unwrap, eyeballing the distortion

   

4. Here's a result. It took less than an hour from start to finish, if I spent more time I guess it'd be possible to get even better results? To me it looks quite ok even now

   

5. Blender supports python as you probably know already and rendering from CLI, so then you could write a script to reload a texture and simply run something like this:

   blender --background sock.blend --python change_texture.py -- new_texture.png

   And in render settings of Blender it'd be also possible to set a set of render passes to render %) so you could have shadows, diffuse lighting, etc as separate images.

   

Answer 2

I repeat my old comment: As long as your images do not have common simple shapes such as straight lines or circles very loose warping should be plausible enough. But the light and shadows should be right. Could you accept it? If yes you can do it in 2D.

An example

One of your unused socks is clipped and pasted over an image of a foot as a new layer. A mesh warp is done to deform the unused sock to the right form. I admit it's not geometrically a masterpiece. Here's a screenshot under the warping:

Here's the result after warping and fixing a couple of not so good places with the eraser:

It has unfortunately no shading, it's flat. To get some depth I drew a couple of black strokes to a new layer:

After blurring the strokes and deleted the extras (selected the emptiness in the warped layer) I got this shading:

Inserting it and a drop shadow I got the final result:

This is still quite artificial. Much better result can be got by having a photo of a leg with white sock. Blending mode Multiply would give both perfect shading and a good fabric surface texture at the same time.

The warping can be reused for another photo if you use a program which supports it. I used Affinity Photo, which has unfortunately non-reusable warp mesh.

Illustrator's envelope distort with mesh allows changing the photo. Only release the envelope distortion and apply the mesh to another photo which has identical pixel and millimeter dimensions. Reuse the released mesh in Object > Envelope distort > Make with Top Object.

In Photoshop you can try Puppet Warp to a layer which is converted to smart object. Then it's possible to change the content, because Puppet Warp is a smart filter.