3

I am looking for something that is good at recognizing faces and create a 3D model from a 2D photo, especially when the face is slightly turned, like those portraits which have the subject looking to the side, but not fully in profile.

I am not looking at photogrammetry software, takes multiple shots of the object and creates a model from it, but software with some intelligence that can create a good model from a single snapshot taken in part profile.

  • 2
    Maybe it's just me, but this sounds impossible, in the same vein as a program that could output a full website based on a scanned, hand drawn mockup. – Scribblemacher Dec 23 '16 at 17:21
  • 1
    @Scribblemacher Im sure its possible one day. But that day is not today. We can barely do a 3D model of a face with photogrammetry due to the very exotic nature of skins shading qualities. And again if this tech existed it would as you say be able to do stuff that is much more involved. So if it exists then none us can afford it, more likely it does not exist in a meaningfully usable way. – joojaa Dec 23 '16 at 18:20
5

No, software like that does not currently exist.


Now that the state of the art has progressed slightly, there is in fact some bleeding edge tech that can do this quite well:

It is still not perfect, even usable in many cases. For example noses tend to be a bit crooked, the meshes are a bit generic and so on. But the model fit is getting there.

enter image description here

Image 1: Reconstructed mesh, without textures. Looks quite generic. But once you add textures it becomes recoginzable.

As you see the 3d model is telf is not really super good, since it does not capure human detail. Offcourse projecting a texture of a face makes a big difference. Unfortunately this makes the tech unsuited for stuff like 3d printing or accurate relighting of the model.. But might be suitable for a game character.

Contrast that to a real scan were the details are really there. So you can do much better. Even with just a single snap from a 100$ kinect makes 10 times better results with fewer polygons.

  • Software like that must exist. It just may not be available for sale – vfclists Dec 26 '16 at 10:45
  • @vfclists no there is no indication in lore of that such software exists, while there is indication of unsolved problems that such software would need. Indeed if such software existed it would make many of the problems that exist in robotics, industrial automation, xray microscopy etc etc solved. That would mean that such a software would solve fundamental issues in many fields of research. There is a high demand for this kind of software, so most likely it does not yet exist. Note that such software would change the world once it does exist. Note software do not have intelligence. – joojaa Dec 26 '16 at 11:51
  • Please see my answer below. Such software DOES exist albeit it is not easily usable by layman. – Lennart Rolland Jul 9 '17 at 12:37
  • @LennartRolland how can a software that works on one image and requires no user input not work for a layman. Immediately when you add human intervention to the mix it moves from the realm of not possible to possible. But then artists do that all the time. – joojaa Jul 9 '17 at 14:54
  • What i mean is, you might need to be a developer to get the software runnable on your platform, that's all. – Lennart Rolland Jul 9 '17 at 18:16
3

In the future (perhaps soon), there will be software that recognizes objects in photos and then looks up 3D models based on the identified objects.

Carnegie Mellon University did some research on this subject, which you can check out here. The video is pretty sick/scary. This is from 2014, so they may be further along now than they were back then. I haven't kept up with the development of the program.

  • 2
    The video is impressive, but even in the longer one they gloss over how the background is filled in. As for OP's question: all we need now is a comprehensive database containing everyone's face ... in 3D. – usr2564301 Jan 3 '17 at 1:23
3

Yes there is! But I couldn't find a way to export the texture though . . .

http://www.cs.nott.ac.uk/~psxasj/3dme/index.php

This one is very easy to use too!

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • 1
    Hi Arnold, could you please explain what this tool does and why you are recommending it? That way, your answer is still of some value in case the link breaks at a later time. Link rot is rampant, and the community tends to downvote answers that rely on just a link. Thanks for understanding! – Vincent Sep 20 '17 at 13:48
  • I noticed this tool a few days ago, and perhaps I should have come to answer my own question :) – vfclists Sep 20 '17 at 14:33
  • @vfclists that might be the case but linking to other sites without an example of what it does is spam. SE seeks questions and answers that will help others in the future. When people provide answers like this over time the links may rot and it doesn't add value. Q&As should be able to live over time without relying on external help. – DᴀʀᴛʜVᴀᴅᴇʀ Sep 20 '17 at 14:48
  • Yes i noticed this paper . The texure is the image you originally sent projected. – joojaa Sep 21 '17 at 7:48
  • OK the projection seems to be 200 units wide and 200 units high centered at 100 100 If you want to replicate the texturing – joojaa Sep 21 '17 at 8:18
2

You say you don't want to look at photogrammetry software, but that is exactly what you should be looking at.

More specifically Stereophotogrammetry is software that compares multiple photos to estimate 3D data to reconstruct objects in the photo. You can find a list here that is getting close to 100 programs. There are three options in the list that work with two images. One old CMU project worked with one photo but you will find it projects the image onto some planes for a "3D effect" and didn't actually create a 3D model as you are looking for.

At this stage I am sure there is no option to construct a 3D model from one image, it may be possible in the future but I don't expect it will be available any time soon.

1

If we have one photo of an object, there exists infinitely possiblities for "What is object's real 3D geometric shape". In mathematics there is no way to figure out the 3D shape from a single 2D picture.

We - people - are jailed by our thinking practices and learnings. If we see a well taken portrait, we very soon figure out who that person is, if somebody who we remember, fits. Even, if the portrait is taken straight from the front, we easily imagine how he looks out sideways and that happens even if we never have seen him. But the portrait can in reality be taken of a flat photo or of some other person, who happens to look out same from camera's direction. That other person can have much longer nose, but the straight onto face portrait tells nothing about it. The portrait even can be taken of a concave (=a negative mask) that is appropriately coloured inside.

Our limitations quide us to say, for example: "That's John Doe!". When we are told "No, this photo is taken of his portrait which is flat, John Doe weights 100 kilograms" , we justify our hasty decision by saying "Who in the hell takes a photo of a photo!"

Evolution or God has made us to take actions which are based on sparse information. The probability of choosing right has stayed high enough to keep us alive. So, sparse information can be sufficient. Thats why there is not any theoretical obstacle in front of developing a computer program that, like us, chooses a plausible 3D face model from the infinite set of possiblities, which becomes available, when somebody inputs one 2D portrait.

If we Google "single photo face modelling", there pops out at least as much to choose from as if somebody fatally ill tries to find the magic snake oil or the miraculous healer, who takes his cancer off.

Some of the face modelling suites are only frameworks, where the user makes all decisions. The framework only give the tools to input those decisions and to see their results instantly.

Some face modelling suites really have some knowledge about faces. There exists some general bacic face types to choose from. The user must point or draw feature by feature, which part in his photo corresponds a feature in the basic model. When the feature list has been walked through, the program outputs the model that the user can fine tune.

Some of "face model from a single photo" suites are claimed to be automatic and there are available demo videos that look out very respectable. I really like to know, if those demos are more than the one that an alchemist shows to the monarch to get still a little more gold to be finally able to make wagon loads of gold.

In theory automatic single photo face modelling can be more than alchemy presuming that they really are made to take into the account "what is probable in reality". It means the ability to do proper reasoning - artificial intelligence - at least, when the talking is about the faces.

  • 1
    Right heurestics can be made, but a face is a complex thing. We humans have tuned our brains to face identification. Thus there is a huge gap between something and usable. anyway asking humans for input is not automatic a human can do this no doubt. Anyway for these heurestics to be good one needs to solve many currently fundamental issues in computer vision... Give the toolset about 5-10 years thenits probably done. – joojaa Jan 3 '17 at 6:11
1

Yes such software exists.

But it might not be user friendly enough that anyone can use it.

Here is a video demonstrating the software in use:

https://www.youtube.com/watch?v=nice6NYb_WA

Quite remarkable!

  • This is close but not automatic. It requires user intervention, the fitting alogithm is quite nifty though. But programs like this have existed for a while. – joojaa Jul 9 '17 at 15:03
  • Note that there is a big usage difference between semiautomatic and auomatic, automatic means you can drop it on a database and start crunching. Semiatomatic means well that you habe to pay somebody money. – joojaa Jul 9 '17 at 15:13
0

The best/easiest way we can do what you're asking for with current technology is using a 3D scanner, which is quite expensive. Software nowadays can easily recognize faces, but a "face" in this case is just a bunch of ratios that are put together. Building a whole 3D model involves way more details, and you just cannot get that through an algorithm. Even if it were possible, you would need a lot of images in the exact same light, place, and posture, just different rotation.

0

I would say that it is impossible right now since for the Software the Image is just Flat. It doesn't know how far different Parts of the Image are so It can't make a 3D Model.

0

I'm not sure your negative answers are correct. A 3D model could be produced from one photo which is identical from the angle of the photo. The rest would be based on estimation and average depths. The resulting mesh could then be manipulated to give the desired result.

  • 1
    Could but curretly , does not exist. Like i said it would solve many fundamental issues in many industries so if it were to exist then it would be worth hundreds of millions. Which is why it most likely does not exist. Automatic is different from semiautomatic. – joojaa May 7 '17 at 4:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.