How to remove backgrond using a photo with only the background? (GIMP/others)

Question

I'm trying to remove the background from a photo with a subject, but I would prefer not need to use this kind of techniques to select the background. Some magic-fu would select the shape since it's easy enough to identify the subject from two identical pictures, one with and another without the subject.

(Just as an additional point, in the link above the background is very simple so it is easy to use the magic-wand; a more complex background would not be so easy to deal with. However, if the technique I'm looking for exists, I would expect the same level of difficulty for both backgrouns)

In pictures:

I have both, "just background"

and "background with subject"

Which were, in the real case, obviously obtained through photographing and not digital editing.

I thought it would be possible to do this automatically if I had a picture of the background without the subject. Sort of, programmatically, remove every repeated pixel and only keep the different ones.

Although this makes sense to me I can't find instructions on how to do this. Maybe because I don't even know how to search.

Background to greenscreen doesn't seem to be it.

I'm using GIMP, but if you know how to do it using other application, I can try to "convert" the instructions.

Answer 1

It might be possible to import both images as layers and apply a "grain extract" blending mode to the top image. If the pixels on the top layer and bottom layer were identical, i.e. in the exact same position, at the exact same exposure, perfectly aligned, at the same focus, and with no movement whatsoever. Then the identical pixels in the top would cancel out the identical pixels in the bottom leaving you with a 50% grey background, which theoretically you could select using the Select by Colour tool with the threshold set very low. Then you could use that selection to create a layer mask.

Here's an example:

Then apply the selection as a layer mask to the original image

Like your examples, I simulated these two shots digitally, and the backgrounds are identical down to the pixel level.

In the real world however, the chances of getting two photos with the exact same pixels in the background, are next to zero. Especially if the background is something in the natural world - leaves move in the wind, and waterfalls move, even cameras on tripods can move. Any slight misalignment would mean the pixels would not cancel out as a smooth grey, making the selection more difficult. It could possibly work in a studio where you could control every aspect of a shot, using a tripod, flash, a remote so you don't have to physically touch the camera to fire the shot, manual focus and manual exposure mode, etc.

You get where I'm going with this I hope . . . it's not very practical. That's why methods such as greenscreen were invented. That's why it's used for video, where there might be thousands of frames in a shot that lasts only a few minutes.

Answer 2

How the professionals cut off the background, when the background is natural, not prepared or selected for easy removal?

Answer: They draw a path along the edge and use it as the clipping mask. That's hard work if you must be accurate enough for big size printing in high quality.

Fortunately there exist some tools that generate the path along a clean contrast or color border semiautomatically, you only hover the mouse approximately over the right border. The easy parts of the path come out very easily. In difficult places, where human reasoning is a must, a good tool allow one to switch to bezier curve drawing on the fly - and of course - the errors can be taken back on the fly, too or they can be patched afterwards.

A variation of this uses no visible paths. One paints a color - say blue - over the approximate borderline. In difficult places he additionally paints red and green (red = remove, green=save). But the idea is the same. A human shows the approximate area where the border is and refines the difficult places.

You can try modern semiautomatic background removing without buying anything. There are web services for that. Check https://clippingmagic.com/ . It's not free, but you can try it without downloading the result. Your flying horse was extracted by using it in about 10 seconds.

Can this be automatic?

Modern image processing for security and military reconnaisance purposes can detect new objects by comparing images that are only approximately same. They can classify the new objects and surely give clean prints if they are needed. The classifying is based on advanced pattern matching and on some data about what at least can be expected.

I have not seen the following to happen, but I suppose: If the object must be separated from the background, the automatic software draws the statistically most probable clipping path, when no clear color or contrast border can't be followed or if the following seems to lead to astray.

You do the same, when the clipping path must be drawn by hand due the poor contrast.

Unfortunately I have no idea, from where thislike automatic capablity is available for the ordinaries like us. But surely it exists. The well working automatic face recognition in popular web applications and in many cameras, too is a quite strong proof.

Answer 3

If we're talking about state of the art solutions, I believe BGMv2 can be used for this as it doesn't seem to rely on temporal information. It's designed specifically for the task of removing static background when full background image is available. It handles hair really well compared to other popular options.

Sources, collabs, models and everything else is available on GitHub. There's an After Effects plugin too.