5
$\begingroup$

Apologies for this possibly too vague question, but are there some general guidelines to follow when rendering artificial 3D meshes together with an imported 2D image as the background?

In the attached Blender file (or rendered image), the cube looks so out-of-the-scene. Some reflection from the green background onto the cube should make the cube more realistic in the scene? I don't know how to put it in professional terms -- any suggestions are welcomed!

enter image description here

$\endgroup$
  • $\begingroup$ To pack an image please read: blender.stackexchange.com/questions/44225/… $\endgroup$ – cegaton Dec 25 '16 at 0:27
  • $\begingroup$ @cegaton Ah I see. I've packed the image and re-uploaded. Thanks a lot for the guidance! $\endgroup$ – Sibbs Gambling Dec 25 '16 at 0:29
  • 1
    $\begingroup$ For this type of thing it is generally advised to use a spherical HDR environment map as a world texture, so that the lighting of the scene perfectly matches that of the background. Results will not be the same with a regular 2D image. $\endgroup$ – Duarte Farrajota Ramos Dec 25 '16 at 3:18
  • $\begingroup$ @DuarteFarrajotaRamos Thanks! But I only have a regular 2D image available. :-/ $\endgroup$ – Sibbs Gambling Dec 25 '16 at 3:27
  • 1
    $\begingroup$ Cegaton's answer seems to be quite complete and cover all points mentioned. $\endgroup$ – Duarte Farrajota Ramos Dec 25 '16 at 17:12
13
$\begingroup$

What would make the CG imagery feel integrated with the environment? Realistic and photorealistic are complicated... Let's start with the most important concept:

There is no shortcut or magic button in blender for "realism".

(if that is what you are asking)

You can achieve extremely convincing results if you are willing to put a lot of hours into learning about imagery, light, materials, surfaces and how those work in the real world, and in the virtual CGI generated ones,. The more effort you put into understanding the concepts listed above and the specifics of color representation in digital images, the easier it will be to create realistic renders.

Uff... now that we've gotten that out of the way, on to some ideas where to start.


Camera settings:

Camera height, rotation and lens (field of view) have to match those of the original camera used to take the picture (or video), as close as possible. Wrong perspective will break the illusion of the CG object being part of the image.

Having the correct information for the sensor size and lens will also help you determine the correct depth of field for the virtual camera in the 3D environment.

If the image used as background has the camera model and lens used as part of the EXIF tag, then you can use those values.

If you don't have access to such information you'll have to estimate it. See this post how do I align my grid to backfround footage? and How can I recreate geometry using a photograph? and Lens specs of smartphones for camera tracking in Blender for some ideas.

Materials:

Use Shaders and Textures that resemble those of the real world. Research the Albedo (reflectance) values of the materials and make those be the "color" of your shaders.

Materials in CG tend to be too perfect, objects in the real world always have imperfections and irregularities, like scratches, nicks, bumps or dirt. Paying special attention to those imperfections will make images seem more real. There are numerous ways to create scratches and dirt maps, along with bump and displacement maps to create cracks and dents.

You might also want to research the use of PBR shaders(Physically Based Rendering). Starting with version 2.79 cycles there is a Principled BSDF shader that allows for the creation of very realistic materials.

Light:

The complexity of the lighting of the environment you want to recreate has to be taken into account. You have to study the image carefully and understand where the main illumination is coming from and how the light is interacting with the objects. You can recreate it like they do in movie studios, by careful placement of lights with different characteristics (ie: big sources for the sky, smaller sources to have defined shadows and colors, etc) The hardest part is re-creating all of the many ways in which light reflects on to the environment and interacts with objects on the scene. In your example you would have a lot of green on the scene filtering trough and reflecting on the leaves on the trees, but you would also have a lot of reddish light reflected from the the ground. Plus some parts of the object would be lit more directly from the direct sunlight an others from light in the blue sky than others, and there will be many different intensities on the shadows depending on the size of the trees and the density of the foliage.

One way (but certainly not the only) to deal with complex lighting like the one on your example, is to use a High Dynamic Range image as environment for the scene. Ideally the HDR image would have been taken on the same place and is made of several exposures blended together. The idea behind using an image as environment is that every pixel of such image becomes an emitter and contributes to the overall lighting and coloring of the scene. Think of it as a big sphere that has the image of the surroundings mapped on to it, and that uses the light values of the original scene.

Rendering

The image used in the background is already compressed to deal with light in a non-linear way. The CG elements are created using linear light values, and have to be transformed to match the characteristics of the image on the backgound. You can use color correction tools like the ASC-CDL node to make the image feel more integrated with the background.

Using the Filmic-Blender set of curves for color management will make it easier to have an image that resembles the way cameras deal light in the real world. I strongly recommend reading through @troy_s ' post to get more details on the concepts of photorealism and color transformations.

Compositing

Your CG object needs to cast a shadow on surrounding elements. Create extra geometry (of the ground for example) and set it to be a "Shadow Catcher" to be composited on top of the background image.

Adding elements that belong in the environment will also help to make the integration more convincing. Things that interact with the object: dead leafs on top, random dust and debris, parts of the image occluding the object, moisture or mold, etc. If the object is glossy it should also reflect parts of the background. Use masks to create elements that are supposed to be in front of your objects

If you care for quality, you will discover very soon that most imagery in compressed formats (like jpg and most video shot with lots of compression) creates endless problems and is unsuitable for this kind of tasks. Try to work with high quality originals with little or no compression, high dynamic range and a high bit detph (images in 16bit or 32 bit float will give you more information and will lessen artifacts)

It is unlikely that you will get an image that is ready to be composited over a background convincingly using a single pass, so you need to be willing to learn how to modify alpha channels and use different passes like occlusion, indirect rays, reflections, etc.

Other considerations for compositing are the characteristics of the image used in the background. CGI imagery should try to have the same overall noise structure flares and other imperfections as the original, like Adding some lens distortion and chromatic aberration. The goal is to match the quality and defects and peculiarities of the original image as close as possible.


Here's a very rough idea of a scene lit with an HDR as Environment texture:

enter image description here

Long story short: this is too large a topic to discuss on a single post, and there are plenty of books, magazines and websites devoted to it.

A vast topic but endless source of fun and creative obsession... Good luck!

$\endgroup$
  • $\begingroup$ +1 -- many thanks for the detailed explanation! Both you and @Duarte mention HDR images as the background. I know what an HDR image is, but fail to see why it is ideal for this case? Does Blender automatically "use the HDR image as the lighting source" (although I am not sure what the quoted line actually means)? It seems to be the key. If I understand how to "use the background image as the lighting source", I can improve the realism with the 2D image I have at hand (although it's just a regular image). Many thanks, and happy holidays! $\endgroup$ – Sibbs Gambling Dec 25 '16 at 16:48
  • 1
    $\begingroup$ You can use regular non-HDR spherical images as background and lighting source too, the advantage of HDR is that lighting strength will have absolute realistic values, whereas a non HDR is limited to the 256 per rgb chanel $\endgroup$ – Duarte Farrajota Ramos Dec 25 '16 at 17:08
  • 1
    $\begingroup$ The issue with using Non-HDR images is that the light ratios will be severely compressed. $\endgroup$ – cegaton Dec 25 '16 at 17:10
  • 2
    $\begingroup$ The image you provided is not an HDR, so it is not suitable for lighting a scene. Look for an HDR shot in a similar forest environment...I used the HDR to create the scene and then composited that on top the image you provided for illustration purposes. Note that using Low Dynamic Range is not the optimal workflow, such mages will make it much harder to give you the photorealistic results you are after. $\endgroup$ – cegaton Dec 26 '16 at 18:02
  • 1
    $\begingroup$ I did not save the file... plus I can't re-distribute the HDR image. Just do a search for HDRs that resemble your image. Most of the information you need to do your project is in the links posted on the answer and comments. Read, try different things and if you get stuck then post your questions on the site. $\endgroup$ – cegaton Dec 27 '16 at 17:49
3
$\begingroup$

Not to take anything away from cegaton's reply...

If I understand your question, the one thing that makes a mesh appear part of a 2D image is to integrate it into the image such as placing it behind something in the image. That only needs to be something incidental such as part of a fern or tree branch etc. You can even duplicate something in the image and place that in front of the cube.

enter image description here

I've done a quick (and dirty) job here to illustrate how simple it can be in Blender to make a mesh/photo mix convincing enough for practical purposes such as for video. If you're needing an artwork finish, then of course this would be too basic.

I can upload the blend file if you need further information. I'd do it now but it's 4 AM here.

For anyone interested, I've used a plane placed in front of the cube with transparency set to .008 to cut the tree's trunk shape out of the cube. That took just a few minutes. The cube was parented to the plane, then un-parented again immediately after. This ensures the plane will mask out that section of the cube. I think the lighting (Hemi lamp) is aligned roughly to the photo's light source.
It was rendered as a png file (no alpha) using the OpenGL render.

(I did mention it was a quick attempt)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.