Blender Stack Exchange is a question and answer site for people who use Blender to create 3D graphics, animations, or games. It only takes a minute to sign up.
Sign up to join this community
Apologies for this possibly too vague question, but are there some general guidelines to follow when rendering artificial 3D meshes together with an imported 2D image as the background?
In the attached Blender file (or rendered image), the cube looks so out-of-the-scene. Some reflection from the green background onto the cube should make the cube more realistic in the scene? I don't know how to put it in professional terms -- any suggestions are welcomed!
What would make the CG imagery feel integrated with the environment depicted in an image? Realistic and photorealistic are complicated and nuanced...
Let's start with the most important concept:
Uff... now that we've gotten that out of the way, on to some ideas where to start.
You can achieve extremely convincing results if you are willing to put a lot of hours into learning about imagery, light, materials, surfaces and how those work and interact with each other in the real world, and in the virtual CGI generated ones. The more effort you put into understanding the concepts listed above and the specifics of color representation in digital images, the easier and more rewarding it will be to create "realistic" renders.
Camera settings:
Camera height, rotation and lens (field of view) have to match those of the original camera used to take the picture (or video), as close as possible. Wrong perspective will break the illusion of the CG object being part of the image.
Having the correct information for the sensor size and lens will also help you determine the correct depth of field for the virtual camera in the 3D environment.
If the image used as background has the camera model and lens used as part of the EXIF tag, then you can use those values.
If you don't have access to such information you'll have to estimate it. See this post how do I align my grid to backfround footage? and How can I recreate geometry using a photograph? and Lens specs of smartphones for camera tracking in Blender for some ideas.
Materials:
Use Shaders and Textures that resemble those of the real world. Research the Albedo (reflectance) values of the materials and make those be the "color" of your shaders.
Materials in CG tend to be too perfect, objects in the real world always have imperfections and irregularities, like scratches, nicks, bumps or dirt. Paying special attention to those imperfections will make images seem more real. There are numerous ways to create scratches and dirt maps, along with bump and displacement maps to create cracks and dents.
You might also want to research the use of PBR shaders(Physically Based Rendering). Starting with version 2.79 cycles there is a Principled BSDF shader that allows for the creation of very realistic materials.
Light:
The complexity of the lighting of the environment you want to recreate has to be taken into account. You have to study the image carefully and understand where the main illumination is coming from and how the light is interacting with the objects. You can recreate it like they do in movie studios, by careful placement of lights with different characteristics (ie: big sources for the sky, smaller sources to have defined shadows and colors, etc) The hardest part is re-creating all of the many ways in which light reflects on to the environment and interacts with objects on the scene. In your example you would have a lot of green on the scene filtering trough and reflecting on the leaves on the trees, but you would also have a lot of reddish light reflected from the the ground. Plus some parts of the object would be lit more directly from the direct sunlight an others from light in the blue sky than others, and there will be many different intensities on the shadows depending on the size of the trees and the density of the foliage.
One way (but certainly not the only) to deal with complex lighting like the one on your example, is to use a High Dynamic Range image as environment for the scene. Ideally the HDR image would have been taken on the same place and is made of several exposures blended together. The idea behind using an image as environment is that every pixel of such image becomes an emitter and contributes to the overall lighting and coloring of the scene. Think of it as a big sphere that has the image of the surroundings mapped on to it, and that uses the light values of the original scene.
Additional geometry
Create additional objects that will interact with those objects being added. If your object is reflective, the reflection of other objects has to come from somewhere. Shadows that are cast on the object itself have to come from the occlusion of the light by other objects. Likewise the shadow of the object has to be projected on some geometry, like the ground in this case Those new objects can be also invisible geometry (like the ground for example) if you use what is called a "Shadow Catcher", that will only generate the shadows to be composited on top of the background image.
Adding elements that belong in the environment will also help to make the integration more convincing. Things that interact with the object: dead leafs on top, random dust and debris, parts of the image occluding the object or the background image (like if you want the objects to feel that they are behind a tree for example), signs of moisture or mold, etc. If the object is glossy it should also reflect parts of the background. Use masks to create elements that are supposed to be in front of your objects
Rendering
The image used in the background is already compressed to deal with light in a non-linear way. The CG elements are created using linear light values, and have to be transformed to match the characteristics of the image on the backgound. You can use color correction tools like the ASC-CDL node to make the image feel more integrated with the background.
Using the Filmic-Blender set of curves for color management will make it easier to have an image that resembles the way cameras deal light in the real world. I strongly recommend reading through @troy_s ' post to get more details on the concepts of photorealism and color transformations.
Compositing
If you care for quality, you will discover very soon that most imagery in compressed formats (like jpg and most video shot with lots of compression) creates endless problems and is unsuitable for this kind of tasks. Try to work with high quality originals with little or no compression, high dynamic range and a high bit depth (images in 16bit or 32 bit float will give you more information and will lessen artifacts and images that use linear information as opposed to "gamma" corrected will be easier to integrate)
It is unlikely that you will get an image that is ready to be composited over a background convincingly using a single pass, so you need to be willing to learn how to modify alpha channels and use different passes like occlusion, indirect rays, reflections. On top of that sometimes you need several masks for solid objects and those that have semitransparent or blurry edges.
Other considerations for compositing are the characteristics of the image used in the background. CGI imagery should try to have the same overall noise structure flares and other imperfections as the original, like Adding some lens distortion and chromatic aberration. The goal is to match the quality and defects and peculiarities of the original image as close as possible.
Here's a very rough idea of a scene lit with an HDR as Environment texture:
Long story short: this is too large a topic to discuss on a single post, and there are plenty of resources out there, endless websites on VFX, books, magazines and rivers of ink devoted to what seems like a simple task.
A vast topic but endless source of fun and creative obsession... Good luck!
Not to take anything away from cegaton's reply...
If I understand your question, the one thing that makes a mesh appear part of a 2D image is to integrate it into the image such as placing it behind something in the image. That only needs to be something incidental such as part of a fern or tree branch etc. You can even duplicate something in the image and place that in front of the cube.
I've done a quick (and dirty) job here to illustrate how simple it can be in Blender to make a mesh/photo mix convincing enough for practical purposes such as for video. If you're needing an artwork finish, then of course this would be too basic.
I can upload the blend file if you need further information. I'd do it now but it's 4 AM here.
For anyone interested, I've used a plane placed in front of the cube with transparency set to .008 to cut the tree's trunk shape out of the cube. That took just a few minutes. The cube was parented to the plane, then un-parented again immediately after. This ensures the plane will mask out that section of the cube. I think the lighting (Hemi lamp) is aligned roughly to the photo's light source.
It was rendered as a png file (no alpha) using the OpenGL render.
(I did mention it was a quick attempt)
