5
$\begingroup$

As a Corollary to this question (Using Blender with wider gamut than 709 primaries (like ACES)). How can I use blender to integrate CG, and footage from cinema cameras using ACEScg?

My goal would be to convert the footage to ACEScg in Resolve, generate and composite in blender and return to resolve without having to re-grade the images. Is that workflow doable?

$\endgroup$
0

1 Answer 1

9
$\begingroup$

As a loose outline, it’s much better to skip the ACES complexity and focus on what is required. ACES comes with its own entire set of issues, not the least of which is obfuscating the goals and the technical needs.

It’s All About Radiometric-Like Ratios

Integration requires using a specific encoding. That is, the key here is that we seek the radiometric ratios as they existed in the scene. This is perhaps the number one aspect of integration that is, and has been, overlooked for a long, long time amongst many folks.

The key here is to understand that cameras capture scene data. However, despite all cameras doing so, they all do not encode the data in a manner that is useful. Typically, camera encodings can be described in one of two manners:

  1. Ready to view as an aesthetic, display ready encoding
  2. Ready to decode and work with further

The Encodings

You’ll find countless examples online where people take random footage and attempt to integrate it. Some even can look pretty remarkable, such as some of Ian Hubert’s work.

The sad part is that this isn’t the appropriate means to make life easy for themselves. If the footage “looks ok” when played back, it likely is the absolutely wrong encoding to work with, and will result in no end of massaging, sucking up countless hours of time and energy, only to never-quite-look terrific.

This should be common knowledge now for anyone who has tried to take a generic JPEG photo found online and attempted to integrate it into a Filmic based scene; it just can’t be done well. Why? Because the display-ready image has become an impossible to decode combination lock on its way to becoming pretty to look at.

The Cameras

It might shock a majority of readers to learn that you don’t need an Arri Alexa LF to get these sorts of magical encodings. Sony, Nikon, Panasonic, Fuji, Canon, and other vendors now commonly include a log-like encoding scheme in their affordable consumer grade cameras. A good number of them offer the ever-more-useful-and-important on-board 10 bit or greater bit depth recording.

While countless YouTube videos wax lyrical with misinformation about “flat” picture styles and what they are good for, the value of the documented log encodings is entirely and completely missed; log-like encodings capture and encode scene reflectance!

That is, for every log-like encoding scheme such as Sony’s SLog1-2-3, Fuji’s F-Log, Panasonic’s V-Log, Nikon’s N-Log or Canon’s C-Log1-2-3, there is a documented function available in a PDF that describes how the values are encoded with a unique transfer function.

Unlike a display’s transfer function that encodes display linear light however, the gold is in the little detail that those normalized code values can be unlocked back to scene reflectivity!

Where sRGB’s transfer function can be undone to reveal the display linear light output from 0-100%, a log-like encoding can be taken back to the scene’s ratios from some low value to some high value.

Try It Right Now

To really hammer this point home, a simple and revealing test can be done:

  1. Take any old generic render with a reasonable dynamic range to illustrate the point.
  2. Save the image with a sixteen bit integer TIFF using Filmic Log Encoding Base. That’s the very flat, non-aesthetic looking one.
  3. Assert that no looks are baked into the output. We want data, not aesthetics.
  4. Using the Image Viewer, sample around the render using the Right Mouse Button. Note the values from some of the scene energy that appear on the left side of the information bar that appears. That is, the RGB values to the left of the CM values, not the right. They should extend up arbitrarily high.
  5. Also save the exact same image using Filmic with Base Contrast to make it look OK. Save the exact same sixteen bit TIFF.

Once saved. Open up another instance of Blender.

  1. Load the Filmic Log Encoding Base image.
  2. Set the colour space on the image to the corresponding Filmic Log Encoding Base.
  3. Set up a compositor node to assert the file is promoted to float and sample the values in the Image Viewer.
  4. Add in one more input and load the aesthetic output Filmic with Base Contrast. In this instance, the image was encoded for an sRGB display, so the transform is sRGB.

Note the values? Despite the fact that the sixteen bit integer TIFF was encoded using a device referred 0-100% range in both cases, when properly transformed via an appropriate transfer function such as Filmic Log, the values magically are transformed right back into the scene referred domain. The aesthetic “looks good” Filmic with Base Contrast? Hrm... only 0.0 to 1.0 representing the display linear aesthetically warped light, not the scene.

And guess what? Those logs for all of those awesome cameras listed above? They’ll do exactly the same thing!

This allows one to take the log-like encoded footage and seamlessly integrate it into any CGI with nearly no effort whatsoever. Use them as reflections, as emissions, and have interactive lighting on all your work.

But Wait, There’s More

There is one little gotcha with the log-like encodings; they are almost always coupled with a unique set of primaries that are different from the BT.709 assumed lights in your sRGB display, and the default set of rendering lights in Blender for Cycles and Eevee. The details of the particular encoding are unfortunately often referenced as some sort of a “gamut”. That is:

  • F-Log will be also encoded with F-Gamut
  • C-Log 1-2-3 may be encoded with a few different gamuts
  • N-Log will be encoded with N-Gamut
  • V-Log / V-Log-L will be encoded with V-Gamut
  • S-Log 1-2-3 may be encoded with a few different gamuts

One must take care to, after linearization with the proper transfer function, to also take the primary lights to the same primaries in the rendering / working space. This is a slightly nuanced subject, and likely requires another answer unto itself once the camera specifics are known.

No Log Camera? No Problem!

Beyond all doubt, this is a mind blowing experiment for the intrepid 3D peep to muck with, and hugely encouraged for anyone who has a log-capable camera.

But what about all of us schmucks who don’t have the releativey more recent cameras with log encoding options? There is indeed hope with a little bit of effort.

It’s entirely possible to take generic camera footage and document the transfer function manually. It’s a little elbow work, but the results will be similar to a native log, and the output will be significantly better than ignoring the core issues. Thankfully, there is an absolutely amazing set of tutorials to achieve just this via Paul Chambers’ website. After having read that amazing series, there is another little hidden gem on his site to help you use that radiometric scene reflectance too!

The key point in all of this is never let some overhyped bit of a plug-in or system prevent one from learning the core principles. No amount of any canned system can help you here; it takes a bit of elbow grease and learning to unleash a heck of a lot of amazing work. You never know... there might even be a super top secret and visually stunning project being developed in dark crustacean labs. I wonder what would happen if someone asked them...

$\endgroup$
7
  • $\begingroup$ Thanks for the detailed answer, lots to digest there. To further clarify, I'm not dealing with jpegs or any imagery in or sRGB. I'm looking to use footage from Canon c300MK and Blackmagic cameras recorded in RAW (blackmagic does not provide any documentation on their color specs, but can be converted from resolve into any other colorspace). $\endgroup$
    – susu
    Commented Apr 11, 2020 at 5:25
  • $\begingroup$ I guess it makes a lot of sense to work in log encoding, but what color space? The only reason I was looking at aces was to work in a "common" color space, so that I could mix shots with cameras with very different color rendition and add some CG elements created in blender, all using Scene Referred values. Should I just aim for Rec2020 colorspace and log encoding then? $\endgroup$
    – susu
    Commented Apr 11, 2020 at 5:25
  • $\begingroup$ One of my concerns also being able to view the footage correctly while working in blender. My goal would be that the CG elements, once rendered and composited into the footage, don't look different. Roundtripping back to resolve without messing the color correction for the shots. $\endgroup$
    – susu
    Commented Apr 11, 2020 at 5:25
  • $\begingroup$ Correct that BM does not publish white papers, so you are saddled with using Resolve to decode to EXR, and the path is fraught with potential pitfalls. Canon does publish white papers, for example downloads.canon.com/nw/learn/white-papers/cinema-eos/…. The issue with using wide gamut rendering spaces is gamut mapping; the results look like ass without it. ACES suffers this. Gamut mapping is tricky, however, and on smaller gamut devices, one can make a case that the result should be similar to rendering in the smaller gamut. $\endgroup$
    – troy_s
    Commented Apr 11, 2020 at 6:11
  • $\begingroup$ Are there benefits to rendering in wider gamuts? Absolutely. Without gamut mapping however, the results are simply unacceptable. Feel free to look at the garishly over saturated ACES examples out there for a good reason to stay away until more nuanced handling lands. Take all footage to the same rendering space and your concern about mixed sources largely goes away. It’s probably easier to break this open question into smaller sub questions as you work through it. $\endgroup$
    – troy_s
    Commented Apr 11, 2020 at 6:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .