0
$\begingroup$

I have trawled forums for ages looking for a possible answer to this.

I am trying to render shadows to a separate layer then recombining them in the compositor with the rest of the image. This is because I would like to have complete control over the shadow layer and ultimately apply a mask to it (to avoid any shadows extending past the render border creating a hard edge).

I have created a simplified setup to demonstrate the issue.

white line image

In this simplified setup I have 2 view layers, layer 1 containing a cube and layer two containing a shadow catching plane. You can see that the lines appear at the border between the shadows and the object. I have looked into issues of premultiplied alpha but none of this seem to sort it. They appear despite the condition of either Convert Premul checkboxes on the Alpha Over nodes.

I suspect that it may be to to with the Transparency filter in the Film section of the properties, however altering this leads to poor results.

Does anyone have a solution or some tips on what to investigate. This will ultimately be for an animation and so I'm keen to achieve everything in Blender rather than saving 2 sets of frames for the shadows and for the objects

For reference here is an image of just the first view layer with both the cube and shadow cater being rendered showing no lines

correct image

Any help or advise would be massively appreciated

$\endgroup$
0
$\begingroup$

I did a test scene like the one you did. I thought it was due to the premult setting (usually that's the guilty party for compositing edges), but that's not the case here. Part of it is due to anti-aliasing and the pixel filter - not because of bad settings, but just the nature of it.

With the cube by itself, the edges of the alpha are going white to black, with some grey in the middle. For the shadow catcher and holdout cube, the edges between the shadow's alpha are going white to grey to black, in the opposite direction. This edge is on the same pixels for both layers, of course. When you do an alpha over, that edge results in some grey alpha values from both layers competing for the same pixels. Therefore, some of your white background peeks through and shows up as white

The best solution is, simply, don't composite this. If you can avoid compositing layers of the same CG scene, render the shadow and object together. My guess is you want to do this for visual effects; I do that too and run into this situation often. I usually render the shadow together with any CG object I'm putting in a video to get a clean edge.

If you must composite them together, you can do some masking and simply expand the shadow layer's alpha edge with a dilate/erode node. That can get messy and give strange results, but it's possible.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Great answer! if perhaps not the answer I'd hoped for. This makes a lot of sense. You are right I am doing this for VFX but also as I often need to render objects, complete with cast shadows onto completely white surfaces so that there is no visible 'seams' at the edges of the frame caused by tonal differences from slight shadow remaining on the floor. I know there's ways to do this using masks and essentially white vignettes but it always struck me as a bit inflexible. $\endgroup$ – ollieb May 12 at 7:49
  • $\begingroup$ I have actually managed to achieve what I wanted by adapting some of the techniques used in this tutorial: youtube.com/watch?v=qgs5JGy2qIU&feature=youtu.be It seems that the shadow catcher in blender is not very full featured and so its beneficial to essentially build your own approximation. The only issue I ran into was that it does add render calculations.... $\endgroup$ – ollieb May 12 at 7:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.