3
$\begingroup$

I'm currently working on a VFX scene where I add volumetric fire to parts of my scene but in my footage I have objects that move in front of the camera I modeled each object in Blender and decided that the holdout shader would work great for masking out these objects since blenders z-depth pass doesn't work for volumetrics.

The holdout shader works perfectly but the problem is the edges of the mesh with the holdout shader create a very sharp contrast between the alpha and the fire.

Since I don't think it's possible to do blur the edges of the holdout material in compositing is there a way to create a node based holdout material that smoothly fades to alpha on the edges of the mesh removing the harsh edges?

here is a simplified blend file of the issue the holdout shader would be used to mask out 3d geometry with camera tracking but the edges of the holdout where the orange flames are behind the suitcase are too sharp I could blur the alpha channel but this would also effect the blue flames in front of the holdout

heres an example with images from a project i'm working on:

the flames are volumetric materials and the holdout shader would be used to mask out 3d geometry with camera tracking but the edges of the holdout where the orange flames are behind the suitcase are too sharp I could blur the alpha channel but this would also effect the blue flames in front of the holdout

enter image description here

enter image description here

enter image description here

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ "Since I don't think it's possible to do blur the edges of the holdout material in compositing" It is blender.stackexchange.com/questions/15449/… $\endgroup$ – Duarte Farrajota Ramos♦ Jan 14 '20 at 12:00
  • $\begingroup$ It is possible to blur the edges in the compositor. The alpha channel is just like any other image. But you don't necessarily need to use holdout, you can also use ID masks for the object or material ID masks or, even better, use cryptomatte. $\endgroup$ – user1853 Jan 14 '20 at 13:56
  • $\begingroup$ I was able to blur the edges of the holdout object mesh using renderlayers to correctly blur the smoke behind it but the problem is since it uses the holdout material as a factor for adding blur this holdout material is also visible behind the smoke in front of it causing the smoke in front of the holdout mesh to also be blurred I'll add a blend file with an example of this issue $\endgroup$ – Funny1048 Jan 15 '20 at 0:53
  • $\begingroup$ use the dilate>erode node docs.blender.org/manual/en/latest/compositing/types/filter/… read also: blender.stackexchange.com/a/148626/1853 $\endgroup$ – user1853 Jan 15 '20 at 6:04
  • $\begingroup$ Read also: blender.stackexchange.com/questions/73770/… $\endgroup$ – user1853 Jan 15 '20 at 6:07
2
$\begingroup$

From your screens it seems to me like you want to smooth part of an edge that is (in reality) optically smoothed by Bloom Effect. If you can’t use bloom effect in post-production directly, try this … take your mask object, blur its alpha, invert and subtract it from alpha of fire object.

There are two issues to solve ...

  • ad First - your example material setup use for Volume - Emission shader, that is visible in viewport render, but invisible in render (F12). Material is rendered, but alpha for this material is not generated. From developer side the bug is viewport vs render (F12) inconsistency, but F12 is as designed = emission without alpha is correct. This can be solved by adding Volume Scatter shader.

enter image description here enter image description here

  • ad Second - in blender wasn’t a way to get ID pass (or Cryptomatte) of object with volumetric material. From 2.82 beta build, blender use AOV that can output any state of material node tree. So I used AOV to separate specific alpha for Blurring. (Properties Editor > Layers > Passes > Shader AOV, create one. Go to Shader Editor, add AOV Output node with proper name and set white colour. For more about AOV see Blender Manual or Blender Today #86.)

Note: I had to add Transparent shader to Surface (feeding only Volume didn’t object pass as well). Pass of this volumetric object get object shape, not volumetric texture.

Since your scene consist only from fire I blurred mask object (cube). To get "fire" pass you can use Alpha socket ... enter image description here

Show before/after. Sorry for poor render (6 samples only). enter image description here

... or convert render with ColorRamp into black&white image. That is not mathematically accurate, but gave me nicer result. enter image description here

enter image description here

enter image description here

(I created also pass for orange fire, but in this comp was not needed.)

Improve this answer
$\endgroup$
2
  • $\begingroup$ awesome this solution seems to be working perfectly one thing I wondered is if It will still work if the blue fire is partially in front of the cube and with only the part of the blue fire behind the cube getting blurred $\endgroup$ – Funny1048 Jan 19 '20 at 23:27
  • $\begingroup$ Check the last (edit) blend, I will explain later (if needed). $\endgroup$ – vklidu Jan 20 '20 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.