I'm trying to achieve a stylized ink wash or watercolor effect where the image ground remains visible. Since freestyle generates such energetic lines, I want to use them to mask my render during compositing, defining the areas where the ground will show through. So far I haven't had much success.

With the node set-up below, I'm able to apply transparent lines to my image by inverting the alpha of my freestyle render layer before combining my render layers with an "alpha over" node. Unfortunately, flipping the alpha blocks out all areas of the image that aren't freestyle lines. I'm sure I'm missing an obvious step, but so far I haven't been able to figure it out. Does anyone know what I should do?

Freestyle Mask Attempt

An ID Mask seems like it would be just about perfect, but as far as I can tell, it's not possible to create one for freestyle lines.

Here's a simple test blend


You may use the Set alpha node to mask the render result with the parts where the freestyles lines are by picking the freestyle layer alpha cannel.

enter image description here

Notice that I also switch the Base Thickness mode from Center to Inside (see the panel on the right) to be sure to always mask parts that are actually rendered geometry and not background.

With higher values of thickness you may have some problems with the cap in the angles. I would suggest to multiply the two incoming alpha to create a mask capable of cutting them away (Multiply Math node runs the same operation of a Logic AND):

enter image description here

Now that we've "mastered" the alpha channel in a simpe way, we can Invert it's apha channel (check the RGB box to invert its values as if it was an image from the colr channel) to get a stencil look to the image:

enter image description here

You can make the jiggled edges by using one (or a comobo of more) of the several modifiers available, for example a Noise for the Thinkness parameter:

enter image description here

If you need to use the freestyle layer color to mask the rendered object, you'll need to set it of a different color from total black, or you won't be able to get its information without accessing the alpha channel anyway.

  • $\begingroup$ That's an interesting effect Carlo, but not quite what I'm trying to achieve. in fact, I think it's just about the opposite. I want to preserve the transparency of the background, but cut the areas covered by freestyle lines out of the normal render. I suppose mask was the wrong term to use, so that's my fault. I want to uses the freestyle lines similarly to bridges in a stencil, as well as adding some irregularity to the contours of my rendered form. $\endgroup$ – Tric Oct 20 '15 at 16:50
  • 1
    $\begingroup$ Don't worry, it's just a matter of inverting the colors (as you were correctly doing). I've edited the answer in this sense. $\endgroup$ – Carlo Oct 20 '15 at 18:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.