How to make a cell shaded scene using the compositor

Question

Please someone show me how to achieve cell shading by using the compositor and without using ramped color materials. Since nodes are new to me it would be helpful to have a screen cap of the schematic of the node connections required and any additional information on how to set lamps and lighting if it is needed, also what should and should not be checked to allow anyone who wants to reproduce it obtain the same results. Do not refer to this link:

How to render cartoon style with completely flat colors?

This page does not make it obvious what has to be done relevant to my request, it is a dead end to a noob who wants to understand as there are so many answers but none of them seem to be about affecting an entire scene, only materials.

I happily thought how fun would it be to make some cartoons with Blender. Connecting nodes... more like like pulling hair, no tutorial I follow exactly produces the same results described even when I copy an exact demonstration there is something probably not node related that keeps it from working.

The example below shows exactly what I want, cell shading in blue, highlights in yellow that is not gradual or granular. Textures get the same treatment and if I add an entire city to this scene it will also get the same cell shading. Nothing in this scene has any ramped or toon materials.

Answer 1

You can create both cell shadows and highlights using the "Normal" render pass. This is how such pass looks like, when enabled you can view it inside UV/Image Editor selecting the Normal pass:

Render the scene without shading (or with minimal shading - ambient occlusion, etc.) and add cell effects in compositor like this:

Quick intro into compositor can be found here. Add nodes through the menu or with Shift+A.

You change the direction of shadow and highligh with those Normal nodes.

For nice edges Full Sample Anti-Aliasing should be on (Blender Internal setting):

Result:

It is worth noting that the Normal render pass is in global space (= coordinates relative to the world) and not camera space (= coordinates relative to the camera), so camera movement will not change the shading (it will not change how the normal pass looks like).

If you need cell shading working like a MatCap material is working, you will need a custom normal pass transformed into camera space. Since compositor has no Vector Transform node it cannot do this.

The solution is to create another render layer and override all materials in it with this material:

Use this layer instead of the Normal pass to make the shadow and highlight relative to camera and to follow it's movement.

Answer 2

I have a solution using scenes in Blender internal, although I don't see why something similar to this couldn't be made to work in Cycles.

First set up a scene with textured objects as normal, I've gone for three copies of Suzanne with different textures, and haven't put much care into texture mapping. Once your textures are set up, set all your materials to shadeless. since we're going to add cell shading in the compositor later.

At this point your render should look something like this:

Next, add a new scene with linked objects.

In this new scene, select each object, and press U to make it single user, then go to the materials tab and add a new material that is linked to the object.

This material should be simple and black and white, we're going to use this to decide where the dark areas are in the cell shading later. I made these materials entirely diffuse, and used sun as opposed to point lighting.

The second scene should look something like this when rendered.

Finally, navigate to the first layer, and enter the node view, change the node tree type to edit to "Compositing".

Set up the following nodes.

This mixes between the output from the texture scene, and a shadow colour, depending on the black and white value of the shadow layer.

Change the value of the Greater than node to influence how large the shadows should be, and change the Add node to influence the amount that the shadow colour gets blended into the shadows.

It would be possible to use colour curves, and an "RGB to BW" node as an alternative to the "Greater Than" and the "Add" nodes. This would give more control around the sharpness of the cell shading.

My final image looked like this:

Blend File here