Animation Node can be used to assign vertices normals to vertices colors in near realtime. Using those normals in cycles won't yield the same results as the Phong smooth shading cycles uses, however it provides very similar results.
Background Information
Understanding How Vertex Colors Work
The name Vertex Color is a bit misleading, it should be noted that color data is not stored per vertex, but rather per loop. A loop is basically a vertex of a polygon, so if a vertex is part of four polygons, it will compose four loops, to understand this better, lets look the following illustration:
The above is a mesh with 4 polygons, each polygon has 4 vertices and subsequently 4 loops. We represented loops by colored dots, you notice that each vertex compose either 4 loops as blue loops, 2 loops as red loops or a single loop as violet loops. Now, If we were to color all loops white expect the blue loop in polygon three, we will get something like this:
There are some advantages of storing colors in loops instead of vertices, the main advantage being the ability to color polygons. If colors were stored in vertices, it would not be possible to color a polygon with a single color, think about it ! To color a polygon using loops, we just have to color all loops in the polygon with the same color:
On the other hand, to color a vertex, we have to color all loops that are part of the vertex, same-color dots in the above illustration:
In case of coloring vertices, the fragments (pixels or dots) in polygons will be colored according to a convex combination of the colors of its vertices (a bilinear interpolation for quads), such interpolations are particularly useful to represents smooth shading in your case !
So if we were to color vertices based on some color list, we will have to identify the loops that are part of each vertex (same-color dots in our illustration) and color those loops based on corresponding color from the color list. We will also explain polygon coloring for completeness.
Understanding How Loops Are Stored In Meshes
Vertices have indices which we can use to identify and select them by, the same goes for polygons and edges. How are loops ordered and how can we identify them? To answer that, let us consider this simple cube as a study case:
Where polygon indices are shown in red and vertices indices are shown in blue. The ordering of loops follows two rules:
- Loops of the first polygon are stored first, loops of the second polygon are stored next, and so on.
- Loops in individual polygons are sorted from the lowest vertex index to the highest.
We can represent the loops as an ordered pair $(x, y)$ where $x$ is index of the polygon it is part of and $y$ is the index of the vertices that it is composed of, and the set of ordered pairs for quadrilateral mesh will be:
$$
\begin{aligned}
&(0, 0), (0, 1), (0, 2), (0, 3),\\
&(1, 0), (1, 1), (1, 2), (1, 3),\\
&(2, 0), (2, 1), (2, 2), (3, 3),\\
&(3, 0), (3, 1), (3, 2), (3, 3),\\
&\dots
\end{aligned}
$$
To understand this better, we drew the indices of the loops over the study case :
Coloring Polygons
In order to color polygons with flat colors, we have to color all loops in a polygon with the same color, in the above case, in order to color the polygon with index zero, we will have to color indices $(0, 1, 2, 3)$, to color the polygon with index one, we will have to color indices $(4, 5, 6, 7)$, and so on. Notice that the indices are in the pattern $(0, 1, 2, \dots)$, grouped in groups of length equal to the number of vertices of the polygon, so all we have to do is generate a list such that the first $n$ number of colors are the color of the first polygon where $n$ is the number of vertices in the first polygons, the second $n$ number of colors are the color of the second polygon where $n$ is the number of vertices in the second polygons and so on. In practice, we loop (iterate) over polygons, get the number of vertices in it ($n$), get its color from the color list, append it to an initially empty list $n$ number of times. An implementation in animation nodes is as follows:
Don't worry about the Set Vertex Color Subprogram, we will study it in a second. Notice that we used the polygon normals as colors in here. Also, we represent colors using 3D vectors; because they in fact are (Vertex Color doesn't support Alpha Channel).
Coloring Vertices
Vertices coloring is a bit harder to understand but easy nonetheless. Lets look at the study case one more time:
Remember that in order to color vertices, we have to color all loops that compose that vertex, in the above case, $(0, 4, 8)$ should have the color of the vertex at index zero, $(1, 5)$ should have the color of the vertex at index one, and so on. Notice that polygons are merely an indicator of which vertices are in each polygon, now remember that loops are stored in the same order of polygons, so the algorithm for generating the loops colors is as easy as looping (iterating) over polygons, using the indices of the polygon to get the colors at the same indices, append the colors. Take your time to understand it, you will only understand it if you think about it. An implementation in animation nodes is as follows:
Notice that we use the vertices normals as colors in this example.
Set Vertex Color
Animation Nodes currently does not have a node to set vertex colors directly, so we will have to write a small script to do that, a possible way to do it is as follows:
#Create a vertex color map named "Normals" if it is not already created.
if "Normals" in Object.data.vertex_colors:
map = Object.data.vertex_colors["Normals"]
else:
map = Object.data.vertex_colors.new("Normals")
#Loop over Colors and assign them to the vertex color map.
for i, color in enumerate(Colors):
map.data[i].color = color
The script creates a vertex color layer named Normals which is then filled with the data from the Colors list, the Colors list is a 3D Vector list of $n$ Vectors, where $n$ is the number of loops in the mesh.
Answer
You probably already know how to set the vertex normals to vertex color map by now, we did it couple of time above, so such a node tree should achieve what you want:
But wait, there are some things to take care of. Blender only store the colors in the interval $[0, 1]$, but our vectors are signed, ideally in the interval $[-1, 1]$ if normalized. This can be handled by linearly remapping the $[-1, 1]$ interval to the interval $[0, 1]$ in Animation Nodes, then in cycles use the inverse remapping function to transfer the vectors back to the interval $[-1, 1]$. The remapping function can simply be $\frac{V + I}{2}$ where $I$ is the identity vector matrix, and the inverse remapping function will be $2(V - I)$. Notice that $V$ has to be normalized, because while Animation Nodes doesn't care about the magnitude of the normals, cycles does, and will yield unexpected results if not handed normalized vectors. Moreover, normalization has to be done in Animation Nodes due to the assumption we made that the vectors are indeed normalized in the remapping function.
