how can I rig a 2D face onto a 3D model

Question

I'm trying to replicate the faces as seen in the Lego Movie, and somehow they get them to move. Someone recommended using the grease pencil tool, but I'm not sure how. These are like textures, but they rigged them inside the engine they were using. I would understand using black planes as eyes and eyebrows, but not if you want a 2D mouth that can open. I'm sorry if this question has been asked before, but I'm unable to find one like it

I understand this is a different engine, but is there a way to even do this in Blender?

Answer 1

I'd find it a bit hard to animate using textures, even though I know it is sometimes done. For Project Gooseberry, I read in one of the weeklies that they were thinking about animating UVs using drivers on bones to achieve this effect for the caterpillar scene, but it didn't look all too intuitive.

My approach would be to use the traditional armature tools, combined with a shrinkwrap modifier workflow. It would be too long to explain everything, so examine the (simple) blend here:

The trick is to have the armature deform the face mesh before projecting it onto the head mesh using shrinkwrap. So all deformation happens in a flat plane, the result is wrapped onto the head with some offset. This is what it look like before projection:

And this is after:

I use the same technique for the eyes, however you can see the distorsion on the circular shapes a bit too much for my taste, so here deeper involvement would be needed. For the mouth it works a lot better:

I really quickly hacked this together to show the concept only, much more dedication needs to go into carefully weighting this, but alas, where you take this approach to is up to you.

Answer 2

This is not the elegant answer offered by @aliasguru, rather a straightforward approach, posted for those not overly familiar with Blender and wanting to get a face 'talking' using 2D photographed or sketched lips.

The cube shaped head was used purely out of convenience and for speed. Planes can be easily wrapped around cylindrical surfaces such as Leggo heads.

Two planes were used here, one textured with a filled in oval shaped pic for the lips and mouth, another, not textured but subdivided to form teeth, gaps, and mouth cavity, each filled with their respective solid colour.

Pic 1 shows the 3D arrangement of these during construction and the bones used to deform their shape.

The bone structure has four bones on each side, their chain settings in the 'Inverse Kinematic' constraint set to 1-4 respectively and outwards from the center. The vertical or main armature is left at zero.

Both planes are parented to the main armature (vertical one)and set to "Automatic Weight". The planes can also be scaled & transformed (moved about) independently of the armatures in order to enhance their ability to express emotion.

Once completed the planes are moved quite close together (the teeth in front) then the armature structure is moved inside the head so the planes will end up against the face. The bones are parented to the head so the mouth is in effect, 'attached' to the head.

Teeth can be made to disappear alltogther by keyframing either an instant transform behind the lips, or adjusting their transparency.

The mouth gap faces in the Teeth assembly plus the lips plane can be distorted independently in the z axis to "open and shut" to emulate speech.

I see no reason why this arrangement couldn't be synchronized with a sound file although I have yet to explore this Blender feature and subsequently not carried out sound tests.

Answer 3

Two ways that I think you could accomplish this:

1. Animating the texture in after effects: Create some lips, eyes, and brow poses that you can animate between and cycle through them to the audio in something like after effects. Export the video to frames and use that as the texture.

2. I don't know if blender has this but I am pretty sure you can set driven keys in maya to animate textures, which would take out the step of animating the face in after effects.