When my character speaks, I use shape keys to move the mouth, just up and down.
If I wanted to add a voice, do I need to record this separately and then insert as audio?
I'm asking if Blender has a voice sync or lip sync feature.
Blender Stack Exchange is a question and answer site for people who use Blender to create 3D graphics, animations, or games. It only takes a minute to sign up.Sign up to join this community
To drive lips from an audio file -
For this exercise both lips are simply curve-paths bevelled with a small diameter oval and left as curves.
The animated gif pic below is the Graph Editor and where these "f-curves" are created. They are simply graphs measuring time (frames) against amplitude or degree of movement. They can be most useful in that they can be dragged and reshaped to affect animation.
Please note that what you're seeing here is the (speech) audio file after it's been 'baked to an f-curve' The green waveform is that f-curve.
We use that to drive the yellow sound bar (a plane) to expand in the Y axis with the audio's amplitude. The louder it is the further apart the lips go.
The lips are simply curve paths and we drag each lip's center vertice up or down using a 'Hook" empty for each. The upper lip's hook is parented to an empty that's stuck on the upper corner of the yellow sound bar. (parented to) The lower lip's hook is parented to the lower empty at the lower corner of the sound bar.
As the sound bar is expanded and contracted by the audio, the two hooks are dragged up and down to follow the two empties attached to the sound bar.
Details - How to achieve this...
In Edit mode, add a hook to each of the lips' center vertice. (CTL-H --> Hook to New Object)
With the view set to TOP & Ortho, place the cursor to the right of the lips and add a plane. This will be the sound bar. Proportion it as shown. (stretched in the Y axis)
Scale it's Y axis to the max length that you want it stretched to by the audio. Bear in mind that speech may only get to about 30% of the maximum amplitude most of the time, so scale the Y axis to be at least 2 or 3 times that required.
At the upper end of the sound bar, place an Empty inside it and near to a corner vertice. Parent it to the plane. (CTL-P -> Vertice) (child=the empty)
Do the same at the bottom corner or near to it. Parent that to the plane/bar also. ("vertice") (child=empty)
Select the upper lip's Hook and parent that to the Upper Sound bar empty. (Child=hook) Do the same with the lower lip's hook. Parent that to the sound bar's lower empty.
This setup will force the Hook Empties to follow the sound bar's expansion and contraction in the Y axis and drag the lips' centers according to the sound level.
---> POINT A <--- (Return to here if the sound bar (plane) needs rescaling)
Driving the Sound bar's Y axis scaling from an audio file -
In the 3D window, take the timeline cursor to frame 1, select the sound bar and insert a "Scale" keyframe. (Hover in the scale slots at right and press i )
With the sound bar still selected go into the Dopesheet and delete the sound bar's other two axis' keyframes. In this case that would be the X & Z axis. We only want the Y strip.
Select and highlite the one remaining keyframe. (Y axis)
Go to the Graph Editor and you should be seeing the Y axis scale F-curve. That's a graph that draws frames (time) against scale. (size)
At the bottom center of the window, expand the "KEY" menu, select "Bake Sound to F Curves". A file explorer window will open up. Browse and select the audio file.
Blender will automatically sample this and convert it to a waveform. (Press Home to see it properly)
If it doesn't, try converting the audio to a wav file.
Back in the 3D window press play. (ALT-A) to see the sound bar's size expanding and contracting to the sound's amplitude. The lips should also be curving open and shut.
If the sound bar isn't expanding enough, remove it's keyframe by selecting that axis strip's name in the DOPESHEET at extreme left and pressing X. Check the graph editor to see that the F-curve has indeed disappeared.
Go back to "POINT A" above...
Information - Sound can also modulate lateral movement, rotation and even colour and brightness, the last two requiring some manual trickery because with most audio, it won't drive it bright enough.