1
$\begingroup$

I want to animate a picture of a puppet with two images, one with a closed mouth and one with an open mouth. I want to trigger the open mouth when an audio track goes above a certain volume.

This thread shows how to bake an audio track to an f-curve that controls the opening of two lips. I searched how to do the same to trigger an object's visibility, with words such as blender bake sound f-curve dopesheet show|hide|visibility layer|object and couldn't find a solution.

How can I trigger a layer's visibility from the volume of an audio track?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ I have no idea if you can change the baked F-curve, but if not, you can still use a driver to control the visibility of an object or control a shader to display different images. Actually, the property for the object's visibility is the opposite "disabled in render/viewport" so you need to flip the control value. For images you can use a Mapping node or a UV Wrap modifier or even a Mix node. $\endgroup$
    – Blunder
    Commented Nov 10, 2023 at 2:09
  • $\begingroup$ @Blunder Thanks for the tip: I was able to setup a driver and control the visibility of an image (following youtube.com/watch?v=SwcMyAf3MCk ). The driver can take a Python expression; how can I connect it to a threshold volume in the audio track or to the baked f-curve? $\endgroup$
    – emonigma
    Commented Nov 10, 2023 at 10:07

1 Answer 1

Reset to default
3
$\begingroup$

Here is an example where I do it with three images of a closed, slightly open and more open mouth and two different thresholds, don't know if this is what you are looking for:

I take a Value node and give it a keyframe so that I can bake a sound to the curve in the Graph Editor (under Channel > Bake Sound to F-Curves).

Then I have the three Image Texture nodes and two Color Mix nodes. First I'm mixing the closed and slightly open mouth. The mix factor is a threshold I've set with a Math node set to Greater Than and some value which I want to be the one where the mouth switches from closed to open.

In the next Color Mix node I plug the result of the first, and the other input is the fully open mouth. The mix factor is another threshold that's higher than the first.

mixing images with sound curve

These mouth images do not move realistically to look like they were actually saying the words since they do not adjust the shape to vowels etc., but for a simply two-image version just mouth close/open this should work fine. Only the threshold value is something you might find out with try and error when it looks best or at least good enough.

Since we can only upload GIFs here and they have no sound, below the mute animation there is a link to a video with sound. This is far from perfect, just a quick test.

moving mouth

Video: mouth with sound

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ just one addition: the "Bake Sound to F-Curves ..." menu entry is in the "Channel" menu of the Graph Editor (Blender 3.x). It's no longer in the "Key" menu as mentioned in the question linked above. $\endgroup$
    – Blunder
    Commented Nov 10, 2023 at 13:30
  • $\begingroup$ @Blender Of course you're right. I disregarded explaining more in detail how to bake sound to F-curves as this did not seem to be the problem and already been done. But good additional information. $\endgroup$
    – Gordon Brinkmann
    Commented Nov 10, 2023 at 13:35
  • $\begingroup$ Perhaps I should have posted it as a comment under the answer, right? Just saying this because I couldn't find it right away when I followed the instructions. $\endgroup$
    – Blunder
    Commented Nov 10, 2023 at 14:36
  • $\begingroup$ @Blunder You posted it as comment under the answer, didn't you? I can see it there ;) $\endgroup$
    – Gordon Brinkmann
    Commented Nov 10, 2023 at 15:57
  • 1
    $\begingroup$ @miguelmorin You said you wanted to switch between two images, so I assumed you were doing this in the material for an object. What you see in the screenshot is the Shader Editor, in which you can edit materials. If you do not know what this is, I would highly recommend to watch Youtube tutorials on material setup and texturing to learn the basics of this. $\endgroup$
    – Gordon Brinkmann
    Commented Nov 16, 2023 at 12:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .