7
$\begingroup$

I would like to use Blender to construct a mesh based on a sound file. I've seen the bake sound to graph function, and I tried to look in the source code how it works internally. This function is great, if we want to change some parameters based on the sound data, but it was not meant to be used for mesh creation. I thought, I must somehow get the raw data (integer value from the sample), then use python to create the mesh (I've already created meshes based on frequency and dampening factors).

I couldn't find the implementation of the bake sound function, only an user interface reference to this function. I think it calls some kind of C++ code (and I really don't want to mix my script with low level code).

I saw a post with blender game engine, which presented the usage of the aud API. This API can load any kind of format (even the sound channel from videos), but I couldn't figure it out how to get the sound data between some given frequencies, like in the bake sound function.

Can someone suggest an easy way to load the raw sound data between a given frequency range.

$\endgroup$
  • 1
    $\begingroup$ I can't find any related functions in the aud API that would be useful here.. If it's realtime audio to mesh you seek, perhaps openFrameworks is more appropriate. $\endgroup$ – zeffii May 4 '15 at 12:03
7
$\begingroup$

Say with a Plane object selected, you bake a sound file to its location keyframes.

>>> obj = bpy.data.objects['Plane']
>>> obj.animation_data.action.fcurves[0].sampled_points[0].co
Vector((1.0, 0.0))

>>> obj.animation_data.action.fcurves[0].sampled_points[1].co
Vector((2.0, 0.34159383177757263))

>>> obj.animation_data.action.fcurves[0].sampled_points[2].co
Vector((3.0, 0.8729846477508545))

>>> obj.animation_data.action.fcurves[0].sampled_points[3].co
Vector((4.0, 0.8701881766319275))

>>> obj.animation_data.action.fcurves[0].sampled_points[4].co
Vector((5.0, 0.6890017986297607))

This gives the points that the graph makes. And you can limit the frequencies used for the bake.

enter image description here

Then making a mesh is simple (relatively)

import bpy

rescalex = 20
rescalez = 0.2
obj = bpy.data.objects['Plane']
points = obj.animation_data.action.fcurves[0].sampled_points

# turn 2d points into 3d coordinates and rescale
verts = []
add_vert = verts.append
for idx, p in enumerate(points):
    z = p.co[1] / rescalez
    x = idx / rescalex
    add_vert((x, 0, z))

# connect sequential vertices
edges = [(i, i+1) for i in range(len(verts)-1)]

mesh = bpy.data.meshes.new("mesh_name")
mesh.from_pydata(verts, edges, faces=[])
mesh.update()

obj = bpy.data.objects.new("obj_name", mesh)

scene = bpy.context.scene
scene.objects.link(obj)

that gives this.

enter image description here

$\endgroup$
  • $\begingroup$ Wow, a good one, thank you for all the effort! My mesh will be way more complex, but I understood the concept. If you find another method, with less workaround, please feel free to post it! I will use this method for now. $\endgroup$ – danalizieors May 4 '15 at 11:52
  • 2
    $\begingroup$ I like to think of this as a 'work towards' not a 'work around' :) $\endgroup$ – zeffii May 4 '15 at 11:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.