Get raw sound data in Blender using python

Question

I would like to use Blender to construct a mesh based on a sound file. I've seen the bake sound to graph function, and I tried to look in the source code how it works internally. This function is great, if we want to change some parameters based on the sound data, but it was not meant to be used for mesh creation. I thought, I must somehow get the raw data (integer value from the sample), then use python to create the mesh (I've already created meshes based on frequency and dampening factors).

I couldn't find the implementation of the bake sound function, only an user interface reference to this function. I think it calls some kind of C++ code (and I really don't want to mix my script with low level code).

I saw a post with blender game engine, which presented the usage of the aud API. This API can load any kind of format (even the sound channel from videos), but I couldn't figure it out how to get the sound data between some given frequencies, like in the bake sound function.

Can someone suggest an easy way to load the raw sound data between a given frequency range.

Answer 1

Say with a Plane object selected, you bake a sound file to its location keyframes.

>>> obj = bpy.data.objects['Plane']
>>> obj.animation_data.action.fcurves[0].sampled_points[0].co
Vector((1.0, 0.0))

>>> obj.animation_data.action.fcurves[0].sampled_points[1].co
Vector((2.0, 0.34159383177757263))

>>> obj.animation_data.action.fcurves[0].sampled_points[2].co
Vector((3.0, 0.8729846477508545))

>>> obj.animation_data.action.fcurves[0].sampled_points[3].co
Vector((4.0, 0.8701881766319275))

>>> obj.animation_data.action.fcurves[0].sampled_points[4].co
Vector((5.0, 0.6890017986297607))

This gives the points that the graph makes. And you can limit the frequencies used for the bake.

Then making a mesh is simple (relatively)

import bpy

rescalex = 20
rescalez = 0.2
obj = bpy.data.objects['Plane']
points = obj.animation_data.action.fcurves[0].sampled_points

# turn 2d points into 3d coordinates and rescale
verts = []
add_vert = verts.append
for idx, p in enumerate(points):
    z = p.co[1] / rescalez
    x = idx / rescalex
    add_vert((x, 0, z))

# connect sequential vertices
edges = [(i, i+1) for i in range(len(verts)-1)]

mesh = bpy.data.meshes.new("mesh_name")
mesh.from_pydata(verts, edges, faces=[])
mesh.update()

obj = bpy.data.objects.new("obj_name", mesh)

scene = bpy.context.scene
scene.objects.link(obj)

that gives this.