Quick keyframe frame insertion for Shape Keys with `foreach_set` from data

Question

I would like to animate an objects' shape keys from a data set. Imagine a default cube that will get longer (side face vertices will move in +X direction) according to a data.

The date looks like this {'2000':1, '2001':2.5, '2003':3.4, ... '2020':11.9} As you can see, we have time versus accumulative values. In reality, the data set has 4000 data points.

In the function below, I can receive a new value, add a shape key for it and move my target face in the desired direction. Frame by frame, this works just fine. However the code execution gets really slow after about 100 keyframes insertion.

I am looking for a way to create this animation from the data set quickly. After research, I found about foreach_set function of Blender API. However I could not find a way to use it for my case.

Could anybody please explain me how to use foreach_set function for FAST shape key animation from a dataset?

Relevant questions I could find. However they were not helpful for my case:
Fast keyframe insertion
Editing fcurve.keyframe points in FAST mode?

Thanks in advance!

def resize_bar(obj_name, newValue = 0, active_face = 2, start_frame = 1, end_frame = 2):    
    # Get the active mesh
    obj = bpy.data.objects[obj_name]
    obj_mesh = bpy.data.objects[obj_name].data
    obj_mesh_matrix = bpy.data.objects[obj_name].matrix_world
    # get base shape key ready
    if start_frame == 1:
        basis_key = obj.shape_key_add(name="Basis", from_mix=False)
        basis_key.keyframe_insert('value', frame=start_frame)
    # Get a BMesh representation
    bm = bmesh.new()   # create an empty BMesh
    bm.from_mesh(obj_mesh)   # fill it in from a Mesh
    # get top face of mesh
    bm.faces.ensure_lookup_table()
    top_face = bm.faces[active_face]
    top_normal = bm.faces[active_face].normal
    # add shape key and insert a key frame before transformation
    new_shape_key = obj.shape_key_add(name=str(start_frame), from_mix=False)
    new_shape_key.value = 1.0
    new_shape_key.keyframe_insert(data_path='value', frame=start_frame)
    # apply transformations
    bmesh.ops.translate(bm, vec=newValue*top_normal, verts=top_face.verts)    
    bm.to_mesh(obj_mesh)
    bm.free()
    obj_mesh.update()
    # insert a new key frame after transformation
    new_shape_key.value = 0.0
    new_shape_key.keyframe_insert(data_path='value', frame=start_frame+end_frame)

Answer 1

Could be simplified to one shape key.

In the initial state of the cube, it is dimensionless (squashed down to a flat object). In the animation, it should be clear that the Cube starts growing from 'a flat surface'.

Also, how can I describe the scale in Z direction, when the values are accumulative? This gets complex already.

IMO you have totally overcomplicated it, by adding a shapekey for each data point. For this case even Using foreach methods isn't going to help particularly, ie for 4000 points of data creating as many shapekeys and keyframing 2 points per.

Am also of the opinion this could be reduced to simply keyframing object scale, a single shapekey, or solidify thickness to name but 3.

Using solidify thickness on faces defined by vert group "Group"

Shapekey emulating how scale works.

• Add a default cube
• Transform such that it is unit size, origin at base
• Copy the coords
• Scale it flat, make that the basis shape,
• Make the other shape key unit.

Ok now have a shapekey that if zero has zero height Basis, and if 1 unit height. Changing the range allows scaling outside the (0, 1) range. As illustrated, but appears to be hardcoded to only 10

There is really no need for the shapekeys, since the result is same as scaling in z axis, So could instead create no shapekeys and simply keyframe scale.

Test script. Creates the unit cube illustrated above

import bpy
from mathutils import Matrix
bpy.ops.mesh.primitive_cube_add()
ob = bpy.context.object
me = ob.data
me.transform(
        Matrix.Translation((0, 0, .5)) @        
        Matrix.Diagonal((.5, .5, .5, 1))
        )

coords = [None] * len(me.vertices) * 3
me.vertices.foreach_get("co", coords)
me.transform(
        Matrix.Diagonal((1, 1, 0, 1))
        )
basis = ob.shape_key_add(name="Basis")   
unit = ob.shape_key_add(name="Unit")
unit.data.foreach_set("co", coords)

Some animation basics

When you add a keyframe to a shapekey, it creates an Action if there is not already one, an fcurve for the data path being animated (if there already isn't one) and inserts a 2D point for (frame, value)

It is in populating the fcurves keyframe points where foreach_set is then used.

>>> C.object.data.shape_keys.animation_data.action
bpy.data.actions['Key.018Action']

>>> action = C.object.data.shape_keys.animation_data.action
>>> for fc in action.fcurves:
...     fc.data_path
...     
'key_blocks["Unit"].value'

Besides, to my understanding, foreach_set method creates and assigns all the keyfames from a data set at once. This is the efficient way when you have thousands of data points. But I just could not figure out a way to use it in shape keys

Yeah?, thanks for the tip.

Knowing where the action belongs and what the data path looks like, is the key to using the methods outlined in my answers to links posted

Fast keyframe insertion

Editing fcurve.keyframe points in FAST mode?

Example

Using data from question

• Created mesh with shapekey as above,
• Make the shapekey maximum (as big as it gets from data)
• Have offset data such that 2000 is frame 1, and taken one from value for example sake, to get zero at frame 1.
• Created an action for the "Unit" shapekey and populated it.

Test script.

import numpy as np
import bpy
from mathutils import Matrix

data = (
    (2000, 1),
    (2001, 2.5),    
    (2003, 3.4),    
   
    (2020, 11.9),    
    )

bpy.ops.mesh.primitive_cube_add()
ob = bpy.context.object
me = ob.data
    
d = np.array(data)
d -= (1999, 1)
frames, values = d.T

# max range of 10 ..
#unit.slider_max = values.max()


me.transform(
        Matrix.Translation((0, 0, .5)) @        
        Matrix.Diagonal((.5, .5, .5, 1))
        )
me.transform(
        Matrix.Diagonal((1, 1, values.max(), 1))
        )
        
coords = [None] * len(me.vertices) * 3
me.vertices.foreach_get("co", coords)
me.transform(
        Matrix.Diagonal((1, 1, 0, 1))
        )
basis = ob.shape_key_add(name="Basis")   
unit = ob.shape_key_add(name="Unit")
unit.data.foreach_set("co", coords)
# animation
me.shape_keys.animation_data_create()
action = bpy.data.actions.new("Grow")
action.id_root = 'KEY'
fc = action.fcurves.new(f'key_blocks["{unit.name}"].value')

values /= values.max()
fc.keyframe_points.add(count=len(frames))
fc.keyframe_points.foreach_set("co", d.ravel())

me.shape_keys.animation_data.action = action