# How to prevent the decimate modifier from working on vertices outside the vertex group (with python)?

I am using the 'decimate modifier' to decimate my mesh. The modifier should only work on a given vertex group. But sometimes (when the ratio is getting lower) the modifier is working on vertices outside of the group. But I really don't want that! How can I disable this behaviour? If the modifier can't decimate more than it should simply stop.

Can I somehow find out if the modifier is working outside of the group and simply set the ratio to a higher value (with python of course).

Thanks!

My solution was terrible slow. So I got the blender sources and changed the modifier. It was fairly easy!

I added following code on line 311 (https://git.blender.org/gitweb/gitweb.cgi/blender.git/blob/master:/source/blender/bmesh/tools/bmesh_decimate_collapse.c#l311)

// Only work on vertex groups
// vweights is false when no vertex groups are used
if (vweights) {
float weight_v1 = roundf(vweights[BM_elem_index_get(e->v1)]);
float weight_v2 = roundf(vweights[BM_elem_index_get(e->v2)]);

if (weight_v1 != 0.0f && weight_v2 != 0.0f)
{
eheap_table[BM_elem_index_get(e)] = BLI_heap_insert(eheap, cost, e);
return;
}
else
return;
}


As you can see I used the weight to find out if the vertex is really in the group. I need to do that because in deform.c (https://git.blender.org/gitweb/gitweb.cgi/blender.git/blob/HEAD:/source/blender/blenkernel/intern/deform.c#l742) the weight is set to 0.0 even if the vertex isn't inside the group. If you need this you need to add vertices to a group and set any value between 0.01 and 1 as vertex weight.

I wrote a simple (stupid) decimate algorithm which only works on a vertex group or the whole mesh. It is only collapsing edges and the angles are totally ignored. It is not trying to preserve the shape really well.

Here is the code, it is not optimized (I am pretty new to python) and I know I could optimze it here and there (all the bpy.ops api calls could have faster api methods) but for now I am happy. Nevertheless: Here is the code!

If you have an idea how the algorithm would be faster, spit it out. Thanks!

# how_often: how often should a edge be collapsed
# group_name: if it is None it will work on the whole mesh or on a given group.
def decimate_mesh(self, how_often=0, group_name=None):
import bpy
ob = bpy.context.active_object
mesh = ob.data

# go to edit mode and deselect everything .. just to be sure
bpy.ops.object.mode_set(mode = 'EDIT')
bpy.ops.mesh.select_all(action='DESELECT')
bpy.ops.object.mode_set(mode = 'OBJECT')
how_often_decimate = how_often

if group_name is None:
# decimate all edges
while how_often_decimate != 0:
edge_length = dict()
# calculate edge lengths
for edge in mesh.edges:
edge_length[(mesh.vertices[edge.vertices[0]].co-mesh.vertices[edge.vertices[1]].co).length] = edge
# pick the shortest edge, collapse it and make all polygons concave
min_key = min(edge_length.keys())
edge_to_collapse = edge_length[min_key]
edge_to_collapse.select = True
bpy.ops.object.mode_set(mode = 'EDIT')
bpy.ops.mesh.merge(type='COLLAPSE')

bpy.ops.mesh.select_all(action='DESELECT')
bpy.ops.mesh.select_all(action='SELECT')
bpy.ops.mesh.vert_connect_concave()
bpy.ops.mesh.select_all(action='DESELECT')
bpy.ops.object.mode_set(mode = 'OBJECT')
how_often_decimate -= 1
else:
# decimate edges where both vertices (of the edge) are part of the vertes group
group_index = ob.vertex_groups[group_name].index
while how_often_decimate != 0:
vertices_which_are_assigned_to_group = set()
edges_which_can_be_decimated = set()
edge_length = dict()
# I am unsure if I really need to do this every time. I know that the indexes could be altered by blender
# but I need to know which vertices are inside the group. So yeah...
for vert in mesh.vertices:
for vert_group in vert.groups:
if vert_group.group == group_index:

for edge in mesh.edges:
vertex_0_index = edge.vertices[0]
vertex_1_index = edge.vertices[1]
if vertex_0_index in vertices_which_are_assigned_to_group and vertex_1_index in vertices_which_are_assigned_to_group:

if len(edges_which_can_be_decimated) == 0:
print("No edges anymore!")
break
for edge in edges_which_can_be_decimated:
edge_length[(mesh.vertices[edge.vertices[0]].co-mesh.vertices[edge.vertices[1]].co).length] = edge

min_key = min(edge_length.keys())
edge_to_collapse = edge_length[min_key]
edge_to_collapse.select = True
bpy.ops.object.mode_set(mode = 'EDIT')
bpy.ops.mesh.merge(type='COLLAPSE')
bpy.ops.mesh.select_all(action='DESELECT')
bpy.ops.mesh.select_all(action='SELECT')
bpy.ops.mesh.vert_connect_concave()
bpy.ops.mesh.select_all(action='DESELECT')
bpy.ops.object.mode_set(mode = 'OBJECT')
how_often_decimate -= 1