2
$\begingroup$

I am trying to make a lot of objects with a python script. I have read multiple posts about low level vs high level programming and the slow-down due to scene-update of all objects after the creation of every one, and have based my second loop on them ('low level' programming).

The problem is that while the low level loop runs much faster, especially with the more objects added (I've tried up to 26,000), the scene never updates after creation. The windows 'waiting wheel' just spins and spins.

First, here is my data. I've shortened it to just be the first 1,000 rows. The view here is as a Pandas DataFrame, but I've converted it to a numpy array for speed before use.

        id  xcentre  ycentre  zcentre  xlength  ylength  zlength
0   783782  10236.5  14098.0    596.5      3.0      4.0      1.0
1   783783  10245.5  14091.0    596.5      5.0      2.0      1.0
2   783784  10247.5  14091.5    595.0      1.0      1.0      2.0
3   783785  10248.0  14093.0    594.5      4.0      2.0      1.0
4   783786  10245.5  14093.5    594.5      1.0      1.0      1.0
5   783787  10248.5  14087.5    595.5      1.0      1.0      1.0
6   783817  10240.5  14092.5    596.5      1.0      1.0      1.0
7   783818  10232.5  14099.5    596.5      1.0      1.0      1.0
8   783819  10234.5  14097.5    596.5      1.0      1.0      1.0
9   783820  10237.0  14095.5    596.5      2.0      1.0      1.0
10  783821  10238.5  14095.5    596.5      1.0      3.0      1.0
11  783822  10244.5  14089.5    596.5      1.0      1.0      1.0
12  783823  10246.5  14087.5    596.5      1.0      1.0      1.0
13  783833  10245.5  14091.5    595.5      3.0      1.0      1.0
14  783834  10234.0  14099.0    596.5      2.0      2.0      1.0
15  783835  10242.0  14092.0    596.5      2.0      2.0      1.0
16  783836  10246.0  14089.0    596.5      2.0      2.0      1.0
17  783837  10249.5  14082.0    596.5      1.0      4.0      1.0
18  783838  10247.0  14090.5    595.5      2.0      1.0      1.0
19  783839  10246.5  14096.0    596.0      7.0      8.0      2.0

Here is the first script ('high level'):

#shorten the DataFrame and convert to numpy matrix
cut1 = cut[0:1000]
cut1 = cut1.as_matrix() 

#Make the material for the cubes
purple = bpy.data.materials.new("Purple")
purple.diffuse_color = (1.0, 0, 1.0)

#Make the material for the labels
label = bpy.data.materials.new("LABEL")
label.diffuse_color = (1.0,1.0,1.0)

#Make the label and parent
bpy.ops.object.text_add(view_align=False, location=(cut1[:,1].mean(), cut1[:,2].mean(),cut1[:,3].max()+500), rotation=(0,4.72,0))
txt = bpy.context.object
txt.name = "Cubes"
txt.data.body = "Cubes"
txt.dimensions = 200, 100, 0
txt.data.materials.append(label)

start = time.time()
for i in range(len(cut1)):

    bpy.ops.mesh.primitive_cube_add(location=(cut1[i,1], cut1[i,2], cut1[i,3]))
    block = bpy.context.object
    block.scale = (cut1[i,4]/2, cut1[i,5]/2, cut1[i,6]/2)
    block.data.materials.append(purple)
    block.parent = txt
    block.matrix_parent_inverse = txt.matrix_world.inverted()



end = time.time()

print("elapsed time is: {}".format(start-end))  

This runs in 12 seconds, and the results are perfect, where the blocks are exactly as I want. The scene interacts (zoom, rotate, etc) very fast.

The problem here is that after about 3000 objects, the script slows down dramatically. By 10,000 objects it takes well over a second per object. To make the 26,000 objects that I want would take days.

Here is the second script ('low level'):

#shorten the DataFrame and convert to numpy matrix
cut1 = cut[0:1000]
cut1 = cut1.as_matrix() 

#Make the material for the cubes
purple = bpy.data.materials.new("Purple")
purple.diffuse_color = (1.0, 0, 1.0)
#Make the material for the labels
label = bpy.data.materials.new("LABEL")
label.diffuse_color = (1.0,1.0,1.0)

#Make the label and parent
bpy.ops.object.text_add(view_align=False, location=(cut1[:,1].mean(), cut1[:,2].mean(),cut1[:,3].max()+500), rotation=(0,4.72,0))
txt = bpy.context.object
txt.name = "Cubes"
txt.data.body = "Cubes"
txt.dimensions = 200, 100, 0
txt.data.materials.append(label)

#make the first cube to be the base for the obs array
bpy.ops.mesh.primitive_cube_add(location=(0,0,0))
ob = bpy.context.object
obs = []
sce = bpy.context.scene
start = time.time()

for i in range(len(cut1)):

    copy = ob.copy()
    copy.location += Vector((cut1[i,1], cut1[i,2], cut1[i,3]))
    copy.scale += Vector((cut1[i,4]/2, cut1[i,5]/2, cut1[i,6]/2))
    copy.data.materials.append(purple)
    copy.data = copy.data.copy()
    copy.hide=True    
    copy.parent = txt
    copy.matrix_parent_inverse = txt.matrix_world.inverted()    
    obs.append(copy)


for ob in obs:
    sce.objects.link(ob)

end = time.time()
print('elapsed time is: {}'.format(end-start))  

sce.update() # don't place this in either of the above loops!    

end2 = time.time()
print("scene update took: {}".format(end2-end))

This is the result:

enter image description here

My question is this: Is there some activity that goes on when doing a bpy.context.scene.objects.link() that slows things down? I have waited until after the program is run, and all of the objects are created, but the program is still very slow. It takes minutes to interact with it (rotate, zoom, un-hide, etc). You'll notice that the objects are all created with hide=true, so its not trying to display them in the 3D window.

I'm at a loss here, and any help would be appreciated!

Thanks!

EDIT:

The problem is with moving the new cubes from the data-block to the scene. I can run the 'low level' script with any number of cubes up to the 26,000 and it runs very fast and they are all viewable within the data-blocks.

enter image description here

Can anyone help me with a fast way to either duplicate these to the scene or link them to the scene? I've tried both this:

for ob in obs:
    sce.objects.link(ob)

and this:

for j in range(len(obs)):
    sce.objects.link(obs[j])
    bpy.ops.object.duplicate()
    sce.objects.unlink(obs[j])

And they are both very very slow.

$\endgroup$
  • $\begingroup$ Looks like you have messed up your addon folders, for example the osm_parser.py should be inside the blender_geo folder in your addons. Do you need a separate mesh for each mesh copy? (copy.data = copy.data.copy()) if not remove this. I can't see a reason not to in code posted. Then you will have 26000 objs 1 mesh instead of 26000 of both. For monster data might look at using a modal timer loop or similar to split into smaller chunks and be able to kill it with <esc> $\endgroup$ – batFINGER Sep 30 '16 at 12:24
  • $\begingroup$ Related #1, #2, #3 $\endgroup$ – pink vertex Sep 30 '16 at 12:45
  • $\begingroup$ @batFINGER- Thanks for the advice. I've gone and cleaned up some of the mess in my console, but the addons I've written need the 'bl_info' text block to get rid of the rest I believe. I did remove the copy.data = copy.data.copy() just now and re-ran it with no change. $\endgroup$ – joswhite Sep 30 '16 at 12:54
  • $\begingroup$ @pink vertex- thanks for those links. I'll give the dupliverts a try here now. Question for you that I've been searching for- with dupliverts can unique properties like diffuse_color and scale be applied to each piece? $\endgroup$ – joswhite Sep 30 '16 at 12:57
1
$\begingroup$

Ok - So after messing around with this all day, I came across batFINGER's answer to this persons question (link below- While it wasn't the chosen answer, I was able to modify it to my needs). I don't yet know why it works so fast, but it just does. So thanks batFINGER! The 'collar' here is the same data set as 'cut1' in the question above. This will generate 10,000 objects in just a few seconds.

Creating many primitive objects quickly with Python

Here is the code with my modifications:

import bpy
import pandas as pd
import numpy as np

context = bpy.context
scene = context.scene

collar = collar.as_matrix()

def base_object():
    #Creates base cylinder
    bpy.ops.mesh.primitive_cube_add()
    return

def clear_scene(scene):
    #Clears the current scene
    for obj in scene.objects:
        scene.objects.unlink(obj)
        bpy.data.objects.remove(obj)
    return    

def object_creation(scene, clouds, collar):
    bpy.ops.object.text_add(view_align=False, location=(0,0,0)) 
    txt = bpy.context.object
    base_object()
    obj = scene.objects.active
    obj.parent = txt
    obj.matrix_parent_inverse = txt.matrix_world.inverted()
    mesh = obj.data
    purple = bpy.data.materials.new("Purple")
    purple.diffuse_color = (1.0, 0, 1.0)

    for attempts in range(clouds):
        x = collar[attempts,1]
        y = collar[attempts,2]
        z = collar[attempts,3]
        obj.location = (x, y, z)
        obj.scale = (collar[attempts,4]/2, collar[attempts,5]/2, collar[attempts,6]/2)                  
        obj.show_transparent = True
        obj.active_material = purple
        obj = obj.copy()
        obj.data = mesh.copy() # a new mesh for each object.
        scene.objects.link(obj) # link it to the scene

    return

clear_scene(scene)
object_creation(scene, 10000, collar)
$\endgroup$
  • $\begingroup$ How could you set up pandas in python? I have been trying per hours and I'm just getting errors lol $\endgroup$ – user52870 Feb 24 '18 at 2:22
  • $\begingroup$ That's a good question- it took me a minute too. Basically you have to copy whichever package you want to use into the site-packages folder. If you're using blender 2.79 you should be using python 3.5. If you have anaconda installed with python 3.5 just copy the c:/anaconda/Lib/site-packages/pandas folder to the c:/blender/2.79/python/Lib/site-packages folder and you should be able to import it. Those paths may not be exact- I'm not in front of my computer $\endgroup$ – joswhite Feb 25 '18 at 6:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.