3
$\begingroup$

For my addon I need to store large amounts of data in a .blend file (mostly floats, but other types are possible as well).

I tried using a normal CollectionProperty. Performance wise this works fine (with foreach_get and foreach_set). However some rough measures show that Blender stores roughly 128 bytes per float this way, which is quite a lot of overhead and therefor not really acceptable.

Is there a better solution to this problem?

$\endgroup$
  • $\begingroup$ Could a numpy array give less overhead and control of item size? $\endgroup$ – sambler Mar 16 '17 at 6:01
  • $\begingroup$ right, but I can't just store a numpy array in a .blend file afaik $\endgroup$ – Jacques Lucke Mar 16 '17 at 17:32
4
$\begingroup$

Collections are indeed not ideal for this use case.

You can store raw arrays in ID properties, which will native data types that don't take up excessive space in memory or on-disk. eg:

bpy.context.object["my_array"] = [1.0, 2.0, 3.0]

You can assign from an array type too:

import array
data = array.array('f', [1, 2, 3])
bpy.context.object["my_array"] = data

Internally float values are stored with double precision (8 bytes per value), as with Python's floats.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I have two little follow up question, tell me if I should ask them in a separate question: 1. Blender uses only doubles here, right? Because even if I give it a memory view with floats, it seems to take twice as much memory as expected. 2. Is it possible to access this data more quickly? Like with foreach_set/get? $\endgroup$ – Jacques Lucke Mar 14 '17 at 21:54
  • $\begingroup$ It seems like there always has to be a normal Python list as intermediate data type. Even when I have the data as raw array and Blender as well. This is not the case when I use a CollectionProperty (although there I have to create many items in a loop before I can use foreach_set). $\endgroup$ – Jacques Lucke Mar 14 '17 at 22:13
0
$\begingroup$

For the float / doubles issue:

Let's say X is your double and you only need 3 decimals of precision...

from math import floor
floor(X * 1000)

Then store your value as an int to conserve space.
Divide by 1000 when importing to get a double value again.

If you need more precision, add a few more 0's on there.
7 decimals of precision...

floor(X * 10000000)

Divide by 10000000 when importing to get a double value again.
You'd conserve space, but it would take time to do the conversion.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.