4
$\begingroup$

For my addon I need to store large amounts of data in a .blend file (mostly floats, but other types are possible as well).

I tried using a normal CollectionProperty. Performance wise this works fine (with foreach_get and foreach_set). However some rough measures show that Blender stores roughly 128 bytes per float this way, which is quite a lot of overhead and therefor not really acceptable.

Is there a better solution to this problem?

Improve this question
$\endgroup$
2
  • $\begingroup$ Could a numpy array give less overhead and control of item size? $\endgroup$
    – sambler
    Mar 16, 2017 at 6:01
  • $\begingroup$ right, but I can't just store a numpy array in a .blend file afaik $\endgroup$
    – Jacques Lucke
    Mar 16, 2017 at 17:32

2 Answers 2

Reset to default
5
$\begingroup$

Collections are indeed not ideal for this use case.

You can store raw arrays in ID properties, which will native data types that don't take up excessive space in memory or on-disk. eg:

bpy.context.object["my_array"] = [1.0, 2.0, 3.0]

You can assign from an array type too:

import array
data = array.array('f', [1, 2, 3])
bpy.context.object["my_array"] = data

Internally float values are stored with double precision (8 bytes per value), as with Python's floats.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I have two little follow up question, tell me if I should ask them in a separate question: 1. Blender uses only doubles here, right? Because even if I give it a memory view with floats, it seems to take twice as much memory as expected. 2. Is it possible to access this data more quickly? Like with foreach_set/get? $\endgroup$
    – Jacques Lucke
    Mar 14, 2017 at 21:54
  • $\begingroup$ It seems like there always has to be a normal Python list as intermediate data type. Even when I have the data as raw array and Blender as well. This is not the case when I use a CollectionProperty (although there I have to create many items in a loop before I can use foreach_set). $\endgroup$
    – Jacques Lucke
    Mar 14, 2017 at 22:13
0
$\begingroup$

For the float / doubles issue:

Let's say X is your double and you only need 3 decimals of precision...

from math import floor
floor(X * 1000)

Then store your value as an int to conserve space.
Divide by 1000 when importing to get a double value again.

If you need more precision, add a few more 0's on there.
7 decimals of precision...

floor(X * 10000000)

Divide by 10000000 when importing to get a double value again.
You'd conserve space, but it would take time to do the conversion.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .