We are in the need to create thousands of objects via python (sometimes up to 100.000) in the viewport, we managed to do so using collection instances inside other collection instances (with this method we even arrived up to 300k with almost no lag in the viewport). Our problem right now is how to reduce the time that blender needs to create all of this emptys in the scene, we have tried using dupli verts, copying instead of creating the emptys and even a solution involving particle systems but it seems that none of this solutions make much difference when it comes to that number of individual instances.
Right now it takes around 5 min to create 70k objects, we are in the need to reduce that time.
We would like to find a way to accelerate this process as much as possible but we are out of ideas and the python API doesn't give solutions for this, so I was wondering if getting into C could help, but as our developers need time to understand such low level language I wanted to know in advance if that is the right solution.
So, in short my question would be, how can we create that many objects as fast as possible?