2
$\begingroup$

I have a depth map / point cloud, represented as a sequence of images (2D layers). They are layered spherically from a centre point, like an onion.

I'm trying to reconstruct the point cloud using geometry nodes (blender 3.6.5) but the frame socket for the Image Texture node won't take a field value, and seems to only let you use one image frame at once.

Is there no way to use multiple frame values, without creating a different Image Texture node for each image frame in the sequence? (there are 150 images in total)

Alternatively if I could iterate through the frames, and bake them into a single vector field or point cloud that would be good, but I don't know how to save geometry node information between the frame changing.

current setup

$\endgroup$

3 Answers 3

2
$\begingroup$

It all comes down to using a Repeat Zone. In my example I have an image sequence of 4 frames, each one a black and white image with different symbols. I want to have points distributed where the images are white, with each image a separate layer, let's say 0.5 m above the previous.

I start off with a plane object and plug the Group Input into a Distribute Points on Faces node with a Density of 5000. I also have an Image Texture node where I opened the image sequence (I use the Named Attribute "UVMap" to get the plane's default UVs as Vector). These are the prerequisites.

I add a Repeat Zone and plug the distributed points into the Geometry input. Then I set the Iterations to 4 because I have 4 frames in the sequence. I connect the Frame input of the Image Texture with the empty socket on the Repeat Zone input node, it gets named Frame automatically and I enter the value 1 (because my image sequence starts with 0001, not 0000). Then I connect the Frame also with a Math node set to Add to increase the value by 1 for each iteration, therefore the result gets connected with the Frame output of the Repeat Zone.

The initial Frame value is not so important, it could be 0 as well, it only matters if you use it directly in the Image Texture node or after adding 1 to it.

Now I add an Instance on Points node and plug the Geometry coming from the Repeat Zone input in. But here I'm doing it a little different than you: I'm not using the image texture for the Scale of the instances (this would result in instanced icospheres on all points of the plane (and using resources) even though only the ones with scale 1 are visible. Instead I'm plugging the texture into the Selection input so that only the white areas are instances of icospheres. Of course you can do it scaling if you want different sizes of spheres. But even then you could reduce the number of instances by using the texture selection with a Greater Than 0 node for example.

The most important thing now is to connect the result of the Instance on Points node to an empty socket on the Repeat Zone output - not the regular Geometry output! This creates a new input/output named Instances. Add a Join Geometry node inbetween. Into this you join the Instances from the Repeat Zone input node now.

The regular Geometry output of the Repeat Zone gets connected with the one from the input. Inbetween I place a Set Position node with an Offset of 0.5 on the Z axis, because each new layer should be 0.5 m above the previous.

nodetree

And this is the result with the four images:

points in sequence

Here is the blend file, but since Blender does not pack image sequences, they are in there as single images which might have messed up the functionality. Best would be to unpack the files and import a new sequence.

$\endgroup$
3
  • $\begingroup$ Unfortunately it doesn't work (for me in Blender 4.1.1) with packed images. Nothing displays until I switch from Image sequence to single image, and then... it's just a single image. So you need to save the images to disk it seems? $\endgroup$ Commented Jul 1 at 10:33
  • $\begingroup$ @MarkusvonBroady I use 4.1.1 as well. My problem is, I wanted to pack the images into the file as well, but Blender does not pack image sequences, so I opened them individually and packed them just so that they are in the file. But then they seem to lose the sequence property... best would be to unpack the images, clean up everything and import them fresh I guess. It seems Blender can load sequences only from an external source. $\endgroup$ Commented Jul 1 at 10:44
  • $\begingroup$ Yea too bad you can't upload a .zip to blend-exhchange. $\endgroup$ Commented Jul 1 at 10:47
0
$\begingroup$

there are different solutions possible (i think), one is convert your images to a movie file (which should be very easy) and then you can use it like this:

enter image description here

$\endgroup$
1
  • $\begingroup$ This works with an image sequence as well, no need for a movie file. It seems to me the problem here is getting all frames displayed at once to recreate the point cloud from them. $\endgroup$ Commented Jul 1 at 7:49
0
$\begingroup$

Take this .blend and render an animation:

To reproduce manually, enable Film→Transparent, select the default cube, Numpad 7, Numpad /, ✲ Ctrl⎇ AltNumpad 0, set Camera to Orthographic, scale 1, camera sensor fit vertical, and use the following geonode tree:

Now that you have your 3D texture, in a new project, in any material node tree add a new Image Texture node, open the first file of the image sequence, and switch the source mode from Single Image to Image Sequence. Unfortunately this seems to be a necessary step... Now delete this node go to Geometry Nodes and select this image in the Image node:

Alternatively, skip the above step with using a shader editor and when opening an image in geonodes, select multiple files [Thanks, Gordon].

Image Info: Frame Count doesn't work with image sequences :( You need to specify the number of frames manually.

I'm converting faces to points for convenience, because you don't want to sample the first pixel at $<0, 0>$ coordinates of the image, you want to sample it at $<{0.5 \over w}, {0.5 \over h}>$ because you're supposed to sample at the middle of a pixel (though I think for "Sample Nearest" mode, you're fine just sampling at 0.001 instead of 0...), and by using faces I just avoid a few math nodes.

The $z$ scale is arbitrary, though you could calculate it if you assumed the texture is a cube… Also because I start at frame 1, I also start at $z = 1$ instead of zero…

$\endgroup$
2
  • $\begingroup$ Instead of just writing my answer I should have looked from time to time to see you already made an answer that makes mine useless... but one question: what do you mean "switch the source mode from Single Image to Image Sequence. Unfortunately this seems to be a necessary step"? Just select all images of the sequence when you open them and it will automatically switch to Image Sequence. And this can be done in Geometry Nodes directly, no need to do it anywhere else and then delete it. The only thing that does not work in Geometry Nodes is the Cyclic setting from other editors. $\endgroup$ Commented Jul 1 at 10:36
  • $\begingroup$ You're right, I tried it at first but failed for some reason. But now I tried it again (even with removing the image and restarting Blender first to make sure it doesn't remember the setting) and it worked... As for your answer being useless, I don't think so. $\endgroup$ Commented Jul 1 at 10:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .