How could I setup a « video feedback » effect ? (camera filming its own output)

Question

Someone asked me about a great challenge here: is it possible to create a « video feedback » effect in Blender; like this one:

My scene setup:

• camera A pointed on a human.
• Behind the human, there is a screen; which « streams » what camera A captures.
• So this would create a video feedback effect.
• That whole scene would be captured by camera B on a larger shot; and this would be my final output.

My first approach was to check if there is a « camera output » node (for the screen material) to feed the screen material, but there isn’t.

I searched the web and found this:

• Create texture dynamically from camera view… possible? (Blender Artists)
• Use file output node for video feedback - can it be done (Blender Artists)
• Render to texture (Right-Click Select)
• Video feedback loop - animation test - Blender (seems someone managed to do it in 2015)

But no real solution. Someone wrote a python script, but it has been done in 2015 for the Game Engine, so i’m not sure I could use it here : Render To Texture - Game Engine.

Another solution I see it to do it manually, but that is quite long/boring to do! I can't imagine rendering an animation manually like this...

• screen texture: (none) >>> render cameraA to cameraA-001.jpeg
• screen texture: cameraA-001.jpeg >>> render cameraA-002.jpeg
• screen texture: cameraA-002.jpeg >>> render cameraA-003.jpeg
• screen texture: cameraA-003.jpeg >>> render cameraA-004.jpeg
• screen texture: cameraA-004.jpeg >>> render cameraA-005.jpeg
• Finally, render the camera B view. I would have a loop of 5 « iterations » in the screen.

Back plane is a "screen", bottom and right planes are mirrors.

So, is there a way to script this or is there another way to achieve it ?

Thanks a lot !

Answer 1

You mentioned doing it with multiple renders. This can be optimized beyond what you imagined (but still requires 3 images, in 2 separate renders, for every frame).

The first thing we'll do is set up an AOV. For our video monitor, we'll output UV in this AOV. For everything else, we'll output RGB 0,0,1.0. This will let us distinguish between the screen and everything else in the render, and will get proper occlusion of the screen. Let's render both this AOV and our actual rendered output (minus the screen).

Now let's set up our screen material in a little more detail. We haven't worried about it yet, because we'll be replacing it:

Make sure you understand what we're doing here. We're using the screen's UV to look up both renders. If and only if the AOV render is blue, we mix in the color from the render. This is our first pass. Now let's make a node group out of it:

I've added a few details to this node group, so it needs some explanation. I've given it a vector output, which is the RG lookup from our AOV render. I've created a "stop" input, which mixes us back into our original color if it's set. And I've created a "stop" output, which is set if either our stop input is true or if our AOV lookup reads Blue. Notice that the inputs and outputs and now perfectly parallel. That let's us do this:

By chaining the node groups together, we can dive as deep into the render as we want, without making any new renders. Here, I'm doing 4 iterations. Unfortunately, there is no right number of iterations, and you can't do infinite iterations, but at least it is very simple here to add more iterations as needed.

Note that there isn't any kind of antialising going on. I'd recommend that you render your AOV at a higher resolution than your final render to compensate. This should be easy enough because you can render your AOV in Eevee, with 1 sample, so the AOV is a very fast render.