How to cast/teleport a ray in a material for a squinching illusion?

Question

In short I'm wondering if there's a way to not only refract a ray, but also moving it, or somehow cast another ray from a different position in a material shader for an illusion I'm working on.

This situation is pretty complex but I'll do my best to explain what I'm doing.

I'm working on methods to implement squinching, a way to trick the viewer into thinking that a curved (projection) screen, isn't a screen, but extends into a larger 3D world (Like with the Trompe L'oeil illusion).

You can read more about what squinching exactly is in this expired patent: https://patents.google.com/patent/US6462769B1/en

I already got most of it working. I set up a simple shader for the screen, that refracts the light in the opposite direction from the viewer's position (the audience position)

The current result is already working really well, however one situation isn't covered yet.

The render camera (Marked with a P) shoots a ray (blue line), once it hits the screen it refracts in the right direction to show the squinched perspective, which will look normal from the other audience position (The other camera in the image).

However I need to somehow make the green line part of the blue ray so that I can project anything that is in front of the screen, onto the screen itself, the same way anything behind it is shown.

In this image I mixed the refraction with a glossy shader to create a representation of what I'm hoping to achieve, a pretty much need that glossy shader's ray in the opposite direction

• I'm using cycles
• In the end I need to be able to export a flat image from the curved screen (e.g. by baking), so ideally I'd be able to handle this with a material
• In addition to baking, I also need the option to render the screen from the position of a projector (In the example image marked "P")
• In this example the screen is curved, but it may have any shape, so a distorted camera frustrum isn't an option.

What I've done so far:

• Tried looking for a way to somehow "teleport" the ray back. But it looks like I can only influence its direction.
• In the past I've succesfully done this with multiple render passes, where I first render the scene from the position of the audience, and then project it onto the screen, to then capture that from whatever angle I need. This technique however loses a lot of quality as some parts end up being more stretched then others.
• Looked into making a custom node using OSL, however the "ray" function doesn't seem to be able to sampling colors.

My thinking is that I should somehow be able send a ray from the audience camera in the direction of the hit position once the camera "P"s ray hits the material, and use the color from that ray to use on the material. But I've been unable to find a way to do something like that.

Any help would be greatly appreciated!

Answer 1

In the past I've succesfully done this with multiple render passes, where I first render the scene from the position of the audience, and then project it onto the screen, to then capture that from whatever angle I need.

This technique seems to be the most viable to me. If you want to project what the audience is seeing onto the surface, this approach does exactly that - and the simplest solution is often the best.

This technique however loses a lot of quality as some parts end up being more stretched then others.

You can increase the resolution until it satisfies your needs. Yes, you might "waste" some pixels, but that doesn't seem like a critical issue.

A way to somehow "teleport" the ray back. But it looks like I can only influence its direction.

Yeah, I don't think Cycles is designed for custom ray transformations such as the one you describe. This is likely not a common use case, and there is a flexibility-performance trade-off when designing renderers.

Edit: alternative suggestion

Since you have already gotten the projection to work for objects behind the "screen", I can suggest scaling the screen down towards the audience - so that everything, including the cube, is behind the screen. Then you would also need to adjust the camera, and the projection would appear as you intend.

If you need to render the rest of the scene as well, you could combine that render with the rest of the scene during compositing.