I'd like to use OSL to build a shader that gives a conditional (for mixing materials and such) on whether an object is inside another object and has a lower priority.
For instance, if I were to mix an arbitrary material with a transparent shader, I could make intersecting geometry disappear.
My end goal would be for the behavior to be (more or less) identical to using a boolean to cut out the intersecting region.
This could be useful, for instance, with complicated fluid setups, where several transparent objects are inside each other. (Say, a glass of water)
The Boolean modifier doesn't always work out, and in case of animation, it's not always feasible to manually clean up that modifier's results.
I have a script that certainly does... Something along those lines. But it has a few bugs. Here's the code, explaining my problems:
/*
Goal:
In case two meshes intersect, render only the one, that has a higher priority.
Effectively, I want meshes to appear, as if the one with higher priority carves
out a piece of the one with the lower priority like using a Bool modifier.
Problems:
1) For unknown reasons, priority checking seems to not work correctly.
It only even seems to do something with negative priorities, and then
all that matters is, that said priorities are negative.
I tried out other operations with mixed results.
2) I tried I, N, -I, and -N for the trace direction.
They all give different behavior, and I´m not sure what the right values are.
*/
shader Priority(
int mypriority = 0,
// An object´s priority
output int HasLowerPriority = 0
// Whether it has lower priority, than the object it intersects
)
{
int priority = mypriority;
// I did this to see if somehow mypriority isn't updated correctly.
//It didn't work.
setmessage("priority", priority);
// if hit by a ray, this message can be sent.
int DoTrace = trace(P, I);
// Shoot a ray. (Try out -I, N, and -N too).
// They all give different behaviors, none of which appear to be correct.
// It's supposed to check whether the material the ray came from
// has a higher priority than the current material, and whether
// it hit that other mesh on a front or back face.
// If it's back facing, it has come from inside the other object.
if (DoTrace) // If an object was hit
{
// Get that object`s priority and orientation.
int otherpriority = 0;
otherpriority = getmessage("trace", "priority", otherpriority);
int otherorientation = 0;
otherorientation = getattribute("trace", backfacing());
// Then check accordingly
// Instead of "<", try other comparisions to see how the behavior changes.
// To me, that behavior seems completely unexpected,
// with the outcome only depending on the own meshes priority
// sign and not on otherpriority. As if otherpriority is always 0,
// suggesting there is some error there. But the script compiles just fine.
if (priority < otherpriority && !otherorientation)
{
HasLowerPriority = 1;
}
//else // Pretty sure this is unnecessary too. I only added this for debug.
// {
//HasLowerPriority = 0;
//}
}
}
And here is an example blend file. Try changing the two objects' materials and the mentioned details in the script.
Also, for comparison, try activating the boolean modifier on the cube so see how it ought to look.
More or less: The Boolean modifier actually introduces what appears to be some Z-fighting. Ideally, this shader would also circumvent that.
