Check out these pool balls from the PSN game Hustle Kings. They do a pretty cool trick where the balls where they render a quad in front of where the ball should be. Then they essentially do a ray-sphere intersection in the pixel shader. It’s a pretty cool trick and looks pretty good. You can find a more detailed explanation at the VooFoo Studios website.
In the future, I’m pretty bullish on techniques that so some kind of mini raytracing inside a pixel shader. That’s a longer post. But the question becomes what to do with early z-cull for pixel shaders that change the depth. Here is a quick picture showing the problem.
Suppose that we render the green shape first. Now we are going to render the red sphere. We can do that by rendering a cube that bounds it (in blue) and then using a pixel shader that does a raytrace with the sphere. If we want to have spheres intersect other objects, we need that pixel shader to change the output depth.
This causes a problem for early z-cull. Since the GPU does not know if the new depth will be closer or farther than the original depth, the GPU can’t discard any pixels with early z-cull.
We could solve this with a hypothetical extension that I’m tentatively calling “Clamped Depth”. Basically, we could have a state which says that when a pixel shader writes depth, the output depth will be forced to be at least as far as the original sample. That way, we could still perform early z-cull on pixel shaders that write depth.
For the record, I am definitely not the only person who has had this idea. And I haven’t payed enough attention to DX11 changes so for all I know that feature may already be in there. Also, I think I remember a poster at Siggraph 2007 with the exact same idea.
So far, it hasn’t been an issue since there aren’t that many cool techniques that require depth-changing shaders. But I could definitely see them becoming viable in the next 5-10 years, so I really hope that we have that option on the PS4/Xbox 720. And of course, the devs at VooFoo Studios have shown that for certain cases, it is viable today.