One of the cooler things about Siggraph is that we all get to drink some Kool-Aid and yell at each other over the future of graphics. One set of course notes you should definitely check out is the Beyond Programmable Shading course. For me, the theme of Siggraph was “What would it take to actually solve this problem”. Instead of thinking about what hacks we can do to get slightly less-worse results, most talks seemed to focus on actually solving these things for good.
Problems for film-like rendering:
- What would it take to have no aliasing anywhere.
- What would it take to get sub-pixel triangles everywhere.
- What would it take to have perfect shadow resolution with no acne.
- What would it take to have perfect soft shadows with clean edges.
- What would it take to get true environment reflections.
- What would it take to get correct real-time GI.
- What would it take to render hair as unique strands.
- What would it take to get true lighting models.
- What would it take to get correct order-independent transparency.
- What would it take for photoreal water/fire/smoke.
- What would it take for proper caustics.
- What would it take for proper light scattering in media (i.e. smoke, thick liquids, etc).
Yes, there were some talks that focused on incremental improvement via hacks (like my talk), but the focus seemed to be on actually solving these long-term problems. Also, the consensus for most of these is that we can’t really solve these problems by just scaling up current hardware. In order to really tackle them, hardware has to change in some fundamental way. So the question becomes are we going to care about these for next-gen (PS4, Xbox 720), or are we going to wait for next-next-gen (PS5, Xbox 1080).
On the other end, I remember a discussion I had with Dave Cardwell (co-creator of Mudbox) back when I was working for EA. This would have been 2006-ish. Suppose we want to render a blue rubber ball with a pitch-black background. If you do that in a video-game engine, it has the classic “video-game” look. But if you render it in RenderMan, it looks real. It almost looks like you can touch it. I think we also agreed that it looked “juicy”. That’s a perspective that stuck with me. While I’d like to solve the sexy problems like rendering 20 planes of translucent geometry with correct depth-of-field, in games I still have yet to see something as simple as a concrete block look photoreal.
So what’s on my short list for the “next” things I want to get solved? Looking at Uncharted 2, here are the improvements that I’d like to make given more horsepower:
- Turn Everything On: There are lots of features that current hardware handles “well”, but we have to turn off because we just don’t have the cycles. Ideally, we’d like all surfaces to have a blend and all surfaces to have specular lighting, but lots of background geometry has those things cut for performance.
- Better Shading: We still can’t afford good lighting models. At a minimum, I’d like all materials to have two specular lobes and a fresnel term.
- More Lights: While we have support for deferred (light-prepass if you will) shadowed lights, we can very rarely afford them.
- More Triangles: I’d like more triangles, although getting more triangles on the main characters is less of a priority. For main characters (like Drake/Chloe/Elena) it would be nice to get one more subdivision, but they aren’t the bottleneck on visual quality. I’d like to bring the less-important characters and environments up to the same triangle density as the characters.
- Higher Quality Shadows: I don’t want anything crazy here. But if I could use the same shadows but afford more taps, I’d be a really happy guy.
- Full-Res Alpha: It would be really nice to render all particles at full resolution.
- Better Image Quality: This one is a little more aspirational, but it would be really nice to be able to do and afford 4x MSAA (rotated grid) with FP16 render targets. That would be great.
- More Foliage/Overdraw: For any foliage, we have to really cut down the cards and the lighting models because of the pixel-shader cost. I’d really like to allow the artists to have many more triangles/layers and the same lighting models as everything else.
- Better Post: I’d like to have cleaner edges, bokeh in the DOF, higher-quality motion blur, etc.
So that’s my wish-list. It’s not very exciting, but if I had a big chunk of extra horsepower, that’s what I’d spend it on first. Then, after I got those things, I would then want the more aspirational goals in the first list.
All that brings me back to the original question, what do we want in the next generation of consoles. Of course, everyone’s opinion will be different. But then, the question is how much more power will we actually have. If we don’t get enough raw horsepower for everything on that list, then I’d prefer a card that is basically the same but with more power. It’s only after we can afford those things that I’d be asking for a fundamental redesign to get us the more aspirational goals.
Compared to the 360/PS3, I think we could get most everything on my list with about a 10x increase in horsepower per pixel. Of course, I’m betting that the next generation will have 1080p as a standard and 3D as a standard too, each of which doubles the number of pixels we have to render per frame. So we would be about 4x behind in power-per-pixel before we even start. In my mind, if the next-gen GPUs are more than 40x more powerful than current hardware (or more), then they will need some kind of redesign to take advantage of all that power. But if they are less than 40x more powerful than the current crop, I’d be happy with a generic DX10/11 GPU, and I’ll wait for filmic rendering on PS5/Xbox 1080.