Future of AA Techniques
August 15, 2011Hopefully you all had a good time at Siggraph. For game programmers, the one course that you absolutely must check out is Filtering Approaches for Real-Time Anti-Aliasing. Make sure to check out the course webpage which will include slides and sample images. Here’s my general thoughts on it.
1. We don’t need perfection. And FXAA is pretty cool.
When looking at AA approaches, 720p with no AA has very bad aliasing artifacts. It’s really bad, especially if you have long lines and lots of contrast. I remember talking with Alex Fry (from EA) who made the point that you get a huge gain going from 1x to 2x MSAA. When you go from 2x to 4x, it looks better, but by a smaller margin. When you are an average distance away from a real screen, 2x gets you about 80% of the way there and 4x gets you that last 20%. 4x rotated-grid MSAA on the 360 looks very good. You’ll still see some aliasing in the worst cases but artifacts will be very rare. Obviously, there are many considerations here that I’m glossing over (like deferred/light prepass rendering).
I have the same feeling about FXAA. Eric Haines (from the RealTimeRendering blog) wrote the article FXAA Rules, OK? Guess what his opinion is? Generally, I’d agree with Eric’s points. FXAA is not perfect. You don’t get ideal edge fixing. But when I look at it in real games, it seems “good enough”. There are so many reasons why games don’t look photoreal, such as lighting models, faceting, texture filtering/compression, shadows, you name it. And FXAA is good enough that I’d rather focus my development time and GPU time on something else.
FXAA requires no extra memory, only one extra pass on the color buffer, is pretty fast, and is good enough. So I’m not really interested in a more expensive AA technique that requires extra passes but looks slightly better. Let’s say FXAA takes about 1.0ms of GPU time. Even if you have an algorithm that creates perfect AA in 4.0ms, I’d probably still take FXAA. For me, this result beats techniques that require multiple passes. Real levels in most games push the limit on tiny triangles so if you want to do any kind of extra pass or extra render target during your main/depth pass, you’re usually adding at least 1-2ms of time before you even get to the AA fixing part of your algorithm. I’d rather spend those GPU milliseconds fixing something else.
2. Perfect Detection is more important than Perfect Filtering
Post AA has two separate problems: Edge Detection and Edge Fixing. With FXAA as your baseline, I’d rather do a better job detecting edges than do a better job fixing the edges. So an algorithm that is more expensive but gets cleaner edges doesn’t really interest me, but I’m definitely interested in paying for better edge detection.
3. On next gen, 1080p displays make “perfect” AA less important.
I have no inside information on the next gen consoles, but I think it’s a fair bet that all games will be at 1080p. Wouldn’t bet on 60fps or 3D but you never know. With a 1080p display you have slightly smaller pixels. Aliasing is still a problem (you can clearly see aliasing artifacts, even at 1080p) but it will be less of an issue.
So once we get there, I don’t think we’ll get much real improvement from “Overkill” AA approaches. The standard for “Good enough” will actually go down.
4. Why the hell would you still use MSAA for any kind of deferred/light prepass renderer?
4x rotated grid MSAA is still the king if you have a one-pass forward renderer. Then again, how many games do that? Not many. During the God of War talk, I think Cedric and Tobias said that when you add it all up, you spend about 5ms on MSAA. The same was roughly true for Uncharted 2. For any renderer that is not completely forward some kind of Post AA seems like a big win.
Of course, when used the way that it’s meant to be used, 4x rotated grid MSAA is still awesome. If you could actually do 4x rotated grid MSAA with all your render passes for free, then I’d tell you to forget all this post AA nonsense and stick with MSAA. But given the cost Post AA seems like the clear winner.
5. Thank Alexander Reshetov for fixing our groupthink problem
One of the stupidest arguments I ever hear is “you shouldn’t do that because no one else is doing it”. I distinctly remember a phone interview that I had several years ago. This was when I was between things and eventually landed at Naughty Dog. He asked me what I thought was important and I made the case for Filmic Tonemapping. The interviewer’s response was that I must be crazy because the consoles has been out for several years, best practices had been established, and there was no way Filmic Tonemapping could be worthwhile because no one else was doing it. Suffice it to say, I didn’t end up on that project.
So that’s what Alexander Reshetov at Intel fixed. Most graphics programmers have at some point thought about doing AA in post. I personally had a few failed attempts in college. But common wisdom was that it couldn’t work. When he publishing his MLAA paper he essentially slapped some sense into us. That paper was incredibly important, not because it found some incredible technical innovation that we all build upon, but because it brought attention to the issue. And two years later we have the Anti-Aliasing course with tons of options.
We should really keep this lesson in mind. Even when consoles are mature, we should always be on the lookout for new ways of doing things. Alexander deserves credit for pushing MLAA much in the same way that Crytek deserves credit for convincing us that SSAO would work. Today, I couldn’t imagine making a game without SSAO and some kind of Post AA, and neither of those techniques were on anyone’s rader for the first few years of the current console cycle. That’s what innovation is: doing things that no one else is doing.
“4. Why the hell would you still use MSAA for any kind of deferred/light prepass renderer?”
It seems that for sub-pixel detail, the post effect methods actually increase the visibility of temporal aliasing. See the end of the video linked below, for ex. Seems to make a big difference in image stability between MSAA and FXAA. Look at the grass in the distance.
http://www.eurogamer.net/videos/far-cry-2-multi-sampling-vs-fxaa?size=hd
If you have large outdoor type of scenes, it would probably make sense to use deferred/light pre-pass, because it’s likely lots of lights will be visible at once. Maybe biting the bullet and using MSAA might actually be worth it, just to make things look more stable/less jittery. But maybe there are better solutions?
Thing is, even MSAA will not solve temporal aliasing perfectly, it will just make it less noticeable. I guess a special solution for these cases is probably needed. But with things like grass (as in the video) that might be difficult… there’s just so much of it, and so costs a ton of processing power no matter what.
Anyway, this is sort of an edge case, I guess. But it could be important for a fair amount of games set in outdoor environments. But for games set in urban environments and such or close-range environments (anything without much stuff that’s smaller than a pixel at distance), FXAA etc. will do the job just fine. I’m happy that this problem is so close to “solved”. Like you said, more processing power left for other cool stuff.
Having produced the FXAA comparison you talk about, it’s only fair to point out that it was a “worst case scenario” comparison – an ultimate stress test of sorts, based entirely on the inordinate amounts of sub-pixel vegetation in what is a demo loop. The console versions of Far Cry 2 had enough horsepower left over for multisample anti-aliasing (MSAA on 360, QAA on PS3 if memory serves).
In actual fact, I’ve found that SCEE’s MLAA actually seems to work best in organic-looking environments. If you check out Killzone 3, it’s my feeling that the industrial levels generate a lot more sub-pixel issues than the game’s organic outdoor stages. I see a similar thing in SHIFT 2 – the cityscapes seem to cause more of a challenge for MLAA than the traditional “outdoor” tracks.
Generally speaking, FXAA’s blurring works rather nicely on reducing the contrast of sub-pixel detail.
I guess it’s all going to come down to what budgets are available for AA and what techniques suit the overall game aesthetic best.
I’m probably being the naive outsider here, but I’ve never understood why edges needed to be detected. It’s not like you’re anti-aliasing a photograph, where you don’t know how the geometry in the scene is laid out and thus don’t know where the edges are. In video games you know the exact geometry in the scene as well as the viewpoint, which should give perfect information of where edges are and what shape they have.
I think the future of AA is in down-scaling, rather than detecting and blurring edges. If you can afford to render a scene at a resolution higher than 1080p and then down-scale it to 1080p, you’ll do away with all spatial and temporal aliasing, even at a sub-pixel level.
Good post.
Since #2 is basically what my talk was about, I should probably add that edge detection for MLAA-like systems really isn’t just about finding edges. Being able to break apart adjacent edges is really important as well, as soon as the image gets sufficiently busy.
Most people only look at the typical isolated long high-contrast edges when comparing AA methods, which is a pretty uncommon case in most game frames.
@Richard: KZ3 is an interesting case. I’d probably argue that my code does the best job for the junkyard levels, which are ridiculously hard to filter, as there is so much (high frequency, high specular) stuff in there. The organic levels are much easier to deal with (although still very complex). Of course, a player will never see the raw inputs, so this is hard to appreciate from the outside.
@Tomas: Yep, MSAA is still the king if you have lots of thin objects. Of course, this all goes haywire once lighting is involved. If you can afford proper 4x MSAA on all your lighting passes too then MSAA will look the best on that grass. Then again, I’m cringing when I think about how that grass would look with a 1x lighting buffer. (-:
@Richard: I can see that, although in my tests FXAA seems to be better at sparse vegetation because it tends to “overblur” it, which seems like the least-worst thing you can do. When you have dense and thin vegetation, there are no good options. Solution: Don’t have grass. Works for me!
@Lafie: Downscaling is the very very very distant future of AA. But with any consoles that we have now or in the next 20 years we’ll still be short on performance. It will be a long time before we can so Supersampling like the film guys.
@Tobias: Good point. There are cases in U3 where fixable edges meet each other. In those cases, I implemented the “give up” solution. The U3 technique doesn’t do so well with “extra” edges. That’s partially why U3 doesn’t use color or depth–too unclean.
@admin from your reply to Tobias, you imply that Uncharted 3 is using FXAA? Or perhaps some custom built similar technique? Or are you guys sticking with MLAA like most recent SCE games? If not, are you at liberty to tell why not?
If I can be frank, I have to tell you that I didn’t think AA in U3 multiplayer beta was all that great. There were parts that were sticking out like sore thumbs with how much aliasing there was. Then again, looking at the single player gameplay video with the cruise ship, things were looking much better there for some reason.
I used to think FXAA was great from reading everything about it, but then I saw those Eurogamer comparisons, and to be honest those soured me on this method quite a bit. It looks like the image uniformly loses its sharpness thanks to this method which is not the case with MLAA (or MSAA, of course). Yes, edges get antialiased (worse than with MLAA again) but it’s like there’s a blur haze over everything thanks to it, which I really didn’t like. I do like the idea of reducing contrast to alleviate subpixel problems, and I think that if MLAA could implement this, it would be by far a better method.
@poster: You misread it. U3 does not use FXAA.
Regarding the number of samples for MSAA: If you use a non-uniform grid (as most hardware does), it’s effectively a Monte-Carlo sampling approach. There’s lots of analysis of that which basically says the variance decreases as the square of the number of samples. That’s an unhappy curve to be on. To do twice as good as 2x MSAA, you’d expect to have to go to 8x MSAA. That’s a lot of extra visibility samples for only a little gain.
[...] Future of AA Techniques by John Hable FXAA Rules, OK? at Real-Time Rendering blog [...]
@Tomas @Richard I don’t know how the comparison video was created but I’m guessing it’s running the AA filter on the final image. If this is the case (I don’t see how it could be otherwise unless you’re able to hack the runtime somehow) it means the filter runs after all post process with all the potential blurring going on in there (bloom, tonemap, etc) which probably doesn’t help FXAA do it’s work.
@admin: If U3 doesn´t use FXAA, what AA-method do you use instead?
U3 uses a custom solution.