Visual Acuity, Vernier Acuity, Anti-Aliasing, and You
February 6, 2012Well, it’s been a long time since I’ve posted here. My time has been spent focusing on the 19lights.com blog and making HDR video a reality. But somehow I got into twitter fight last evening during the Super Bowl. And as it turns out, it’s not easy to explain the subtleties of Visual Acuity and Vernier Acuity and how it relates to anti-aliasing in hypothetical next gen consoles in 140 characters. I don’t actually have any inside information about next gen consoles, but let’s assume that the next gen games will output at 1080p and have about as much processing power as a top end GPU.
The argument essentially boils down to: Is FXAA enough, which costs <1ms, or do you want (or NEED) better AA. The first thing that you have to understand is the difference between Visual Acuity and Vernier Acuity which unfortunately is one of those things that no one teaches you. So let’s try.
You have all heard about visual acuity before. If you need a refresher you can talk to your good friend Wikipedia: http://en.wikipedia.org/wiki/Visual_acuity. Essentially, as things get really small, the human eye has trouble distinguishing them (duh). So if text is too small and too far away then you can’t read it.
What you probably don’t know about is Vernier Acuity. Wikipedia is of course a great resource: en.wikipedia.org/wiki/Vernier_scale. Vernier acuity should make sense to anyone who has worked in games and seen aliasing. I’ve talked about this issue before when I’ve argued that Apple is lying to you when they call the iPhone 4 a “Retina Display” as well as in an early post about the difference between 720p and 1080p. But it should make intuitive sense you. The human eye has an incredible ability to tell if two lines aren’t exactly aligned with each other. That’s how a vernier caliper works. You can think of vernier acuity as the official term for our eye’s ability to see aliasing.
But when it comes to choosing what AA technique you need for a given resolution, Visual Acuity vs Vernier Acuity is incredibly important. There are three cases:
1: Visual Acuity < Vernier Acuity < Resolution
If you are rendering at a resolution that is finer than Vernier Acuity, then AA is worthless (except for crazy extreme cases which I’ll get to in a moment). If you rendered to a screen with 10,000 DPI there would be no need for AA because the resolution is so fine that your eye can’t make out the aliased edges.
2: Resolution < Visual Acuity < Vernier Acuity
On the other hand, if you are rendering at a resolution coarser then Visual Acuity, then your eye can clearly see blurriness in your original image. In this case any AA technique that increases the blurriness of your image will be easy to see. Also, in this situation techniques like MSAA have a definite quality advantage over post AA techniques (like FXAA) but are usually more expensive.
3: Visual Acuity < Resolution < Vernier Acuity
If your resolution is in between Visual Acuity and Vernier Acuity then you are in a strange land. You’re eye can’t pick out individual details but it can tell if there are artifacts like aliasing. In these cases, FXAA should be good enough for 99% of your cases. Sure, a more expensive option like MSAA might look a teeny bit better. But if you’re AA technique reduces sharpness a little bit around the edges then your eye can’t tell because the resolution is beyond your Visual Acuity.
My belief is that on next-gen consoles, most users will be in category 3. Most people who actually work in games (myself included when I was at Naughty Dog) think that aliasing is much more of an issue than it actually is. Partially, it’s because we are trained to look for it. But also the median viewer is looking at the screen with much less resolution than we do. When I was at Naughty Dog I sat about 4 feet away from a 32-inch screen. But the average user sits about 10 feet away. And then if we change resolution from 720p to 1080p aliasing becomes even less of an issue for the average user.
So that’s about where I stand. Once the next gen consoles come around and we move to 1080p then there is a very negligible difference between what a cheap solution like FXAA gives you and a theoretical “perfect” solution. The original comment that started this discussion was about using tiling to get MSAA. And I find it pretty hard to believe that the quality difference of MSAA (with TILING!) would justify the cost on next gen.
Side Note #1
There are always exceptions. If you are making Flower 2 and you are willing to dedicate 60% of your budget to rendering grass than MSAA might be worth it. But for the “standard” games like Skyrim, Halo, Modern Warfare, Uncharted, etc. I see FXAA as the best solution.
Side Note #2
For me, the #1 problem in video games is Shading and the #2 problem is Lighting. Everything has that “weird video game look” to it. If you compare games to other games, you would think that games look pretty good. But when you compare games to either VFX or real life on the same monitor it breaks down. Every time a commercial on TV comes up showing in-game footage I always cringe a little bit (even on games that I worked on). We can’t even make the easy surfaces like concrete/wood/tile look right. And we are REALLY far away from the harder surface like skin/cloth/hair/marble. That’s where I would rather put cycles if I was planning a next-gen game budget.
Side Note #3
So what about thin objects that cause flickering because they are so small? Simple: Don’t have them. Instead use alpha for things like one million blades of grass. Don’t use individual triangles. The hardware isn’t really effective with lots of 1 pixel triangles so you’re better off using alpha cards anyways. For thin object that are harder to replace I think there are more interesting options for blending them with geometry shaders if you really need to. And if your game is a true outlier case (like you absolutely must render a field) then see Side Note #1.
Side Note #4
Still think I’m completely wrong? This is easy enough to test. Just load up one of Post AA comparison samples onto a PC, hook it up to a 42 inch display at 1080p, and invite random people off the street to sit 12 feet away. Then switch between no MSAA, 4xMSAA, and FXAA randomly and ask them to rate the quality. That’s the real way to settle this issue, although the results would be a bit biased because people are more likely to notice aliasing if they are asked to look for it than if they were just playing at home.
Side Note #5
Users and video game reviewers know virtually nothing about why an image looks good or bad. That’s why you always here them say “texture resolution”. If a game looks good because it has good lighting, nice animation, and high-quality lighting models most forum posters will say “Wow, this game has such detailed textures!”. But when a game has bad animation and gamma space lighting they will say “Those textures look low-res!”. I don’t remember hearing a game reviewer say “Game A has a great anisotropic reflectance model on it’s brushed metal shaders!”. Instead, they’ll say “Game A has really crisp textures” as if the developer magically found an extra 100 megs of texture RAM inside the console.
The same thing tends to happen with anti-aliasing. Often times you’ll see two games, let’s call them Game A and Game B, where A looks much better than B. The reviews will say “Game A has detailed texture and clean edges” whereas “Game B has low-res textures and lots of jaggies” even when they both use the exact same technique. Game reviewers don’t know what to say other than texture detail and jaggies so they use those terms to justify what they already believe.
I imagined that there was a misunderstanding between us here (twitter really is an inappropriate place for these sorts of discussions), but indeed now it is clear.
I’m definitely not saying that you need anything more than FXAA or [insert your favourite reconstruction filter here] for edge gradients. At a decent enough resolution, they do a perfectly adequate job and I completely agree that MSAA is overkill for that, particularly as you start to increase the sample count.
The issue I’m talking about is #3 – namely where you are (locally) sampling the scene at too low a rate and thus as the camera moves you have objects/triangles “alias” on and off the sampling grid. This is incidentally why I immediately reject all users tests of “anti-aliasing” that are static and don’t show moving geometry/camera.
Sure you can always throw out the whole “well use LOD better”, etc. (and I have!) but the fact that not a single game to date that I have played has avoided this fairly distracting artifact entirely leads me to believe that it’s not as simple as it might seem (short of filterable geometry representations of course, but that’s another topic).
Also realize that I’m mainly commenting from the PC side, where you need a lot more pixels to reach the equivalent density as a TV viewed from across the room. This changes the constant factors of course, but not the argument.
So yeah, I don’t disagree with your points above at all, excepting perhaps that #3 is not as easy as you seem to imply and MSAA still feels like a good tool in the solution to it. And if you want to just wave the magic “do everything with alpha blended imposters” wand, then you arrive back at that being inefficient in an immediate-mode renderer and therefore looking at… tiling
Gotcha. I can tell you from experience that the reason games ship with it is because it’s not worth the cost of rewiring the entire engine to fix it. “Well we can fix the aliasing on that bush, but then we have to cut out 100k triangles from your budget”.
Plus, post-effects can help with aliasing inside the triangle too, which MSAA won’t do anything about. That’s quickly becoming the #1 source of aliasing (think sharp specular on a rough surface). You probably want to take direct action against that sort of stuff anyway, but with a post-effect AA pass you can potentially afford to err on the side of “sharpness” (e.g. do clever filtering of the normal map that takes variance into account and does a decent job, rather than manually super-sampling the specular in the shader) and if things indeed end up causing sharp edges in the final frame, the post-effect AA will hopefully smooth it out.
Speaking of high end at 1080p and “good enough”, yesterday I was optimizing FXAA 4.0 with spatial and 2x temporal super-sampling (TSSAA), and 1080p with the highest quality preset on a HD7970 is 0.4 ms/frame. This is a little slower than the highest quality FXAA 3.11, but much higher quality on long edges, diagonals, and especially in motion. Probably “good enough” for next gen for deferred rendering as something 6x the performance of a 360 could do this in under 1 ms/frame, and anything less powerful can use a lower quality preset.
That being said, returning to forward rendering and taking advantage of DX11 to do all lighting in one pass, could be quite awesome when mixed with 4xMSAA with a catmull-rom 2.0 pixel filter for resolve and non-spatial 2xTSSAA post-AA.
I’ll be covering a bunch of this and more in my GDC presentation this year.
John: yeah I understand those constraints. Hopefully on the “next gen” hardware (or current gen PC games arguably) some of the increased performance will be spent on reducing that form of aliasing. I’m still not totally convinced that it’s simple to handle view-dependent stuff like window sills, bevels, etc. (the simplest way may actually be rasterizing them with subsampling) but I’d love to hear your thoughts on that sometime.
Timothy: sounds great! I’m especially interested in hearing your thoughts on better resolve filters (you mention catmull-rom for instance). Assuming no unavoidable conflicts, I’ll definitely be attending your talk at GDC.
Good post, and I’m glad I’m not the only one who thinks graphics programmers are overly obsessed with aliasing. I see arguments all the time, with some incredibly smart people, not on whether they SHOULD spend 20%+ of their available milliseconds on AA, but on how best to do it.
I mean, aren’t there other priorities? The difference between offline CG and realtime is a lot more than just “the former used 64x super sample AA!” Frankly when I’m playing something on my own pc and need to turn down settings AA is always the first thing to go.
Interesting. Also, I just realized that I never noticed any aliasing while playing Uncharted 3, so I guess you’re right. And I usually get really annoyed by it when playing games.
Anti aliasing is actually more important in motion than on still images, games look usually so “unreal”, that on a static image, aliasing is one of the least noticeable issues/feature. bluring edges using MLAA/FXAA works kind of for those nice screenshots, but when you start moving, aliasing becomes visible again.
In reality, to not notice aliasing on a sharp screen, the resolution would need to be proportional to the brightness difference of the discreet pixels that are causing aliasing.
That leads to: geometry aliasing will be less and less important in future, it’s solvable quite easily (if you just want to hide geometry borders to the player). Shader/Material aliasing will become more of a problem. Imagine you want to draw a night sky (pure black:0×000000), with bright starts (pure white:0xffffff) – no fake alpha faded sprites- AND avoid any aliasing in motion. Some will say, you need twice the sampling frequency of the object you sample, … but of what? it’s not of the object size, but… ?
if you don’t want to see any aliasing, you’d need to ovesample in a way, that the brightness difference of some aprticular area (the one that maps to one ’sensor’ of your eye) stays relatively constant. you might assume you don’t see a brightness difference of 1/16th of the screen min/max, with the ‘twice the sampling frequency’ rule, it would mean we need to make around 32sample points.
And that’s just the case if you work with after-tonemapping AA, if you want to anti-alias HDR, you’ll need to have even more samples to not realize any aliasing. That explains why movies vary their super sampling between simple 7×7 and sometimes 39×39 (and add post effects to further hide aliasing, like blur+sharpen)
@Andrew:
Yep. There are some issues in games that are “easy” to solve with the existing pipeline if you have more time to spend, and some that aren’t. Take anything with an alpha cutoff (like trees in the train level in Uncharted 2). The artists could choose between using aliased alpha cutouts, or they could use a two pass system where the second pass is blended but they would have to significantly reduce the number of leaf cards. In most cases, the artists choose to have aliased edges with more cards.
There are some cases that are harder but doable. Like thin pieces of geometry that are far away (like flag poles) could be solved switching to an alpha card far away, but few games do that because it’s not worth the effort. It’s kindof like LOD switching in a way. Back when the PS3 was being released Sony had all these presentations on how to automatically blend between LODs with no pops. And no one that I know of uses it because it’s not worth the trouble.
There are a few cases where to reduce aliasing you would have no option but to go with some variant of MSAA. But those cases are quite rare IMO. Triangles and draw calls will probably still be tight on next gen. That’s why any technique that starts with “and you need an extra render pass” is a nonstarter for me. Tiling with MSAA fits in that category.
One final note: Many artists don’t actually care what the final game looks like. Most do, but you’d be surprised how many obsess over how their art looks on their screen but don’t care what the player actually sees. In many cases they could fix the aliasing with better LODs but they don’t because they literally don’t care. I’ve seen it happen.
@Jonathan Swift:
Yep, I’d agree. With the one caveat that before these post techniques took off I was a huge proponent of at least 2x MSAA. 1x with no good cleanup looks bad enough that I have trouble focusing on anything else. Fortunately, FXAA and other variants fix that problem for me.
@Micki:
Good point on HDR and movement. Crawling jaggies are certainly much worse than regular ones. For HDR, I generally propose doing AA after tonemapping which is yet another advantage for post techniques over MSAA.
This post is surprising to me, I would have said (and still stand) on the opposite. I do agree that shading (>lighting) is way off, but perceptually aliasing is (to me) more disturbing, because it has no place in the real world and it’s a clear visible unstable CG artifact. Shading is wrong, but in a stable way which works I’d argue at a completely different level.
Now, for AA, your observations are incredibly interesting, but I suspect that we will be using the extra resolution also for bigger FOVs… LCDs are increasingly cheap, projectors are very cheap, and today I’m surely filling much more FOV with my screen than I was ten years ago. Until we reach movie theater-like FOV screens, we will be spending resolution I think more on FOV than on acuity.
What do you think?
Fix your basic grammatical errors, then I’ll take your content a little more seriously and maybe actually read all the way through. “You’re”, “your”, “then”, “than”. Get it right.
Ironically, you’re arguing about the important difference between interrelated technical concepts yet getting basic grammar wrong.
The game I’m working on currently uses MSAA. Today I turned it off to test something, and, really, it was ugly. It’s even worst with DoF/bloom/whatever that makes things pop on and off. Of course it’s more obvious on thin geometry. Maybe some anti-aliasing techniques are good enough for static images, but when things are moving, aliasing can quickly become annoying and obvious. I’m not saying MSAA is the only solution of course, but it helps. I did not test on a regular TV from a standard distance, and it was 720p “only”, but depending on the post-process you use, it may not really matter, when a single blinking pixel can become a big blinking halo.