*merged/spinoff* for the Neverending Killzone Discussion on Graphics

[joker454]

One way to selectively target edges, which isn't at all complex, would be to select triangle edges as a mask. Do a plain vanilla geometry pass, edge detect, and derive from that which pixels should and shouldn't be post-processed. Or perhaps, MLAA the whole scene but mask the results so only the edges are composited with the unprocessed original.

I thought about that, but it would have to be more complex than that to work. If you just wrote out triangle edges as a mask then on an image like this:

http://www.philipresheph.com/demodokos/book1/iliad60.jpg

...you have triangle edges right in the middle of that which need to be left alone, and hence would need to be excluded from the mask otherwise detail in there would get blurred. You need to write out silhouette masks, but then it all gets more involved. Plus if you need a pass on gpu just to get this mask data then you are adding gpu workload, which seems counter to the whole idea of relieving load from gpu. Also your idea adds a compositing step which also implies extra work (this should be doable without any compositing step), and extra memory as a compositing step implies having two buffers in memory at the same time which isn't ideal.

You could also use depth value in the process you suggest which would help in the above image, but wouldn't solve this case:

http://www.sandstonedesigns.com/Precast_Concrete/Products/images/image044.png

...at the corner of the column where z values are all real similar. If the sun is off to the right then you'd have quite a contrast betweem the two sides of the colmun and hence aliasing that would need to be smoothed. So mask + z wouldn't quite work there either.

I think the clue is that so far mlaa has been added to games that all use deferred renders. I don't know of any forward renderer games that use it. Said games already have a normal buffer lying around, hence why I'm guessing that they are going by color + normals since that lets them leverage existing data. So use color to find locations with high contrast, and use normals to see if it should be smothed or left alone.

If secrecy/NDA is the concern, I doubt you'll get more from T.B. in a public forum. You have better chance asking creative questions behind the scene.

Who knows, sometimes companies like to flaunt their tech and talk about it publicly. Or maybe we can trick T.B. into leaking it Anyways I did ping a friend for info, I'll see what he says.

 

[patsu]

...you have triangle edges right in the middle of that which need to be left alone, and hence would need to be excluded from the mask otherwise detail in there would get blurred. You need to write out silhouette masks, but then it all gets more involved. Plus if you need a pass on gpu just to get this mask data then you are adding gpu workload, which seems counter to the whole idea of relieving load from gpu. Also your idea adds a compositing step which also implies extra work (this should be doable without any compositing step), and extra memory as a compositing step implies having two buffers in memory at the same time which isn't ideal.

I thought MLAA happens "inside" the SPUs ? Can they generate the "masks" themselves ? They know where the data are since they did the culling.

 

[joker454]

I thought everything happens inside the SPUs ? Can they generate the "mask" themselves ? They know where the data are since they did the culling.

I don't think it does. If they did it 100% on spu then they would need a functional geometry engine working on spu to transform all the geometry, etc. That's a metric fuck ton of work to do, plus I think that would make integrating it into existing games much more difficult. I think it's more of a post process step where you jsut give the mlaa library a pointer to your color buffer and a pointer to your normals buffer, and they presumably have to be in a certain format. So render your scene on gpu to fill the gbuffers, call MlaaThatSucker( pColor, pNormals, fAgresiveness ) which runs on spu, and then do other crap on the ppu+gpu while the spu's churn away. That would be 'relatively' easy to plop into existing games. Also it wouldn't need any extra memory since it could operate on everything in place, and presumably it doesn't really add extra load to the gpu, perhaps it may make things stall at some point if whatever you are doing in parallel with spu mlaa finishes before the spu's are done.

Just a guess though, no way for me to be sure. But we know T.B is watching so we can just bait him. So let's set a trap.

(clears throat)

Mlaa on PS3 does not process geometry on spu. Instead it relies on the gpu to fill the gbuffers, then the spu's are used to smooth out the color buffer after geometry processing has taken place. This is a fact and everyone knows it, stamp it no changes.

There we go, now we just wait

 

[patsu]

I think it's more of a post process step where you jsut give the mlaa library a pointer to your color buffer and a pointer to your normals buffer, and they presumably have to be in a certain format. So render your scene on gpu to fill the gbuffers, call MlaaThatSucker( pColor, pNormals, fAgresiveness ) which runs on spu, and then do other crap on the ppu+gpu while the spu's churn away. That would be 'relatively' easy to plop into existing games. Also it wouldn't need any extra memory since it could operate on everything in place, and presumably it doesn't really add extra load to the gpu, perhaps it may make things stall at some point if whatever you are doing in parallel with spu mlaa finishes before the spu's are done.

...

Mlaa on PS3 does not process geometry on spu. Instead it relies on the gpu to fill the gbuffers, then the spu's are used to smooth out the color buffer after geometry processing has taken place. This is a fact and everyone knows it, stamp it no changes.

Eh… I edited my post to say 'MLAA happens "inside" the SPUs' before your note. ^_^
Certainly, not "everything" happens inside the SPUs.

You have just described what I had in mind. It's a post processing step. But can the SPUs also do their own prep work in parallel, before the actual detect and blend steps.

 

[Shifty Geezer]

I thought about that, but it would have to be more complex than that to work. If you just wrote out triangle edges as a mask then on an image like this.

Yes, you'd need to refine the method somehow. The other way I'm thinking of (from offline rendering) is indexed triangles that belong to a specific model. That wouldn't work either though, not on its own, as a model that overlaps itself, like a character with their arm in front on their chest, would still create an edge that may need MLAA applied.

Combined with things like a threshold test of the z depth, I think a decent mask could be made, but that sounds a little convoluted.

I think the clue is that so far mlaa has been added to games that all use deferred renders. I don't know of any forward renderer games that use it.

LBP2. Alex Evans : "Oh and also it means that the engine is no longer 'deferred' in any sense - being a traditional forward renderer makes alpha/transparency easy to do again, without special code paths."

 

[Nesh]

God of War 3 used a deferred renderer?

 

[JB9861]

God of War 3 used a deferred renderer?

For the shadows. I thought the lighting was deferred too, but apparently after skimming through CG society and DF articles nope.

 

[joker454]

Interesting, so LBP2 and Gow3 aren't deferred. That LBP2 article that Shifty linked says they dropped in mlaa support in a day, so I can't image that is has to be tightly tied to geometry to work. Maybe both of those games aren't deferred but they already had normal buffer representations of the screen due to how they handled their lighting. Barring that how else could it be so quick to integrate mlaa into the games? The geometry side on ps3 games can get fairly complicated and customized, it just doesn't seem like a one day job to fit mlaa in there.

 

[patsu]

Do those games use SPUs to cull their geometry ?

Integration can be quick if "everything related to MLAA" happens within the same framework.

 

[Cyan]

SCE WWS ATG, or so they keep telling me.
But yes, that would be me. I'm the designated nurse of our MLAA algorithm.

I seriously can't stand this right now.

Just kidding. But OMFG, you actually got to create the MLAA algorithm. MLAA!!!!!!! I want it for the 360 too, more so knowing that it doesn't blur the image -if I knew it does I wouldn't like it much, there's a reason I set the Sharpness on my TV to 75, I love sharp graphics-. I am so jealous that I am about to spontaneously combust. It's a technique that would do wonders in an amazing title like Halo Reach (I wrote a very brief review of it here)

http://forum.beyond3d.com/showpost.php?p=1526242&postcount=2762

I wonder if that effect is possible on the 360, or if it makes sense taking into account there's the eDRAM on the GPU for that. I would love to know your opinion on the subject. No need to break any NDA, I'm just curious.

But I hope you can respond. If you get any pictures of/with it on the PS3, or if you can't -np at all- MAINLY just saying if it can be applied or used on the 360, would you be willing to post it on here?

 

[ShadowRunner]

Yes, you'd need to refine the method somehow. The other way I'm thinking of (from offline rendering) is indexed triangles that belong to a specific model. That wouldn't work either though, not on its own, as a model that overlaps itself, like a character with their arm in front on their chest, would still create an edge that may need MLAA applied.

How do cell shaded games like valkyria chronicles or XIII detect which edges on characters should be outlined? Maybe a similar technique could be used for MLAA?

 

[Cyan]

MLAA is still new, there haven't been too many games using it yet. But don't worry, UC3 will provide the HDR + MLAA in a shooter.
Still, there is no better AA that's doable in a console. If you want a FPS with cutting edge graphics, you use MLAA on PS3 and TAA on 360.

Agreed on the better AA on console. MLAA + HDR, which I am not sure is doable on a console, sounds like a win/win to me.

Regular MSAA and FSAA are great from AAx4 and above, especially at 1080p native resolution. But Sony's solution is unbeatable at lower resolutions if it actually doesn't blur the image.

Even so, KZ3 still looks muddy to me. I didn't like KZ2 art at all because of that, regardless the use of colour. KZ3 looks a lot better, more colourful, crisper and sharp.

Even so, my main gripe about the game is that it still looks muddy and not as sharp as let's say UC2, Halo Reach and GoW3. I wonder if it's because they want to portrait an environmental dirt setting, as they invented a dark, gloomy and tenebrous planet.

In doing so they seem to have reduced the colour palette to four colours; black, white, grey, dark grey.

KZ3 looks crisper but still not as sharp as other games, and I wonder if it's because the true high quality MLAA solution was too much for the game with all the things that happen at the same time and all the tasks they delegate to the SPUs, so they went with MLAA but a less demanding setting, or it's just an art decision.

 

[patsu]

In doing so they seem to have reduced the colour palette to four colours; black, white, grey, dark grey.

Are you talking about KZ3 ? If so, you should be able to see more colors in other KZ3 screenshots or videos.

The MAWLR Graveyard gameplay video looks rather clean too.

 

[AlNom]

How do cell shaded games like valkyria chronicles or XIII detect which edges on characters should be outlined? Maybe a similar technique could be used for MLAA?

Crackdown used the depth buffer to determine edge discontinuity. The amount of work depends on the number of edges being processed though, so they limited it to a certain distance from the camera. You can also do a colour contrast/luminance difference so you're doing it on edges that will be more noticeable, but that may not be ideal because texture details will be detected as well. Having access to depth at full resolution would give a superior edge detection, but then you'd need to have a copy of it in XDR on PS3 in addition to the colour copy of the frame so that the SPEs could use it. XDR is already arguably a priceless commodity to begin with so it might not be budgetable. I doubt a low res copy of depth would be adequate for MLAA purposes as well compared to other post-effects or fast occlusion querying.

 

[patsu]

Yes, is it possible to devise a more compact way to keep the depth or geometry info for edge detection alone ? Instead of storing the depth for the entire frame.

Since the SPUs need to cull the geometry in the first place, they'd need to touch every edge right ? Can they preprocess the info there ?

 

[Cyan]

Are you talking about KZ3 ? If so, you should be able to see more colors in other KZ3 screenshots or videos.

The MAWLR Graveyard gameplay video looks rather clean too.

I didn't see that video because I don't have the link -I will look for it in the KZ3 thread-. I saw other videos where it was rather obvious they have increased the colour palette. I was talking about KZ2 actually. KZ3 handles the colour in a different way, the little I could see anyway.

I still don't know. I may not even be remembering it right...

But I remember watching a couple of videos where it was clear the game has a richer palette.

 

[Arwin]

@Cyan: I would expect your impression of muddyness to stem from lower resolution for transparency effect due to bandwidth limitations (together perhaps with LDR) rather than anything to do with MLAA. Look at the game and find anything blurry that is not hidden behind smoke/dust fx or a rare low-res texture, and then we can see if it is (ML)AA related.

 

[joker454]

Crackdown used the depth buffer to determine edge discontinuity. The amount of work depends on the number of edges being processed though, so they limited it to a certain distance from the camera.

Could be they did that as well because Z isn't linear, so up close to the camera you have more precision so it's easier to located edges whereas further out it gets tougher. Ultimately z + color don't work on all cases anways like that column picture I had linked earlier because the z values will be too close together to locate an edge. Hence why I suspect normals must be involved somehow as well.

 

[Cyan]

@Cyan: I would expect your impression of muddyness to stem from lower resolution for transparency effect due to bandwidth limitations (together perhaps with LDR) rather than anything to do with MLAA. Look at the game and find anything blurry that is not hidden behind smoke/dust fx or a rare low-res texture, and then we can see if it is (ML)AA related.

Oookay. That was interesting. Perhaps some cause for the confusion might be because "muddyness" and any kind of assimilation to it is conceptual and a phenomenon that can happen to different games depending on the setting. I've seen it sometimes in Crysis 2 demo and something similar with the occasional ghosting in Halo Reach... but it wasn't comparable. But what does "muddy" mean? In all honesty, when it's about Killzone 3 it doesn't have to mean a thing.

GG portray a dark planet and I have watched so many Killzone 2 images and some videos that I got used to think that it was the typical Killzone setting. But, like they say, you cannot put all of your eggs in one basket; just because Killzone 3 changed positively in regards to KZ2. The later is undeniably muddy, and I never understood the hype nor what the fuss was all about. I always liked crisp games, like when you can easily discern the metal shader in Crysis 2 big trucks.

The problem with talking about this over the internet without having the PS3 myself is that there is absolutely no certifiable way that I can figure out a real reaction towards the game and its graphics, if they look darker or not to me, etc...

I judge by screenshots or videos and the muddyness isn't as present as it was before, but they certainly use a lot of filters for this game, it's very difficult to tell what it is exactly. I never thought about the smoke/dust thing, to be honest.

Another game I found muddy at first was Mass Effect 2, because of the use of the fish eye camera, or whatever it's called, and the film grain filter which I turned off. But I must admit it was a feeling I had at first, then I realized it looked really fine. I wouldn't mean it to be a criticism... I think some people just have a knack for starting "verbal" uproars on the net anyways. :smile:

Regarding KZ, maybe they try to recreate something similar to ME2, a heavy atmosphere in a different planet. ME2 is crisp, while KZ2 is blurry I could have a hard time playing it, and KZ3 has improved this, it's closer to ME2 than KZ2.

I'm using ME2 as an example of a game with a similar setting. Other than that, they are two completely different games; art style, 3rd person/1st person, etc etc

 

[JB9861]

there's a reason I set the Sharpness on my TV to 75, I love sharp graphics-.

Shouldn't be any particular reason. You're most likely getting noise and artifacts in return instead of sharp graphics.