Digital Foundry Article Technical Discussion Archive [2014]

[betan]

Assume that you can, I don't dispute the conditional.
I dispute the claim though, it's where I'm getting at.

I'm clear on your stance against the _condition_, which is fine and interesting but you also seem to imply Nyquist prevents one from using previous frame info to predict parts of next frame. Wasn't that the point of that ugly graph?

That's a good argument, the less details there is in the original scene, the better you can reconstruct, because there's less to reconstruct. I actually don't dispute this, but I think this is skipping a more accurate model and trading more more artificial details?

Texture filtering is just one dimension, when you change the camera, even just so slightly, don't you change the light as observed on the pixels, at least in some extreme cases like on the tangent? It's probably not detectable with human eyes, which is why it works pretty well, but that's not to say that you can reverse a pixel-to-pixel perfectly?

Regarding obtaining even columns of the ideal rendering (that is ignoring the temporal issues of the technique), I don't think pixel shading or any form of sampling that does not require neighboring sample values would prevent one from accurately "half rendering".
I agree, though, the likes of postprocessing and texture filtering makes the problem non-trivial.

I don't dispute on the fact this is quite a feat given the the quality obtained and the amount of computation saved. I dispute on the claim that this is somehow exactly equal to what you get with native 1080p60 (maybe it's obvious, it just seem to me that few people are still yammering about it)

Yeah, it is obvious.
The practical question is though how much it is better than 960x1080 (or worse than 1080p). GG has the engine, let's hope we will see some quantitative data at GDC.

 

[HTupolev]

Except that the scene is technically not static, because the camera got moved to created the offset.

Actually, that's not necessarily true. The real-world case is a little flaky, because for actual camera motion you'd have to use some kind of projection that ensures alignment (I'm thinking some sort of equirectangular-esque projection, but I haven't thought too hard about it?).

But when rasterizing computer graphics, you don't need to move the camera at all. You can just tell the system to sample from even and odd points.
I'm not sure what it involves to tell a GPU to do that, but in the software rasterizer I wrote, it's trivially easy. You can simply tell it to rasterize just the even lines, or just the odd lines.

Perhaps a better camera analogy is that, rather than moving the camera, on one frame you only record data from even lines, and on the next you only record data from odd lines.

 

[taisui]

I'm clear on your stance against the _condition_, which is fine and interesting but you also seem to imply Nyquist prevents one from using previous frame info to predict parts of next frame. Wasn't that the point of that ugly graph?

I don't dispute use past frames with motion compensation is a good approximation of the new frame, being able to get away with just 50% of the resolution while finding a cheap algorithm to plug the gaps is ingenious.

I guess this is what bugs me at the core:

People think there is a way, that you can render 2 frames at half resolution, given that the scene data are actually processed twice and all that additional trickery or whatnot, can produce the same quality as the full native render, at less computational cost.

Regarding obtaining even columns of the ideal rendering (that is ignoring the temporal issues of the technique), I don't think pixel shading or any form of sampling that does not require neighboring sample values would prevent one from accurately "half rendering".
I agree, though, the likes of postprocessing and texture filtering makes the problem non-trivial.

That's fair, I suppose that's what I'm at least trying to say.

 

[HTupolev]

People think there is a way, that you can render 2 frames at half resolution, given that the scene data are actually processed twice and all that additional trickery or whatnot, can produce the same quality as the full native render, at less computational cost.

Nobody is saying it's for less computational cost. At a minimum, the computational cost to do two half-res renders and then combine is going to be equal to a full-res. That's for a theoretical static case where the game engine isn't doing anything and we're only considering per-pixel costs.

Obviously in the real world, the cost will be higher. It'll just be lower per frame cycle.

 

[taisui]

But when rasterizing computer graphics, you don't need to move the camera at all. You can just tell the system to sample from even and odd points.
I'm not sure what it involves to tell a GPU to do that.

Magic, is what's involved when you can do that.

 

[HTupolev]

Magic, is what's involved when you can do that.

Is it also magic that a frame of video can contain several pixels at several locations? Because that's literally what you're saying.

You don't need to move your camera to select new sample points.

 

[taisui]

Is it also magic that a frame of video can contain several pixels at several locations? Because that's literally what you're saying.

You don't need to move your camera to select new sample points.

I'll have to ask Shifty politely to explain how this works to you.

you can, easily. When you render the scene, you sample at discrete points. The first frame you sample at relative positions 0,2,4,6 etc. The next frame, you sample at points 1,3,5,7 etc. This is absolutely doable in game. It's trivial. You can just offset the camera's horizontal position by a half pixel between each 960x1080 frame.

 

[HTupolev]

I'll have to ask Shifty politely to explain how this works to you.

Shifty wrote that imprecisely, using camera shifting when he probably meant sample point shifting; "half a pixel" for objects on-screen does not correspond to any particular camera shift amount. Offsetting the horizontal position of the camera creates obvious problems; anything closeby translates by a huge amount in screen-space, while anything at "infinite distance" does not translate at all.

Rotation can be more correct (it avoids parallax), but it'll only be "correct" in certain cases where you have a constant correspondence between buffer space and angle space (which might work with some sorts of equirectangular-like projections). Otherwise, a 1-degree rotation might be a half-pixel shift at one location, and a 5-pixel shift at another.

The most correct approach is to not move the camera at all, and simply move your sample points.

 

[Shifty Geezer]

Shifty wrote that imprecisely, using camera shifting when he probably meant sample point shifting;

The most correct approach is to not move the camera at all, and simply move your sample points.

Yeah, I'm using 'camera' as the projection matrix sampling the data. But TBH I was really talking more about 2D images. This started with taisui saying KZSF's upscale was equivalent to Ryse's 2D upscale and it wasn't possible to reconstruct an image of alternating black and white lines. It was never specified that the source image was a quad with pixel-aligned texels, and that scenario doesn't seem relevant to the application. When talking about interlacing frames of a game, the frames are '2D images' (using our simplified model for discussion purposes and not the extended data sources of KZSF). So we'll have a 2D render of the game scene, which can be reconstructed from interlaced half-width images to 100% original fidelity as long as the scene is unchanged. It's not mathematically impossible as taisui suggested. But otherwise I don't care to reply. This debate is going nowhere. Taisui is clearly not going to budge and his meandering points are starting to make the discussion awkward IMO. If other people want to carry on butting their heads against the wall, they can, but I'll stick to talking about the interesting points of how the technology (actually) works.

 

[taisui]

and I think you are taking my comments of out the context of a typical 3D scene, and said it doesn't apply to your 2D image simplified model.

 

[(((interference)))]

So does this technique lend itself well to be used on XB1?

Or will the need to have all those framebuffers stored in ESRAM cause issues? Maybe main memory is fast enough?

 

[HTupolev]

So does this technique lend itself well to be used on XB1?

Or will the need to have all those framebuffers stored in ESRAM cause issues? Maybe main memory is fast enough?

Well, temporal reprojection for increased sampling must be at least somewhat non-horrifying on XB1, as Ryse is evidently using it to get a "supersampled" result prior to the SMAA post pass.

 

[Billy Idol]

I have a question: can this method be used to compress video data?

Suppose you have a 1080p video stream, you only need to stream half the data...right? You locally store the three most recent frames to reconstruct the high quality output, but it seems to me that with this, you can basically double your bandwidth

As a follow up question: can this be consequently used for a game streaming service like playstation now?

 

[sebbbi]

I have a question: can this method be used to compress video data?

Suppose you have a 1080p video stream, you only need to stream half the data...right? You locally store the three most recent frames to reconstruct the high quality output, but it seems to me that with this, you can basically double your bandwidth

As a follow up question: can this be consequently used for a game streaming service like playstation now?

Every modern video compression codec already heavily relies on reprojection from last frame data, and only stores full frames very infrequently. This is how you achieve high quality at low bit rate. Video codecs usually only move data in 2d direction (while a 3d engine can move and reproject along any shape). This discussion is basically about using similar techniques for game rendering that video compression has used for long time. People tend to think that Blu-Ray looks goods (even at 24 fps and regenerating pretty much never more than 20% of pixels per frame), so using similar kinds of methods to save cost in real time graphics rendering is definitely a good idea, and shouldn't degrade image quality too much.

 

[Rangers]

Highlights how Xbox One should be $299 imo. That is the level of the hardware you're getting, or the level where the hardware you're getting becomes a great deal.

Of course MS would scream "Kinect!"

 

[dobwal]

Highlights how Xbox One should be $299 imo. That is the level of the hardware you're getting, or the level where the hardware you're getting becomes a great deal.

Of course MS would scream "Kinect!"

Its a mainstream consumer device. You are not purchasing a bunch of components to DIY yourself. Little things that may be irrelevant to you may not be irrelevant to the consumer base as a whole.

You purchase gives an all encompassing warranty when you purchase a console. When one of your console's components goes out prematurely, you don't call the manufacturer of that component. You simply call MS. AMD or Intel is not going to walk you through a problem unrelated to their products. Even if you buy a PC with a 750Ti in it, the probability of the hardware vendor troubleshooting and remedying a software issues that fall outside the scope of their responsibility is small.

The cost of building a gaming PC is not directly comparable to cost buying a console for the vast majority of the gaming market.

 

[taisui]

I'm quite sure that interleaved sample patterns are well known concept..
http://liris.cnrs.fr/Documents/Liris-2476.pdf
https://developer.nvidia.com/sites/...VOIL_ParticleShadowsAndCacheEfficientPost.pdf
http://www.mpi-inf.mpg.de/~rherzog/Papers/spatioTemporalUpsampling_preprintI3D2010.pdf

So I spent some time and quickly read through the links.

link1 talks about a method, where GI pass can be sped up by doing it in lower resolution samples, which are downsamples of a high resolution G buffer already rasterized from the scene.

link2 talks about a way to do jitter sampling without doing the actual jigger, from a texture (which can be a high resolution frame buffer of the scene), given that a texture is already rendered, it is trivial.

In both cases, they are downsampling on an existing 2D images, which is trivial (and not interesting)

link3 talks about a temporal upsampling method that's similar to Killzone MP, there is a high resolution G buffer where past frames at lower resolution are jittered back to the G buffer resolution. The trade off is a good approximation with performance boost.

Which brings to the point, where some think it's a trivial, to sample directly from the 3d scene at different "subpixels" at will, even though the how is never explained.

If we can do that, we'd solve the age old AA problem by running another supersampling pass only on the known edge pixels (obtain by pre-Z) at "subpixel" level, after all, we know exactly where and how detailed that point should be sampled at, w/o going to a higher resolution buffer. (maybe you can? love to know how, serious, still stuck at DX9 era)

 

[HokutoNoKen]

Tech Analysis: Dead Nation: Apocalypse Edition on PS4

http://www.eurogamer.net/articles/digitalfoundry-2014-dead-nation-ps4

Interestingly, anisotropic filtering is absent from the game - not unlike a number of other recent PS4 releases - though in fairness, its absence only becomes visible when pressing up against the wall and it's not a serious detriment to image quality.

Hmm, the third game on PS4 that lacks AF. Any developer here that could give an explanation or theory?

/ Ken

 

[KKRT]

Its not 3rd game. Killzone and The Order have low values or lack of any AF most of the time.
I think COD: Ghost also had low AF values.

 

[DJ12]

Could it be that they haven't saw fit to add it to the low level api yet? Seems a bit odd for it not to be enabled.

I have a 7870 so could check what the hit is for filtering but my pc is generally cpu limited so probably won't notice any difference.