What you are talking about is large saccades with a 200 ms interval where you can reconstruct. However, smooth pursuit of a target moving across the scene won't have this interval. Consider a viewer seeing a wildlife scene, like HorizonZD. The viewer may watch a large dino, then jump to a flying saurus, follow it past trees, then jump to a robotic butterfly.
Yes, but if we can track eyes exactly with low latency, no need for reconstruction of that tiny 200^2 pixels square we have to do at full res. We might even afford true multisampling in that region.
I'm mostly worried about inaccuracy from tracking, so making a guess on focus if input is noisy might help. Tracking from the top of a flat screen surely is much harder than in a headset. (Also: a couple could no longer play such game using just one couch, console and display
)
Reconstruction seems more usable for the peripherical regions, like shown in the video.
I wonder how much the AI reconstruction is neccesary versus just having the periphery blurred out or otherwise naively reconstructed?
For this question, i often try to analyze what i see on out of focus regions.
E.g. i look down on the ground, and there is soil with many tiny bright boulders on it. The boulders are at high frequency. I can not see them precisely, but i still do know it's many tiny boulders causing high contrast. I guess the brain makes up this information from knowledge, although the eyes can't see it.
If we did a naive foveated render, the boulders would appear blurred, and the sharp high frequency contrast would get lost. So eventually our brain would complain and detect the trick.
If so, advanced upscaling like neural techniques might be worth it to prevent this issue. I feel like we still have some sense of texture and patterns at high frequency, although we can't see it sharply.
Also, how does this apply to frame interpolation? Approximate updates in the periphery at lower frequency??
I share this question too. Ideally we can have lower resolution and lower fps, and we can also use heavy temporal sampling to smooth out aliasing and jitter so it's not confused with motion triggering reflexes.
Personally i assume we can't do lower fps eventually, but i hope TA is acceptable. Just guessing.
In the worst case, the requirements on temporal stability are so high that most advantages get lost, and a speed up of 4 remains all we can expect in practice.
Idk. But i assume VR industry will pay the bill to figure it out, together with all the changes needed on realtime rendering. Once they have it working and widely adopted, people may start to consider it for flat screens too.
Could be a godsend for the gaming industry: AAA visuals on mobile HW, lifting the barrier between mobile and PC/console development, further growth of the market.
Maybe the bottleneck of future games is no longer gfx, but becomes the actual simulation of the game, which would make some sense.
