Xbox Series X [XBSX] [Release November 10 2020]

[iroboto]

They could have mentioned a custom solution to this problem when talking to DF about the cool stuff in XSX. It's not like there was a word limit on the DF article.

They may not want to or find it necessary to. Likewise for Sony. With this generation ending and some more information coming out we find there are features on PS4 that MS marketed but Sony did not for instance.

 

[Deleted member 11852]

They may not want to or find it necessary to. Likewise for Sony. With this generation ending and some more information coming out we find there are features on PS4 that MS marketed but Sony did not for instance.

I don't know why Microsoft would reveal so much about the cool stuff XSX, spend so much time focussing on the Velocity Architecture but fail to mention that this also brings problems and not mention their solution? I mean it's not necessary to do these deep dives with DF at all, this is really all fan service. The vast majority will never read it and the vast majority of those who do rad, will not come close to fully comprehending it.

But in my experience, when an engineer comes up with cool solution to a problem, they want to talk about it.

 

[Ronaldo8]

In the Eurogamer deep-dive, Andrew Goossen candidly mentioned about a hadrware feature included in the xbox one x that enabled the monitoring of texture usage and memory footprint:

As textures have ballooned in size to match 4K displays, efficiency in memory utilisation has got progressively worse - something Microsoft was able to confirm by building in special monitoring hardware into Xbox One X's Scorpio Engine SoC. "From this, we found a game typically accessed at best only one-half to one-third of their allocated pages over long windows of time," says Goossen. "So if a game never had to load pages that are ultimately never actually used, that means a 2-3x multiplier on the effective amount of physical memory, and a 2-3x multiplier on our effective IO performance."

What can this special monitoring hardware be? Well, it is described in a patent filed by, you guessed it, Andrew Goossen and others titled: Automatic real-time shader modification for texture fetch intrumentation (US8872823B2). This patent was obviously filed back in 2014 confirming that SFS is the end-result of decades of research spanning multiple consummer products and harking back to Mark Grossman's research back in the early 90s.

 

[iroboto]

I don't know why Microsoft would reveal so much about the cool stuff XSX, spend so much time focussing on the Velocity Architecture but fail to mention that this also brings problems and not mention their solution? I mean it's not necessary to do these deep dives with DF at all, this is really all fan service. The vast majority will never read it and the vast majority of those who do rad, will not come close to fully comprehending it.

But in my experience, when an engineer comes up with cool solution to a problem, they want to talk about it.

I suspect that they don't want to get trapped. I recall the Xbox marketing team indicating the shader model support for X1X. But I know some of those required hardware features it doesn't have, I wrote to Phil and explained that this was false and that it would be used as fodder. I do recall that they no longer marketed that type of thing since then.

We're smart guys, we can figure out when something is amiss. Like we are now. But we aren't smart to know everything under the hood and can make mistakes on our assumptions. Generally speaking, Sony is a lot more closed up than MS. And that's worked out to their benefit. People just assume it's not there or that they have it. MS revealing more and more, makes people dive in deeper, ask more questions etc and could work against them. They may not necessarily want to discuss everything.

 

[AlNom]

They may not want to or find it necessary to. Likewise for Sony. With this generation ending and some more information coming out we find there are features on PS4 that MS marketed but Sony did not for instance.

They need to hire me to figure out an easy-to-understand marketing name. :sly:

Meet The Cachetodian.

 

[Ronaldo8]

Just on the topic of whether a console has or does not have.
They don't market everything. They only want to market what they want to market. Doesn't mean their competitor does not have that feature. I'm not saying XSX has cache scrubbers, but it also doesn't mean they don't have something equivalent either. Just really hard to market cache scrubbers when it's meaningless to their consumers. It either works or it doesn't work.

If MS has indeed the implemented the cache pre-fetch solution detailed in the patent, they don't need cache scrubbers at all.

 

[Ronaldo8]

They need to hire me to figure out an easy-to-understand marketing name. :sly:

Humble MS is best MS. I hope they have learnt their lesson and will not try to pull "power of the cloud" PR fluffs again

 

[iroboto]

I think they learned some valuable lessons from their mistakes with XBO and their victories with X1X. Just let the performance of their games do the talking.

 

[Ronaldo8]

I think it's just been held back by HDDs. JIT texture fetches are too slow, requiring lots of caching and adding complexity for minimal gains. Trials showed VT for all objects and had no issues, proving the theory. RAGE had to limit it to scenery for megatexturing. I reckon it'll become the norm and both console companies are counting on streamed assets to offset the relatively low RAM pools.

I suppose RTRT throws a spanner in the works as sampling is all over the shop. I models have enough detail, I wonder if we can't just use vertex data for reflections? Or maybe some large reflection texture-atlases of low fidelity textures? But I think within a few years, 50% of (larger budget) titles could be streaming textures at a minimum.

JIT texture fetches presupposes that the right texture tiles can be fetched from non-volatile memory to the GPU caches "just in time" for it to used by the shaders: that this is done in subtantially less than one frametime. In real practice, the GPU will have to "do without" the requested texture for some time. The "do without" part is where all virtual texturing implementations have met problems.

 

[Shifty Geezer]

JIT assumes a degree of prefetching. You'll never have true JIT from anything on storage. Heck, you won't even have JIT effective from RAM as it's not fast enough and the majority of data is prefetched into GPU caches.

VT prefetching tiles with the required data has only been a problem because HDDs are so slow especially at seeks. The elimination of seek delays and high transfer speeds should see SSDs able to serve VT tiles very well indeed. Heck, we saw almost the eradication of pop-in in RAGE on PS3 with an SSD and that was first-gen VT on a slow-arse SATA drive interface. We see in the UE5 demo on PS5 complex 8k texture detail streamed in, mostly very strongly with some blurring where it's uncertain where the problem lies, though that's more than likely the mesh data not being streamed in time than the textures.

 

[Ronaldo8]

JIT assumes a degree of prefetching. You'll never have true JIT from anything on storage. Heck, you won't even have JIT effective from RAM as it's not fast enough and the majority of data is prefetched into GPU caches.

VT prefetching tiles with the required data has only been a problem because HDDs are so slow especially at seeks. The elimination of seek delays and high transfer speeds should see SSDs able to serve VT tiles very well indeed. Heck, we saw almost the eradication of pop-in in RAGE on PS3 with an SSD and that was first-gen VT on a slow-arse SATA drive interface.

You said it:

Prefetching for a graphics shader (US10346943B2)
Instances in which the shader is long and has a plurality of threads of execution running in parallel, the impact on time of execution of the shader when failing to locate the requested subsequent cache line in the level one cache can be significant. As such, improvements are needed corresponding to the execution of the shader.

Accordingly, the present methods and apparatuses may provide an efficient solution, as compared to conventional solutions, by prefetching of cache lines during the execution of a shader. The present disclosure provides one or more mechanisms, including a jump instruction in a cache line, for initiating prefetching of a subsequent cache line of the shader stored in a memory. Further, the present disclosure provides one or more mechanisms for executing the respective cache line of the shader and skipping to a next one of the plurality of cache lines while the prefetching is occurring contemporaneously based on the at least one jump instruction.

 

[Shifty Geezer]

That's something else, populating the shader cache from RAM data to prevent the shader stalling. VT is about getting the tiles into RAM in time for their drawing. US10346943B2 is zero use if your data is sat on storage for the next three frames, but thankfully that's not an issue with the SSDs.

In short, I'm not really sure what your argument is. You've jumped from saying the tiles need to be in RAM to a patent about L1 cache misses. I've said VT has been held back by HDDs. Do you still dispute this, or do you agree with me?

 

[Ronaldo8]

That's something else, populating the shader cache from RAM data to prevent the shader stalling. VT is about getting the tiles into RAM in time for their drawing. US10346943B2 is zero use if your data is sat on storage for the next three frames, but thankfully that's not an issue with the SSDs.

In short, I'm not really sure what your argument is. You've jumped from saying the tiles need to be in RAM to a patent about L1 cache misses. I've said VT has been held back by HDDs. Do you still dispute this, or do you agree with me?

When was this disputed? However to assume that all the tiles will be available all the time in RAM for prefetching because of an SSD replacing an HDD is highly optimistic seeing that there are still artefacts present in the UE5 demo due to texture streaming and thats using what is perhaps the most performant SSD currently available.

 

[Shifty Geezer]

When was this disputed?

When I wrote it was mostly because of HDDs and you replied about data not being present and JIT solutions being problematic. There was nothing in your post to suggest any degree of agreement, such as starting with, "though true, there's still..." or somesuch.

However to assume that all the tiles will be available all the time in RAM for prefetching because of an SSD replacing an HDD is highly optimistic seeing that there are still artefacts present in the UE5 demo due to texture streaming and thats using what is perhaps the most performant SSD currently available.

As I already highlighted, we don't know the cause of missing detail in the UE5 demo. Is it missing textures, or missing geometry? Streaming geometry could be requiring an order of magnitude more storage workloads than VT textures alone would require. It's also a WIP and on the most part very robust, only failing in a few fringe cases. I pointed to a better case which was RAGE on PS3, which got near perfect results on a far slower SSD on a far slower bus than next-gen consoles. Surely that's a better reference point for potential than a WIP demo showcasing production-level assets instead of game-optimised assets?

Call me optimistic, but it's been shown that 100 MBs should be very adequate for 4k gaming and simple back-of-an-envelope guestimating tells us 2+ GB/s (well above for both consoles for texture data) with hundreds of thousands of IOP/S and a healthy tile cache in RAM of a few hundred MBs should have the problem covered. The only reason we haven't seen VT at its best is because of storage limitations. I think MS agrees with me having considered PRT as a good optimisation for their next console.

 

[Ronaldo8]

When I wrote it was mostly because of HDDs and you replied about data not being present and JIT solutions being problematic. There was nothing in your post to suggest any degree of agreement, such as starting with, "though true, there's still..." or somesuch.

As I already highlighted, we don't know the cause of missing detail in the UE5 demo. Is it missing textures, or missing geometry? Streaming geometry could be requiring an order of magnitude more storage workloads than VT textures alone would require. It's also a WIP and on the most part very robust, only failing in a few fringe cases. I pointed to a better case which was RAGE on PS3, which got near perfect results on a far slower SSD on a far slower bus than next-gen consoles. Surely that's a better reference point for potential than a WIP demo showcasing production-level assets instead of game-optimised assets?

Call me optimistic, but it's been shown that 100 MBs should be very adequate for 4k gaming and simple back-of-an-envelope guestimating tells us 2+ GB/s (well above for both consoles for texture data) with hundreds of thousands of IOP/S and a healthy tile cache in RAM of a few hundred MBs should have the problem covered. The only reason we haven't seen VT at its best is because of storage limitations. I think MS agrees with me having considered PRT as a good optimisation for their next console.

Well, the expectations have now been set. The entire internet now believes that next-gen will consist of this type of graphical fidelity.

 

[eastmen]

Humble MS is best MS. I hope they have learnt their lesson and will not try to pull "power of the cloud" PR fluffs again

The cloud will be back don't worry about that. MS is putting a massive amount of cloud compute into the cloud with xbox series x hardware

 

[iroboto]

The cloud will be back don't worry about that. MS is putting a massive amount of cloud compute into the cloud with xbox series x hardware

well... i guess Flight Simulator is officially cloud powered. It's loading mips for you as you play.

 

[Kreten]

Okay.
Only in regards textures. There's a lot more data to be read, and written. Something like Nanite might stress it to the max. In fact, you'll never have an excess resource and devs will always use as much as you give them and them want more.

Weren’t there like 20MB/s streaming limits imposed on PS4/x1 even though HDDs themselves were 80MB/s. Now I know this was because even though consoles had USB3.0 you could plug in older USB2.0 drive and be stuck with as low as 20MB/s, so wouldn’t the same story happen with SSDs that games have to be developed for the slowest nvme that person might plug in despite it not being officially supported, just as USB2.0 drives weren’t?

 

[BRiT]

The only expansion system that will allow for Series X Games to be played is the Official Expansion Card. No need to be worried.

You can play legacy Xbox One X, Xbox One, Xbox 360, and Original Xbox from any external storage on the Series X.

 

[AlNom]

Weren’t there like 20MB/s streaming limits imposed on PS4/x1 even though HDDs themselves were 80MB/s. Now I know this was because even though consoles had USB3.0 you could plug in older USB2.0 drive and be stuck with as low as 20MB/s, so wouldn’t the same story happen with SSDs that games have to be developed for the slowest nvme that person might plug in despite it not being officially supported, just as USB2.0 drives weren’t?

IIRC, the caps were 40MB/s for OneS and 60MB/s for OneX. PS4 could go up to 100MB/s.