Next Generation Hardware Speculation with a Technical Spin [pre E3 2019]
- Thread starter AlNom
- Start date
-
- Tags
- 5 bytes non sequitur
- Status
yes.Just because your guesses were bad...
I am more reserved as a result of such failings lol.
But you are right in that the concepts of storage acceleration is definitely where teh discussion needs to be.
I don't have a doubt Sony is working on something/ has something.
How it works, and how much it actually does is what I'm interesting in reading about. There are as I can see a variety of methods presented, and SSD is definitely among the leading reason backed in great degree which honestly I'm among them as well, but I'm not ruling out other probabilities.
As I put SSD forward in my mind, it only makes me want to pull back on other items on the console. If we push for a cheaper solution, perhaps the idea that the console could be more performant is higher.
Last edited:
I think the key factor in my mind is if you believe that they will continue to allow user-upgradeable storage in the form of internal/external mechanical hard drives. If they do, a caching setup would seem most likely as it would be, IMO, the most user-friendly solution. Whether that hybrid setup is shipped in the console is another question, but I expect that the consoles will all be able to do it. It doesn't seem as if it would be hard to implement, after all.
D
Deleted member 11852
Guest
PlayStation has offered user upgradable storage since the ethernet/HDD adaptor for PS2 and I hope the next PlayStation offers it too. But if that is the price for virtually no loading I will gladly pay it.
PlayStation has offered user upgradable storage since the ethernet/HDD adaptor for PS2 and I hope the next PlayStation offers it too. But if that is the price for virtually no loading I will gladly pay it.
Well, they could allow it, but require the drive to meet a performance profile (only SSDs, only fast SSDs). That would not be very user friendly, so I'm not expecting this to be the approach. And I'm not expecting them to remove a feature from last-gen as that's always a bad look. Call me an optimist, but I choose to assume the people making these decisions won't make bad choices. There's plenty of time to rage about their stupidity once they actually demonstrate it.
Transparent caching have cache misses, which I don't think a game streaming can tolerate. It creates inconsistent performance.
It seems so much simpler to tag the data which is used for streaming, and allow devs to be 100% sure of the streaming performance from the storage layer.
Then when starting a section of the game it simply asserts everything it needs (the entire map, or level) is in the flash area. They can always have storage management tools in the API to automate it.
It seems so much simpler to tag the data which is used for streaming, and allow devs to be 100% sure of the streaming performance from the storage layer.
Then when starting a section of the game it simply asserts everything it needs (the entire map, or level) is in the flash area. They can always have storage management tools in the API to automate it.
I don't belive that Cerny talked about plain SSD, things he said point to something different. I would say some 100GB of storage for content that need to be accessed quickly, question is what interface (maybe something exotic?) and what type of flash it would be. Maybe it would be soldered directly to motherboard, if they are confident about lifespan.
D
Deleted member 13524
Guest
The full paragraph of the sentence you quoted is:
It doesn't matter what hardware was in that devkit to achieve that. What matters is the practical result of the near instant loading between scenes, which is a result of faster storage and very fast decompression to the RAM.
I don't know if that devkit has a 64 core EPYC or a dedicated Vega 7 to brute force the decompression from mass storage to RAM, or if it has a dedicated FPGA, or an engineering sample of the final SoC that has dedicated hardware for that. And that doesn't really matter that much at this point, IMO.
What matters is they're promising a final result of practically eliminating loading times on the consoles they'll be shipping next year. This is of course made possible by the advent of cheaper and faster mass storage, but that tech in itself isn't new.
My guess is console makers didn't really worry about accelerating asset decompression because so far they had to settle with the ~50MB/s common denominator from HDDs, so allocating some percentage of a single CPU core was more than enough. The moment they get from 50MB/s on a HDD to over 1000MB/s on a NVMe drive, that's when they need to worry about some kind of hardware acceleration and/or heavier optimization.
Another important consideration about how modern games data is organized currently.
Most AAA games have open-world-likeish engines by now, even for linear games. That means, they rely heavily on real time streaming to allow for much greater variety within one level than they can fit in memory at one time. A robust streaming and memory management system also allows for clever things like to starr pre loading the next level as you aproach current level end, or keeping data from the last checkpoint always in memory for insta re-start after death (ND is good at this).
These streaming and memory management systems are buitl under the assumption of running on the low-end HDDs of PS4/XBO. That means, they are optimized for slow read speeds, yes, but most of all SLOW SEEK SPEEDS, which is many times slower than the max read spead. That means, their engine, and mostly the data layout is optimised to reduce data seeks from storage. Data is, as much as possible, packed into large blocks that are read all at once and decompressed all at once.
These blocks are often conservative, in that, they read more data than the game actually needs at the moment, but once the HDD is at that track, its faster to just read that bugger than skip it because that entails another super slow seek.
Also, these blocks often contain redundant copies of data already present in different blocks. That is because, again, given seek high latency, it's faster to read redundant data more than once than to expect the HDD to jump from one side of the HD to another all the time.
One can have an idea of how that is done by reading Insomniac's GDC presentation on streaming on SpiderMan. They have some hard numbers in there about this stuff.
So, given that, what changes for a game built from the ground up to run on an SSD?
No more worries about seek time latencies. No need to pack data into large monolithic blocks (well, kind of, segmenting data too much might affect compression efficiency). Game can load only what is needed, when it's needed.
It has already been mentioned how this would save a lot of fast memory, but this also has important effects on storage.
Without the need to store redundant data to avoid seeks, we might see game files go down (!!!) or at least, they will have more MEANINGFUL data without having to require bigger files.
As an aside, within the topic of slow seek speed. Once you realize seek latency is what detracts the performance of HDDs much more than their actual max read speed (which while indeed slower than SSD's is actually not that terrible overall) the idea of a hybrid large size but cheap (and user replaceable) HDD + medium size SS "cache" sounds more effective.
Loading the game into the cache may not be as slow as some think, given it requires only ONE seek (ignoring fragmentation). Once it starts loading the game, the drive will be operating at maximum performance the whole time. That might actually be faster than the bootup of some badly optmised open world games (and some well optmised ambitious ones) because even if the bootup is not loading the whole game data, its reading data from many different areas, requiring many dozens of seeks.
Most AAA games have open-world-likeish engines by now, even for linear games. That means, they rely heavily on real time streaming to allow for much greater variety within one level than they can fit in memory at one time. A robust streaming and memory management system also allows for clever things like to starr pre loading the next level as you aproach current level end, or keeping data from the last checkpoint always in memory for insta re-start after death (ND is good at this).
These streaming and memory management systems are buitl under the assumption of running on the low-end HDDs of PS4/XBO. That means, they are optimized for slow read speeds, yes, but most of all SLOW SEEK SPEEDS, which is many times slower than the max read spead. That means, their engine, and mostly the data layout is optimised to reduce data seeks from storage. Data is, as much as possible, packed into large blocks that are read all at once and decompressed all at once.
These blocks are often conservative, in that, they read more data than the game actually needs at the moment, but once the HDD is at that track, its faster to just read that bugger than skip it because that entails another super slow seek.
Also, these blocks often contain redundant copies of data already present in different blocks. That is because, again, given seek high latency, it's faster to read redundant data more than once than to expect the HDD to jump from one side of the HD to another all the time.
One can have an idea of how that is done by reading Insomniac's GDC presentation on streaming on SpiderMan. They have some hard numbers in there about this stuff.
So, given that, what changes for a game built from the ground up to run on an SSD?
No more worries about seek time latencies. No need to pack data into large monolithic blocks (well, kind of, segmenting data too much might affect compression efficiency). Game can load only what is needed, when it's needed.
It has already been mentioned how this would save a lot of fast memory, but this also has important effects on storage.
Without the need to store redundant data to avoid seeks, we might see game files go down (!!!) or at least, they will have more MEANINGFUL data without having to require bigger files.
As an aside, within the topic of slow seek speed. Once you realize seek latency is what detracts the performance of HDDs much more than their actual max read speed (which while indeed slower than SSD's is actually not that terrible overall) the idea of a hybrid large size but cheap (and user replaceable) HDD + medium size SS "cache" sounds more effective.
Loading the game into the cache may not be as slow as some think, given it requires only ONE seek (ignoring fragmentation). Once it starts loading the game, the drive will be operating at maximum performance the whole time. That might actually be faster than the bootup of some badly optmised open world games (and some well optmised ambitious ones) because even if the bootup is not loading the whole game data, its reading data from many different areas, requiring many dozens of seeks.
Last edited:
I like the idea of rapid decryption to memory etc, I hope there's more expansion on this idea.
I mean, on this thought process, how do we know we aren't seeing HBCC (maybe a customized variant of) in action here?
They presented a loading metric, I'm not exactly sure if Sony promised it to be elimination of load times across the board. I wouldn't hold them to it, seems like a large undertaking.
Fair point. I'll be the first to admit I was grossly uninformed of current price trends for that tech. If the price realities are as you say, I think it is more likely that sony will forego a sophisticated but complicated tiered storage system, even if that means sacrificing support for industry standard user replaceable HDs and complications for use of external USB drives. This is exactly the kind of oversight I would expect.
Last edited:
yea that was my big oversight as well; if prices are that cheap, go for a simpler more reliable solution.
Vega SSG -- 2 TB SSD built into the video card itself to maximize HBCC with 16 GB of video memory.
https://www.amd.com/en/products/professional-graphics/radeon-pro-ssg
edit: Did a quick search and Kaotik and DmitryKo discussing these exact concepts back in Jan. Full circle here.
Last edited:
The cost provisions for main storage in a console's BOM have been between $30 and $40 since 2013. It never went up. New baseline capacities were based on improved platter density.
SoC costs should go up since 2013 (7nm is expensive, more masks, more steps) so a bigger part of the BOM will have to go there (ps4 was $100 for the 1.8TF SoC, $88 for memory). Also, 1TB is already borderline inadequate for current consoles (see GT7 and RDR2 install sizes, or spiderman with dlcs, try to predict Death Stranding or TLOU2), and that's current gen on an 8GB platform!
It's really difficult to imagine a 2TB NVMe drive anywhere near $40 considering even Apple paid that much for 128GB of MMC in their iPhones last year. We're looking at 512GB being quite possible after the ongoing price drops, 1TB being optimistic and require a BOM compromise, and 2TB requiring a dramatic move beyond any sort of SSD supplier contract predictions.
SoC costs should go up since 2013 (7nm is expensive, more masks, more steps) so a bigger part of the BOM will have to go there (ps4 was $100 for the 1.8TF SoC, $88 for memory). Also, 1TB is already borderline inadequate for current consoles (see GT7 and RDR2 install sizes, or spiderman with dlcs, try to predict Death Stranding or TLOU2), and that's current gen on an 8GB platform!
It's really difficult to imagine a 2TB NVMe drive anywhere near $40 considering even Apple paid that much for 128GB of MMC in their iPhones last year. We're looking at 512GB being quite possible after the ongoing price drops, 1TB being optimistic and require a BOM compromise, and 2TB requiring a dramatic move beyond any sort of SSD supplier contract predictions.
And I say, why not both? Ship with all flash storage and allow for adding bulk storage via mechanical drives. Not allowing people to add cheap bulk storage to their console is an unnecessary sacrifice, IMHO, and if one does and the other doesn't allow it, it's going to be a bad look for the one that doesn't.
anexanhume
Veteran
Isn’t QLC substantially slower than TLC, and because of the increased number of states to resolve for the cells, it’s a physical limitation - not a question of product segmentation or yield.
I don't see why this has to be true in any way. I think simply demanding games to be installed, and installed means "on the internal NVMe drive", would work just fine.
What I'm predicting is explicitly just a NVMe flash drive, which can easily be user-replaceable.
The total amount of written storage per dollar you get is more if you buy QLC than if you buy a small amount of SLC or MLC. You just also get more capacity, so it's less in terms of full drive writes. It's really hard to spend all the writes on a drive unless you stream a lot of video on it, or use it as a cache. The few megabytes of save data really *don't* pile up, not even if you save every second, and even the "daily suspend of 32GB" is just 0.000007% of the write budget of a 1TB 500-full-writes drive. Assuming even halfway decent write leveling, the only way it is actually possible to spend the full write budget of the drive is to use it as a cache of some sort, or stream video on it.
NVMe is the industry standard interface, and by late 2020 most new consumer drives sold will likely use it. Right now it's still in ramp-up, but uptake is rising rapidly. If they want to, they could easily fit two NVMe slots in the machine, to allow not just user-replaceable but also user-expandable storage, without having to toss the existing drive.
What I'm proposing is not anything as radical as Sony going "you have to use our proprietary, non-replaceable storage". It's just them saying "NVMe is the new industry standard, and it gives much better performance. You now have to use it for games. Also, you should use a PCIe-4 one for best performance".
Beyond just using NVMe, I suspect that they have a bit of "special sauce" designed to make the flow of streaming data from the drive to ram be as smooth as possible, maybe even with hardware units that uncompress/unpack some common formats or something. Another possibility that helps is just using a huge blocksize in the drives. This isn't really a detriment for almost any gaming loads, but will reduce the overhead of transferring data.
But he showed the problem is about giving devs a guaranteed performance which is sufficiently stable to design a game on this guarantee for streaming, specifically how fast you can move through an open world game. Doing one of the speed run trials in spiderman must not be dependent on the user's chosen drive and it must not create bugs. So the game must limit the speed based on worst case performance of the underlying storage layer.
It cannot be optional, it cannot use external mechanical hdds. This is less about the loading time, and more about real time guarantees. It's why he thinks this is a game changer, it allows new types of game design.
Even fast travel, if you make an online multiplayer shooter big map where you can use teleports points, you can't have different loading times based on users drives speed. It needs to be on the predictable local fast storage for anyone playing the game to avoid competitive advantages.
It cannot be optional, it cannot use external mechanical hdds. This is less about the loading time, and more about real time guarantees. It's why he thinks this is a game changer, it allows new types of game design.
Even fast travel, if you make an online multiplayer shooter big map where you can use teleports points, you can't have different loading times based on users drives speed. It needs to be on the predictable local fast storage for anyone playing the game to avoid competitive advantages.
Again, and this really shouldn't need repeating at this point, this wasn't a technical expose. It was a first look. Absolutely nothing shown was false, but that doesn't mean the end result is going to perform the same, same as God knows how many demos and technical showcases!!
It's just a simple statement of fact that between a preview and final product, things don't necessarily look the same, without any deceit being involved (sometimes it is, due to Marketing, but sometimes it ain't).
Cerny is excited about his storage solution, and fast loading, and showcased a great example that, running on the devkit, shows how a good storage solution can be way faster. If PS5 ends up taking 2.2 seconds to load instead of 0.8, it wasn't a lie, because PS5 wasn't shown - only the devkit and the demo. And if PS5 takes 2.2 seconds to load instead of 0.8, that's still bloody brilliant as opposed to 15 seconds!
No-one should be making an argument that we takes these figures as gospel, especially in a technical prediction thread. We should still be discussing possibilities and scenarios, like NAND costs. If you aren't happy with ideas being questioned, please just refrain from conversing and let those who do want to discuss the viability of different solutions and interpretations to engage in their discourse without interruption.
Last edited:
- Status
Similar threads
