Digital Foundry Article Technical Discussion Archive [2013]

Status
Not open for further replies.
Err, what about SSD's? And as mentioned, the 4GB 360 and 12GB (flash) PS3?
Er, what about them? If you had an 8 GB SSD that you completely or mostly "refilled" several times a day, it would not last very long. Luckily, that's not how SSDs are typically used. It's also not how the flash on those consoles are used. (They seem carefully sized to be inadequate for use as a place to install disc games to, and that's the only conceivable use case that would result in lots of writing per day.)

Also can overprovision to extend life.
Can I? There's only 8 GB of flash here to play with, maximum. Are you recommending that only a fraction of that be used for any given game's cache? Since the entire pool is so small, I'm not sure that would be worth the effort. A ~2 GB flash cache sandwiched between 8 GB of RAM and 500 GB of disk? Sounds like a poor design. Imbalanced, if you will. Even a 6-8 GB cache is still on the skimpy side, IMO. 16 or 32 GB sounds better, but now we're starting to talk real money, and you probably would still need to limit individual game caches to no more than a quarter that to minimize wear.

Seriously, the only way the game caching scheme works is if you can come up with a user-friendly way of limiting how often the cache gets "re-built". For me and a lot of others, it would "just work", because I tend to play my "big" games pretty serially, without a ton of jumping back and forth. I'd also be fine with a "wear indicator" that tells me that I have X number of years left on my flash if I continue my current usage patterns. But that all assumes that I can choose to not cache games, and that assumes that game will still run okay with no caching, which in turn implies that developers will have to test their games both ways. Sounds like a giant PITA, frankly.

I can't believe I ever suggested it. :cool:
 
Er, what about them? If you had an 8 GB SSD that you completely or mostly "refilled" several times a day, it would not last very long. Luckily, that's not how SSDs are typically used. It's also not how the flash on those consoles are used. (They seem carefully sized to be inadequate for use as a place to install disc games to, and that's the only conceivable use case that would result in lots of writing per day.)


Can I? There's only 8 GB of flash here to play with, maximum. Are you recommending that only a fraction of that be used for any given game's cache? Since the entire pool is so small, I'm not sure that would be worth the effort. A ~2 GB flash cache sandwiched between 8 GB of RAM and 500 GB of disk? Sounds like a poor design. Imbalanced, if you will. Even a 6-8 GB cache is still on the skimpy side, IMO. 16 or 32 GB sounds better, but now we're starting to talk real money, and you probably would still need to limit individual game caches to no more than a quarter that to minimize wear.

Seriously, the only way the game caching scheme works is if you can come up with a user-friendly way of limiting how often the cache gets "re-built". For me and a lot of others, it would "just work", because I tend to play my "big" games pretty serially, without a ton of jumping back and forth. I'd also be fine with a "wear indicator" that tells me that I have X number of years left on my flash if I continue my current usage patterns. But that all assumes that I can choose to not cache games, and that assumes that game will still run okay with no caching, which in turn implies that developers will have to test their games both ways. Sounds like a giant PITA, frankly.

I can't believe I ever suggested it. :cool:

huh? an 8GB of flash might really be 12 or 16. But only 8GB is usable the rest is redundant.

Just like if you buy a 120GB SSD, you dont think there is only exactly 120GB of flash do you? There is redundancy to increase life.

For the 4GB and 12GB consoles, well the PS3 has many mandatory install games, so I assume one must install to the 12GB frequently (otherwise it would be a useless SKU out of the box)
 
They "handle it" by assuming that a customer is not going to delete and then re-download 4 GB of content multiple times a day. The use cases for the the 4 GB of storage in the 360 bear very little resemblance to those of a hypothetical "game data cache" on the XB1. In the latter, the cache would need to be emptied and re-filled every time you decided to play a different game. (This assumes that the flash can only hold a single game's cache at a time. I think that's a plausible assumption given the limited size of the flash as compared to the size of next gen games.)


So the 360 doesn't write to its hdd during gaming and only handles writes during install? And how do you think flash handles wear leveling with games installed. It writes that data to other parts of the memory to keep wear fairly even across the memory.

Plus, P/E figures are used to indicate when flash will start to wear out at a quicker rate not when flash will suddenly stop functioning. P/E figures are used to guarantee that a certain percentage of flash cells will remain viable until those P/E figures are reached. Wear leveling and error correction tend to allow flash to live far longer than they are rated.

Plus you are presenting a case where someone is changing full blown consoles games 1800 times a year where the average attachment rate of a console is like 10 games. Expecting MS to design around such a small population is like expecting MS to accommodate a population of desert dwelling people that like to game outside in 120 degree weather.
 
Last edited by a moderator:
I think TLC at 19nm is now 1000 P/E. It's getting worse at every node, and TLC really messed it up even more than MLC.

Apps cache and OS cache I think is the most likely use of the flash (the bloated stuff with ridiculous amount of seek, unoptimized apps). Game data is kind of borderline, I don't know. Game PVR is out of the question. With a perfect wear leveling algorithm from the OS (because the eMMC doesn't do it), let's assume a hardcore gamer plays 3 hours per day on average, the game recorder is 1MB/s HD video to a 8GB eMMC, that would brick the console in about 2 years. Wow, it's a wonderful planned obsolescence scheme I wish I thought about it and patented it. Make the warranty 1 year, and CASH IN :LOL: If they wanted to write to the flash that much they would have used SLC, just like the hybrid drives, no way around it. But for performance concerns, it's a negligible bitrate so I don't understand what the fuss is about, my guess is that they probably write it to the HDD.
 
Last edited by a moderator:
So the 360 doesn't write to its hdd during gaming and only handles writes during install? And how do you think flash handles wear leveling with games installed. It writes that data to other parts of the memory to keep wear fairly even across the memory.

No, AFAIK, the only time "games" write to the HDD on X360 is for saves and if you install a ame to HDD.

For the X360 with a 4 GB flash drive, that drive is primarily used for game saves and not for any game data unless it happens to be an XBLA game that happens to fit in that small storage pool. And even then, once installed the only time the game writes to the flash is when saving your game.

If it were used for caching of game assets then the flast would wear out extremely quickly if a user played multiple games. Not terribly uncommon if they play multiple multiplayer games along with the occasional single player game. Or if they are achievement hunting in multiple games while also playng multiplayer games. Or any number of other scenarios.

Scenarios, that Microsoft as a hardware manufacturer that warranties their devices have to take into account.

For next gen console, 8 GB is going to be entirely too small for a flash game cache pool that may potentially go through multiple write-erase cycles per day for every NAND storage cell depending on a user's game playing profile. And that's only from a write-erase endurance standpoint.

From a performance standpoint it'd have significantly better access latency, but worse sequential read speed than any modern 5400 RPM drive. One of the faster eMMC drives recently announced by Sandisk for use in Baytrail devices tops out at ~45 MB/s sequential reads. At that point you're roughly in the same balllpark as random read speeds with a 5400 RPM notebook drive even with the high penalty you pay for random access due to the high latency.

It makes the most sense, IMO, to use it to store the OS and OS side apps such that there is never any contention from the OS for the HDD when running a game. The OS and apps should be light enough that they won't be burdened by the relatively low sequential read speeds of the small flash storage pool. And the small size of those apps means that access latency becomes far more important in making them feel snappy (assuming they aren't always loaded in the OS memory partition).

Of course, that's just my speculation on what would be the best fit. For all we know they could be using high endurance SLC manufactured on 50 nm class or 60 nm class nodes (endurance goes down with smaller process nodes). But I find that highly unlikely.

Regards,
SB
 
Halo 3 dos some auto caching that breaks if you install the game, right?

No, where did you hear that?
The optical media layout is optimized for loading, so installing to hard drive (it's actually just a hard drive image, not really "installation" in traditional sense, meaning mandatory installation on the PS4) does not save loading time. Again, this is an exception, most games benefit from hard drive images.
 
Last edited by a moderator:
It's throttling on the hard drive's physical ability on the first map load, it doesn't break anything. The point is that Halo 3 does not require hard drive to play at all.

No, AFAIK, the only time "games" write to the HDD on X360 is for saves and if you install a ame to HDD.

Regards,
SB

SlimJim said:
I believe Halo 3 at least does not fit into that description

I don't see why you are arguing, he was correct about Halo 3, let it go, you will win the next one I'm sure.
 
No, AFAIK, the only time "games" write to the HDD on X360 is for saves and if you install a ame to HDD.

For the X360 with a 4 GB flash drive, that drive is primarily used for game saves and not for any game data unless it happens to be an XBLA game that happens to fit in that small storage pool. And even then, once installed the only time the game writes to the flash is when saving your game.

If it were used for caching of game assets then the flast would wear out extremely quickly if a user played multiple games. Not terribly uncommon if they play multiple multiplayer games along with the occasional single player game. Or if they are achievement hunting in multiple games while also playng multiplayer games. Or any number of other scenarios.

Scenarios, that Microsoft as a hardware manufacturer that warranties their devices have to take into account.

For next gen console, 8 GB is going to be entirely too small for a flash game cache pool that may potentially go through multiple write-erase cycles per day for every NAND storage cell depending on a user's game playing profile. And that's only from a write-erase endurance standpoint.

From a performance standpoint it'd have significantly better access latency, but worse sequential read speed than any modern 5400 RPM drive. One of the faster eMMC drives recently announced by Sandisk for use in Baytrail devices tops out at ~45 MB/s sequential reads. At that point you're roughly in the same balllpark as random read speeds with a 5400 RPM notebook drive even with the high penalty you pay for random access due to the high latency.

It makes the most sense, IMO, to use it to store the OS and OS side apps such that there is never any contention from the OS for the HDD when running a game. The OS and apps should be light enough that they won't be burdened by the relatively low sequential read speeds of the small flash storage pool. And the small size of those apps means that access latency becomes far more important in making them feel snappy (assuming they aren't always loaded in the OS memory partition).

Of course, that's just my speculation on what would be the best fit. For all we know they could be using high endurance SLC manufactured on 50 nm class or 60 nm class nodes (endurance goes down with smaller process nodes). But I find that highly unlikely.

Regards,
SB

Samsung emmc 4.5 flash is rated up to 150 MB/s sequential reads and 60 MB/s. it's emmc 5 is rated at 260 GBs.

So SHDDs which are offered at 500 GB HDD and backed by 8 GB of flash cache uses its cache only for the OS?

And why should a console need 3 GB of RAM and 8 GB for just its OS and system apps. Desktops don't need that much RAM when lacking any flash to allow its OS to remain snappy. Nothing I've seen makes me believe that the Xbox one OS needs that much memory when smartphones offer some of the same transition features while sharing ram with applications at sizes below what's allocated just to the XB1's OS.

And again P/E rates don't indicate when the flash memory in question will fail but the point where the failure rate of flash cells will increase. Wear leveling, ECC and over provisioning allows flash to remain viable much longer than their P/E would indicate especially when you use that figure as some sort of finite EOL stat.

The viability of the Flash as a texture cache is dependent on how much MS will allow any one game to allocate, how robust the wear leveling algorithm, ecc and over provision performs as well the expected usage of the vast majority of its userbase.

With virtual addresses you may not need to store the complete texture in a cache but only heavily accessed parts. Which would be far more useful to the game application versus the UI. It's a console first and foremost, not a Skype/Facebook machine. And I doubt a channel guide needs that much memory to provide better performance than your typical cable box.
 
So the 360 doesn't write to its hdd during gaming and only handles writes during install? And how do you think flash handles wear leveling with games installed. It writes that data to other parts of the memory to keep wear fairly even across the memory.

I'm not really following your question, but I'll try to make what I was saying more clear. Because of potential issues with wear, any game-caching scheme using the XB1's 8 GB of flash may need to be limited in one or even both of the following ways:

  1. The cache would need to use a write-once-read-many sort of strategy. You wouldn't, for instance, want to refill the cache every level, or every 10 minutes. To have any hope of this working, you're going to have to limit yourself to (re)filling the flash no more than once "per unique game played". I probably referred to that as an "install", since writing only happens, at worst, once per game played. At best, once per many game plays.
  2. You could choose not to fill the flash up "all the way" for a single game, and use, say, only 20% of the flash as an individual game cache. If you do that, and leave a significant chunk of the flash "empty", wear-leveling strategies could start to become very effective. However, I would now start to worry about the actual efficacy of a 1.5GB cache, when dealing with 20 GB games, streaming into 8 GB of RAM. At some point, it ceases to become worth doing. Still, it might be worth doing.

Wear leveling only works if you have some spare capacity. But at the same time you've got to have a "big enough" cache for it to do any good. How big is big enough? I don't know. This isn't like a RAM cache on a HDD. Those get away with being fairly tiny since they are constantly and preemptively refilling themselves. Flash doesn't have that luxury, unless it is extremely "overprovisioned". Do you think the XB1's "8 GB" flash cache is really a 32 GB part? 'Cause if so, I happily withdraw my objections.

Plus, P/E figures are used to indicate when flash will start to wear out at a quicker rate not when flash will suddenly stop functioning. P/E figures are used to guarantee that a certain percentage of flash cells will remain viable until those P/E figures are reached. Wear leveling and error correction tend to allow flash to live far longer than they are rated.

Well, that's good to hear. How much further are we talking? 4 times? Say to 12,000 writes? Assume 6 GB caches. That might make "big" game caches survivable, even in nearly worst case scenarios. Fantastic news.

Plus you are presenting a case where someone is changing full blown consoles games 1800 times a year where the average attachment rate of a console is like 10 games. Expecting MS to design around such a small population is like expecting MS to accommodate a population of desert dwelling people that like to game outside in 120 degree weather.

I'm sure MS does test their consoles in various "adverse" environmental conditions, and if the hardware can't handle 120F, MS will include a disclaimer somewhere in the fine print. Do you recommend that they include a similar disclaimer about playing too many games within the life of the console? They certainly could do that, and the limits would seem comfortably high to someone like me, but oh can you imagine the shitstorm they would get from the usual suspects on the internet? It's a perception issue, mostly. Nobody wants to know that the clock is ticking on their purchase, even if it's ticking pretty slowly for most.

And what does attach rate have to do with switching games? It takes a grand total of two games to run up any arbitrary number of switches. Just have little Jimmy and Jamie fighting back and forth between two beloved games ad infinitum. Add in some Mom-time and Dad-time, and it really doesn't seem that preposterous to have a "well used" console, switching-wise. I'm not saying it's normal use, just that it seems like it could happen. And that means it will happen. If you can think of a way for MS to offer this feature, while simultaneously protecting themselves from the outrage those folks with what you consider to be "unrealistic" expectations will express, I'm all ears. Believe me, I'm the one that proposed this feature. I want it to be possible. It would work for me just fine.

Hmmm. Maybe they could make it a secret challenge. If you manage to put your cache through 9000 cycles, and your XB1 manages to phone this news home, you win a new console! It won't happen that often, and when it does happen, it's non-damaging to perceptions. :D

Or, you know, just stick seldom updated OS and app stuff in the flash instead. One of those two.
 
Its eMMC controller is rated at that. The drive itself won't come close to approaching that speed.

Regards,
SB

No EMMC 4.5 is generally rated as supporting speeds up to 200 MB/s. Samsung fastest emmc 4.5 part is a 16 GB part rated at 170 GBs for sequential reads while they have slower rates for other emmc 4.5 products at the same and different memory sizes.
 
Last edited by a moderator:
I'm not really following your question, but I'll try to make what I was saying more clear. Because of potential issues with wear, any game-caching scheme using the XB1's 8 GB of flash may need to be limited in one or even both of the following ways:

  1. The cache would need to use a write-once-read-many sort of strategy. You wouldn't, for instance, want to refill the cache every level, or every 10 minutes. To have any hope of this working, you're going to have to limit yourself to (re)filling the flash no more than once "per unique game played". I probably referred to that as an "install", since writing only happens, at worst, once per game played. At best, once per many game plays.
  2. You could choose not to fill the flash up "all the way" for a single game, and use, say, only 20% of the flash as an individual game cache. If you do that, and leave a significant chunk of the flash "empty", wear-leveling strategies could start to become very effective. However, I would now start to worry about the actual efficacy of a 1.5GB cache, when dealing with 20 GB games, streaming into 8 GB of RAM. At some point, it ceases to become worth doing. Still, it might be worth doing.

Wear leveling only works if you have some spare capacity. But at the same time you've got to have a "big enough" cache for it to do any good. How big is big enough? I don't know. This isn't like a RAM cache on a HDD. Those get away with being fairly tiny since they are constantly and preemptively refilling themselves. Flash doesn't have that luxury, unless it is extremely "overprovisioned". Do you think the XB1's "8 GB" flash cache is really a 32 GB part? 'Cause if so, I happily withdraw my objections.



Well, that's good to hear. How much further are we talking? 4 times? Say to 12,000 writes? Assume 6 GB caches. That might make "big" game caches survivable, even in nearly worst case scenarios. Fantastic news.



I'm sure MS does test their consoles in various "adverse" environmental conditions, and if the hardware can't handle 120F, MS will include a disclaimer somewhere in the fine print. Do you recommend that they include a similar disclaimer about playing too many games within the life of the console? They certainly could do that, and the limits would seem comfortably high to someone like me, but oh can you imagine the shitstorm they would get from the usual suspects on the internet? It's a perception issue, mostly. Nobody wants to know that the clock is ticking on their purchase, even if it's ticking pretty slowly for most.

And what does attach rate have to do with switching games? It takes a grand total of two games to run up any arbitrary number of switches. Just have little Jimmy and Jamie fighting back and forth between two beloved games ad infinitum. Add in some Mom-time and Dad-time, and it really doesn't seem that preposterous to have a "well used" console, switching-wise. I'm not saying it's normal use, just that it seems like it could happen. And that means it will happen. If you can think of a way for MS to offer this feature, while simultaneously protecting themselves from the outrage those folks with what you consider to be "unrealistic" expectations will express, I'm all ears. Believe me, I'm the one that proposed this feature. I want it to be possible. It would work for me just fine.

Hmmm. Maybe they could make it a secret challenge. If you manage to put your cache through 9000 cycles, and your XB1 manages to phone this news home, you win a new console! It won't happen that often, and when it does happen, it's non-damaging to perceptions. :D

Or, you know, just stick seldom updated OS and app stuff in the flash instead. One of those two.

Look I'm not looking at this as "it doesn't effect me so I'm all for it as I don't care about its effect on other gamers." I'm looking at this as MS (most big corps operate in this manner) is readily willing to inconvenience small fractions of the users if it see greater benefit to the overall userbase especially if the added feature creates additional revenue.

Live came with a price tag when every other platform offered multiplayer for free. MS also choose not to support dial up to ensure performance of Live services. High price proprietary HDD drives. No wireless support for third party controllers. Was ready to abandon the no internet/spotty internet crowd. Throwing even more stuff behind the paywall this gen. MS seems to have no qualms about not accommodating small fractions of the market if it see benefits for its more general users or itself.

My 360 has never touched the Internet. And mainly because I'm not fond of paying for multiplayer and I refused to pay what I considered an absorbent amount for a network adaptor. I've been on the side of that equation before. I'm just willing to accept that my behavior or my beliefs don't always coincide with the beliefs and behaviors practiced by the vast majority of the userbase. And sometimes I am inconvenienced by it. I've faced it would basically with every console I owned. I consider a natural part of the console market. It is what it is.

Plus, MS can choose to limit the amount allocated to any one title to accommodate a number of titles to reduce the effect of title switching on a daily basis. A game like GTA 5 could you the cache to increase the rate in which car models appear in the world without burdening the HDD. That could include character models and other objects. The data is only erased and rewritten when the gamer switches to a game not currently inhabited by the cache.

My belief is that the cache is pretty large and flash would be far more beneficial to the game apps themselves to restrict it solely to the OS. I think there are ways to accommodate pretty much both without turning it into some type of write buffer or storage and have it die early on an appreciable portion of the userbase.

When talking about P/E cycles consumer grade 2X nm MLC can exceed 2X - 3X it's P/E rating before the number of bit errors due to wear overcome the ECC and other mechanism meant to extend the life of flash. For enterprise MLC it's more lik 5X. For SLC rated at 100k there are products that will readily approach 1 million P/E cycles.
 
Last edited by a moderator:
No EMMC 4.5 is generally rated as supporting speeds up to 200 MB/s. Samsung fastest emmc 4.5 part is a 16 GB part rated at 170 GBs for sequential reads while they have slower rates for other emmc 4.5 products at the same and different memory sizes.

No matter what they claim there won't be an implementation in the real world that exceeds 45-50 MB/s without heavy memory caching on the part of the system (some Android devices do this but most don't). They've made the same claims in the past and in the past the devices were still limited to ~45-50 MB/s in real world sequential reads. What's even worse is that when thrown real world workloads, random reads for the those devices often came in at 10-15 MB/s. At which pont they perform worse in many cases than a 5400 RPM drive. Whch should not ever happen with NAND devices. I can only assume there is something about eMMC that is bottlenecking real world performance in some way. Of course, it doesn't matter much if it performs worse than a 5400 RPM notebook drive, because you can't use a mechanical drive in most devices that need an eMMC based flash drive. And with the small file sizes of most mobile devices apps, the access latency is far more important than how fast the file contents can be transferred.

There's a reason why mSATA drives are used for performance and you never...ever...see an eMMC drive used for performance. I suppose you could make an argument that all real world eMMC device's just have sucky eMMC interfaces while they have good mSATA interfaces. But the end result ends up being the same.

Regards,
SB
 
No matter what they claim there won't be an implementation in the real world that exceeds 45-50 MB/s without heavy memory caching on the part of the system (some Android devices do this but most don't). They've made the same claims in the past and in the past the devices were still limited to ~45-50 MB/s in real world sequential reads. What's even worse is that when thrown real world workloads, random reads for the those devices often came in at 10-15 MB/s. At which pont they perform worse in many cases than a 5400 RPM drive. Whch should not ever happen with NAND devices. I can only assume there is something about eMMC that is bottlenecking real world performance in some way. Of course, it doesn't matter much if it performs worse than a 5400 RPM notebook drive, because you can't use a mechanical drive in most devices that need an eMMC based flash drive. And with the small file sizes of most mobile devices apps, the access latency is far more important than how fast the file contents can be transferred.

There's a reason why mSATA drives are used for performance and you never...ever...see an eMMC drive used for performance. I suppose you could make an argument that all real world eMMC device's just have sucky eMMC interfaces while they have good mSATA interfaces. But the end result ends up being the same.

Regards,
SB

Tell that to the Samsung Galaxy 7/10 (2013 parts) or LG G2 which benchmarks well beyond what you are stating for seq reads.

Mobile devices aren't really an enviroment that readily needs read and write performance of a laptop drive or higher. There is not that big of incentive to use the higher end controllers when your target device is most limited to light weight tasks. 4.41 volumes are probably still much higher than 4.5 and those devices with 4.5 are probably targeting the cheaper versions. And at the end of 2012, 16 GB of 4.41 normally ran in the $6 range. I doubt high end 4.5 memory can be had for as cheaply. EMMC is not only used for mobile devices other product markets like fpga use EMMC and most likely support higher end parts. EMMC 5.0 is in mass production but I haven't heard of any mobile device using them. Samsung is selling those devices to someone.

Do you need robust random read performance when talking about a texture cache?

I am not saying you are definitely wrong and I am definitely right. But using 8 GB of flash for just OS purposes on a lightweight OS (I am highly doubtful that XB1 comes anywhere close to Windows in terms of functionality and utility). I doubt snapping or skype needs that much memory.

There is the concept of downloading games onto flash in an effort to reduce the amount of time the HDD has to spin up, so that DD doesn't end up prematurely killing drives especially when you are talking slow internet connections. But that in itself shouldn't warrant 8 GB of flash.
 
Last edited by a moderator:
No, AFAIK, the only time "games" write to the HDD on X360 is for saves and if you install a ame to HDD.

Actually, on 360, the system gives a portion of HDD so it can cache data. Pretty much any game on 360 uses this, (or at least has the choice to use).

If I'm not mistaken, the system maintain the cache for the last 3 games you played, removing the older one when you play a 4th.
 
Status
Not open for further replies.
Back
Top