Alternative distribution to optical disks : SSD, cards, and download*

I replied with a link that the X1 just can't do partial install now, and MSFT is aware of the issue.

Then you replied and start claiming that somehow the quick boot up time has to do with the zlib hardware, and kept going back what Cerny said, which you had completely misunderstood.
And now you are saying that it wasn't what you were saying?

 

Because there would be nothing to look at in game for about that long? Leaving you in the main UI while data is being cached to RAM makes more sense than launching a blank screen you have to stare at for 40 seconds.

 

I've not been talking about the Xbox One at all. I don't own one, don't know anybody with one and have never used one. I am only talking about my experience with the PlayStation 4, which I do own, and throwing in what Mark Cerny has said about PlayGo. Everything I've said is about PlayGo. Which is why I keep mentioning PlayGo.

But the install process and hardware zlib aren't mutually exclusive. If you bother to read the interview with Mark Cerny, he explains it here.

 

Ya ya ya, whatever, when I said it's just software, you relied:

and kept quoting the same sentences again and again while not understanding what Cerny is saying, is not going to help your case, keep on trucking.

 

Let's recap. The discussion noted the slow install times on Xbox One then moved on to how quick it was on PlayStation 4. You posted that Microsoft was aware of the issue and that it's just software.

I replied saying that it's also a hardware issue. I was referring to PlayStation 4's implementation here and quoted Mark Cerny talking about PlayGo and the use of zlib decompression hardware, which I believe is partially to credit for fast installs on PlayStation 4. Anticipating this may be part of Microsoft's solution I asked if Xbox One also had this - I thought it had but wasn't certain.

I understand what Cerny is saying, it's very clear. And for the avoidance of doubt, this is my understanding of PlayGo working with a Blu-ray disc.

1. User inserts disc and the preliminary mandatory install begins. 2. When the minimum install has occurred, game can be launched and will run from HDD unless the game needs data from the Blu-ray, at which point that data is loaded from disc and also copied to the HDD. 3. When the game is not accessing the disc, the remainder of the game is copied (installed) to the HDD.

If you're only been talking about Xbox One then we're obviously talking about different things.

 

Can we ditch the bickering and turn this into an adult conversation? You may want to clarify your argument. I'm thinking that you're saying PS4 has not got a hardware advantage over XB1 for partial installs where you see DSoup as advocating this position. If so, the argument can be made a lot stronger by talking about the hardwares rather than talking about one's choice of sentences.

It's certainly possible that PS4 is architected to support 'through writes' to HDD from BRD. That little support chip that installs from downloads could, and should for consistency, be working the same from a disc as from a download, the difference being the disc represents a ~200mbps download speed. It might have enough cache for file IO without main RAM needing to be involved. It's a bit of an unknown at this point, unless I've missed some details in the tear-downs.

As such, the PS4 PlayGo install system would be designed to download the starting amount of data. This download will be too slow for realtime loading in the beginning. 1 GB may take, I dunno, 10 minutes. You can't have the game loading for 10 minutes and very slowly building up the main interface - it'd look ridiculous. So instead, you force a wait on the system menu to preload the first piece of the game.

Using exactly that same system, you'd precache the initial beginnings of game from BRD, only it'd take 40 seconds instead of 10 minutes. Then it's ready to launch. There's no reason to do it any other way. This way requires only one solution and is homogenous. Direct play from disc like a PS2 would require a second load and install system which just isn't necessary.

So, a 40 second initial install makes sense. The question then becomes whether the data is loaded from disk into RAM and then copied out to HDD, or copied to HDD and RAM at the same time, or copied to HDD independently from RAM and loaded from there via the game. I don't see how those particulars matter at all. It works and they're all much of a muchness.

XB1's requirement for larger installs seems to be a software issue as some titles have partial installs. That suggests either a manual implementation is needed by devs, or there are caveats.

 

I thought that typical game assets were either compressed beforehand, like textures etc or not very compress friendly. Using a hardware solution also gives us a nice little read boost from the harddrive since filling up 2GB might just involve reading 1GB with a 2:1 ratio..

 

Textures and sound aside, model and level data could very well appear to be loaded "uncompressed" and maybe even "ready to use" behind the zlib hardware.
Think of files that are paged from harddisk directly to ram. You could then walk the structure of a big city/universe/multimegaultraverse directly from ram, the OS will transparently load the missing data and purge out the ones you havent touched in a while.
64bit addresspace should be ample for those kinda things, fixed hardware would allow you to "bake in" pointers, the compression is handled by hardware specialised for it and Im sure lazy gamedevs would love it

 

Most of those things should be handled by the kernel and the MMU I think and disk caching is old hat.

Yes indeed with the address space you can have any file memory mapped. The wikipedia article on that seems decent, talks of "magical" things that are speculated here but that I guess have existed already for 20 years or more.

Talk about compression is done with every console gen since the N64 days. Slightly before it, Donkey Kong Country on the SNES decompressed graphical assets on the fly thanks to a special chip (and the base console supports ADPCM digital sound). Maybe that's similar to using S3TC textures.

 

Yeah, dint say anything is new at all, its just nice and comfortable to use if you have 64bit, fixed addresses for file (unlike generic OSes pointer could already be initialized) and a companion cpu + hardware for decompression.
Games still wont like the erratic delays, so its not as this would be the end to custom solutions

 

Sorry for asking in a technical forum
AFAIK the games are running off the HDD, which is why some modes are locked away upon the first boot until the full installation is complete.
Between that and how people was able to improve loading performance by going to faster HDD/SSD, it's pretty apparent how the data flow from and to at when.

The fact that there are multiple "theories" of how people describing how this works, from the pre-loading into RAM, to concurrent reading with zlib, to the possible free mirroring...
It means that no one actually knows how it works. Just speculations.

or Magic. Can't blame those who actually believe that you can install 40Gs of data under a minute

 

I have no idea what you're talking about. AFAICS no-one's claimed a full 40GB install in 40 seconds. It's always been about the partial installs and PS4 running a game from a small piece of the whole thing from disc (or download) where XB1 is needing more to be installed.

The thread topic is alternative distribution and talking about time to copy a game off a medium. PS4 and XB1 were raised to compare solutions, with PS4 proving there exists a fast partial-install system. How it goes about that isn't necessary information to continue the discussion on game media and downloads/installs. It's also not necessary knowledge for any gamer or dev as it works transparently in the background - devs would just need to know how to structure their distribution to use the system.

Ergo, speculations are all we need in this thread. There are enough ideas to show workable solutions for XB1 and future platforms. If people want to get into the nitty-gritty of why PS4's initial installs are lower than XB1s, that discussion deserves its own thread.

 

I ran my ps4 ssd through acronis to see how the ps4 formats stuff


Here is my normal hdd windows partition


I'm trying to find the cluster size the ps4 uses but it only shows sectors per cluster which is 57. Any way of knowing?

 

That's certainly an original file system name. Edit: If you run that name into a Japanese font, do you get a Japanese name? I suppose realistically there's no way of knowing, because you will get some characters and if they're random, we may not tell due to the interesting approach the Japanese often take to naming things. "Happy birthright within prism" may be gibberish or may actually be the name of the file system.

 

Judging from the weird sector size and cluster size, I'd say that the partition is not in a format that the application recognises. That explains why the file system name and other information (such as the number of total sectors) are gibberish.

 

You don't think the PS4's HDD has 1.4 quintillion sectors? Hmm, you may be onto something there...

 

It could be a block device encryption? That would make everything gibberish, including the partition's sector 0.

The PS4 is using a modified FreeBSD 9.0, so probably the default UFS file system, which have a default fragment size of 4k and a block size of 32k. But if it's encrypted, we'll never know.

 

I'll have a look at the weekend but yeah, it could be using a logical encrypted volume. It'll be easy enough to tell. Not only is such an approach more secure but the encrypt/decrypt implementation is a constant and doesn't impact fragmented file I/O, it's just a mild overhead of the filesystem.

Jaguar supports AES-NI so that would be the way to go.

 

Yeah i think i tried that and random characters still came up.

Be careful, make sure you back up your gamesaves to a usb because the moment you scan the hdd on your computer and insert it back in the ps4 you will have to format it again. It happened to me.

 

Thanks the warning - all my game saves are in Sony's cloud Hopefully I won't have any problems, I'll be using a hardware disc duplicator - it's a PC system designed to clone secure HDDs (BeCrypt, FoRSA etc) but one of the cool things is that you can [strike]mess with[/strike] analyse the filesytem from the source drive.