Playstation 5 [PS5] [Release November 12 2020]

[orangpelupa]

Unless PS5 is doing something weird or stupid, I would expect updates to mostly be writes. The patch file will include new data and that will be writing over existing data, or much more likely, written to new/least-used cells with with the controller and filesystem keeping track of where the data is and how is all linked. There will be reads too, of course, because you need to ensure what you are writing is written successfully.

The reason some (not all) PS4 game updates result in a temporary spike in disc usage during updating is because after a number of updates, or more likely when the PS4 filesystem detects a certain degree of fragmentation, the filesystem will decide to create a new contiguous version of the game install on the disc. To do this, may meaning around game installs around to create a space big enough. Anybody who has written a defragmentation algorithm will know how convoluted and complex this can be become. You may need to move other game installs around more than once while you reorganise the physical distribution of data on the platter, just so that you can make a big enough space for your freshly updated game.

I very much expect this normal PS4 behaviour to be completely mitigated on PS5, but this would create every weird virtualised disks for PS4 games. Assuming that is how they are stored.

PS5 changed the update method for PS4?

On PS4, it use patch on copy or something. I can't remember the name. Hench the very tedious "copying" step

 

[Deleted member 11852]

PS5 changed the update method for PS4?

For PS4 games I believe 'copying' is the process of extracting and preparing all the patch files and that would still need to happen. Would I would expect PS5 to no longer do to spend time creating contiguous files for game installs unless you are using an external spinning HDD for PS4 games. For SSDs and, more importantly, the built-in and any installed internal NVMe drive this behaviour would introduce pointless reads/writes that are unnecessary and would only contribute to cranking up read-write cycles of the cells.

 

[Shifty Geezer]

Copying is literally the copying of the entire game folder and files. Sony chose to play it safe by patching a copy and then swapping the game reference to the new patched duplicate file and then remove the original. This is why PS4 games need excessive free drive space to download a small patch, because there needs to be enough free space to have a complete duplicate of the whole game. The reasoning is that if a patch goes wrong, you have screwed up the game file and need to download it. Well, the chances of that are minimal and downloading these days is potentially quicker than copying.

Cerny intonated that things were different with PS5 and your report that patches are only seconds long suggests to me they are using a conventional update process now. But then for PS4 updates, these are trapped in the legacy process and with a filesystem overhead crippling copy performance on the local SSD, it seems.

 

[Deleted member 11852]

Copying is literally the copying of the entire game folder and files. Sony chose to play it safe by patching a copy and then swapping the game reference to the new patched duplicate file and then remove the original.

I don't believe this the case. Copying is always the first action, literally, for PS4 games patching and the first action would need to be extracting the files from the patch and many patches can be ridiculously large. PS4 isn't using memory because patching can happen when games are running or are in sleep state in RAM.

Some PS4 patches do simply overwrite the current file, those that don't are more likely to be where the patch (just a diff file) needs to insert/add data mid-way through files, e.g. AABBCCDD to AABBBBBBCCDD - unless there is space at of the drive region to relocate 'CCDD' further along and insert 'BBBBBB' and keep the file contiguous, because this is quicker than creating a whole new file and the more games you have, the less space you have to move things around; the classic filesystem defragmentation problem.

If the patch is AABBCCDD to AAbbCCdd then the file is just over-writen with thew new changed data.. Some big patches can be applied quicker than some smaller patches and this is why PS4 feels so inconsistent at times. In additional PS4's linux filesystem does aim to keep entire game installs contiguous - as much as possible. But it does not do this for every install or patch, it seemingly depends - presumably on the level of fragmentation and where on the drive (inner/outer rings) data is located.

This were a presentation or event where Mark Cerny explained much of this in the context of why PS5's game patching would be faster. The big unknown is what is the PS4 patch behaviour is on PS5, because you don't want to be doing any of this nonsense on most solid state drives.

 

[JPT]

In additional PS4's linux filesystem does aim to keep entire game installs contiguous

I assume that BSD can use many different file systems, but it seems weird that they would use a Linux one. I always had the feeling that BSD "people" thinks Linux stuff is not good enough. Then again PS5 is a Sony product so they might hobble cobble anything together
Or did I miss that PS5 is not using BSD?

 

[Deleted member 11852]

I assume that BSD can use many different file systems, but it seems weird that they would use a Linux one. I always had the feeling that BSD "people" thinks Linux stuff is not good enough. Then again PS5 is a Sony product so they might hobble cobble anything together Or did I miss that PS5 is not using BSD?

Outside of Windows, where are you tied to NTFS for the operating system, most other operating systems will run on a variety of filesystems that support a minimum feature set. BSD and linux define the kernel and not the filesystem. PS4 uses a variation of what is the most common filesystem used across linux distros, which is also used used by some BSD options but which is far less common because ZFS is pretty much the default on modern BSD and that's the only reason I refer to it as a linux filesystem.

I have no idea what the OS is for PS5 but it's was FreeBSD for PS3 and PS4/Pro, so....

 

[Arwin]

For the PS5 the file system must be something very customized though I assume what with that being such a major part of the innovation?

 

[Shifty Geezer]

I don't believe this the case. Copying is always the first action, literally, for PS4 games patching and the first action would need to be extracting the files from the patch and many patches can be ridiculously large.

I think the labels on PS4's information during the process are just inaccurate. What we do know is that given a 50 GB game with a 0.4 GB patch, you need 51 GB of free disc space to update. The 50 GB original game is copied to a new folder. The patch is applied either during the copying process or afterwards, maybe with the patch being decompressed first to another folder of temp files; that bit's unsure. Finally, the game on the OS is linked to the new folder and the old folder deleted, freeing up 50GB of space so no more has been consumed, but it was necessary for the patching process.

This contrasts with every other patching system out there that works with the original folder, just patching files as necessary, resulting in no need for large free space to apply small patches, and much quicker execution.

 

[Deleted member 11852]

I think the labels on PS4's information during the process are just inaccurate. What we do know is that given a 50 GB game with a 0.4 GB patch, you need 51 GB of free disc space to update. The 50 GB original game is copied to a new folder.

The labels are certainly inaccurate but it's not true that applying a 0.4GB patch to a 50GB game install requires 51GB of freee space for a second copy. The PS4 is definitely not making an entire copy of an every game during every patch. There are many games over 50GB (or even 20GB) where, if this were the case, patching would require a crazy amount of times on PS4's standard 5400rpm HDD.

We know form the Spider-Man PS4 postmortem that PS4's stock drive as a 50MB/s READ speed but that some user replaced drives have closer to a 20MB/s READ speed. Write speeds are typically much slower so figure how long a copy - lots of reads and writes - would the patching always made a second copy. If a patch is just replacing/overwriting a bunch of bytes in the middle of a file - e.g. AABBCCDD to AAbbccD then there will be no copy, just open-seek-write-close operation for the changed data. This is why some patches are über quick, near instant.

The patch is applied either during the copying process or afterwards, maybe with the patch being decompressed first to another folder of temp files; that bit's unsure. Finally, the game on the OS is linked to the new folder and the old folder deleted, freeing up 50GB of space so no more has been consumed, but it was necessary for the patching process.

The initial copy process is decompressing the patch files. PS4's filesystem (PFS) bears quite a close resemblance to EXT4 with some changes made by Sony. The console, OS and filesystem have been heavily pulled apart at this point there are lots of places to find out more but I don't think I permitted to link to those types of sites on B3D. A little googling is all it takes.

 

[Shifty Geezer]

We know form the Spider-Man PS4 postmortem that PS4's stock drive as a 50MB/s READ speed but that some user replaced drives have closer to a 20MB/s READ speed. Write speeds are typically much slower so figure how long a copy - lots of reads and writes - would the patching always made a second copy. If a patch is just replacing/overwriting a bunch of bytes in the middle of a file - e.g. AABBCCDD to AAbbccD then there will be no copy, just open-seek-write-close operation for the changed data. This is why some patches are über quick, near instant

Okay, I've not heard of any of these on PS4. Generally people experience, or at least talk about, the long ones, the 40-60 minute updates of small patches. If these were just delta patches, PS4 ought not be much slower than other platforms. The general Internet Truth is PS4 copies a full game folder though I can't source a Sony spokesperson on that, although it's obviously happening with those games that require a large amount of free space for a small patch. Furthermore, even on new downloads you need enough free space to apply any updates. As such, the 60 GB COD:MW required 120 GB of free space to install - 60 GB for the game and the 60 GB for the patched copy.

If some games don't do this, that's news to me.

The initial copy process is decompressing the patch files

That doesn't make sense. A moderate patch won't take 40 minutes to decompress but those are the times we experience during the 'copy' phase. Additionally if this were the issue, patching times would be the same on PS4 and XBO. AFAIK no-one complains about 40 minute patches on XBO.

 

[swaaye]

For fun I ran some tests on my PS4 Pro's 5400RPM 1TB HGST HDD that I have in a USB enclosure. Formatted it first.

I guess the 40MB/s number above has to do with worst case inner platter fragmented reads.

 

[Arwin]

I’ve never had any kind of update like that on the PS4 as far as I can remember. Of course many updates happened nightly automatically but even when they didn’t happen it was never long.

 

[Deleted member 11852]

Okay, I've not heard of any of these on PS4. Generally people experience, or at least talk about, the long ones, the 40-60 minute updates of small patches. If some games don't do this, that's news to me.

My anecdotal evidence as it pointless valid as yours of course ;-) Speaking from experience, and like Arwin said below about patches running overnight or when the console is not being used, I cannot recall any patch on my PS4/Pro taking more than maybe twenty minutes and only for massive patches like the one for Days Gone that basically replaced all the textures in the game with better compressed versions.

Where you see tend to see patches in realtime is for newly released games that often get a few hot patches in the following week in which I'm playing them after launch. They won't auto-patch because the game is being played so you get to see through the patch process once you have saved in game, exited the game and allowed it to patch.

For fun I ran some tests on my PS4 Pro's 5400RPM 1TB HGST HDD that I have in a USB enclosure. Formatted it first.

The 50MB/s and 20MB/s numbers I took from Insomniac's Spider-Man GDC technical post-mortem, the part about the differences in performance of the stock HDD compared to some slow ones that people may chose to 'upgrade' it with starts around minute 17:

I personally was very choosey when picking replacement drives for my PS3 and PS4. Even the SSDs I have on my PS5 and Series X to hold previous generation games are Samsung Q870s.

I’ve never had any kind of update like that on the PS4 as far as I can remember. Of course many updates happened nightly automatically but even when they didn’t happen it was never long.

Same. Like an hour to patch a game? Holy lobster balls that just sounds insane. I would have been very frustrated to be forced to take long breaks when playing Ghost of Tsushima, GTA V, Red Dead Redemption 2, God of War, Spider-Man, The Last of Us Part II - all games I played on launch on a PS4 Pro and which were patched several times shortly after to fix various issues.

I only ever had HDDs in my PS4 Pro because a) 4Tb SSD were absurdly expensive and b) I recall Digital Foundry testing the performance differences between HDD and SSD on PS4 and found most games benefited very little.

 

[Silent_Buddha]

We know form the Spider-Man PS4 postmortem that PS4's stock drive as a 50MB/s READ speed but that some user replaced drives have closer to a 20MB/s READ speed. Write speeds are typically much slower so figure how long a copy - lots of reads and writes - would the patching always made a second copy. If a patch is just replacing/overwriting a bunch of bytes in the middle of a file - e.g. AABBCCDD to AAbbccD then there will be no copy, just open-seek-write-close operation for the changed data. This is why some patches are über quick, near instant.

That will be for a mix of random and sequential reads. In games, even if you have duplicate assets to increase the frequency of sequential reads, you'll still run into cases where random reads are unavoidable. That's because you don't have infinite storage space and not every situation requiring reading game assets can be laid out sequentially.

Copying a game folder should be mostly sequential reads so that'll get closer to a drives maximum read speed. Depending on whether that hits the inner or outer tracks that could be up to 90-100 MB/s (or higher). Of course, if there's a lot of small files that aren't packed into some type of PAK file that'll reduce read speed as well. Also, if the drive is heavily fragmented (Modern Windows does a pretty good job of avoiding this, but I have no idea about the PlayStation's OS) that'll slow down copying as well.

Since a game developer can't know if a game will be written to the inner or outer tracks of a drive that's likely where the 50 MB/s read speed is coming from. Worst case sequential reads if the game is installed on the very inner tracks of the drive. The 20 MB/s will be the worst case modified by the knowledge that some amount random reads during gameplay will be unavoidable and/or LOTS of small files are being read.

Regards,
SB

 

[Deleted member 11852]

TSince a game developer can't know if a game will be written to the inner or outer tracks of a drive that's likely where the 50 MB/s read speed is coming from.

I agree with your post but it should not not be an issue for the game developer to consider whether their game is installed on the inner or outer segments of a console's HDD, this is for the filesystem (any more than a PC with a slow HDD or fast SSD). It's not possible for every game to be installed on the outer rings of a HDD where performance is highest.

My impression is that this was never an issue on previously generation console (hence the Spider-Man PS4 technical port-mortem) but it definitely feels like a technical consideration this generation. This why why Xbox Series consoles only permit approved NVMe drives and why PS5 ask has minimum performance requirements and benchmark news drive when they are inserted into an PS5.

 

[Shifty Geezer]

How often did these games patch in standby mode behind the scenes?

As for me, it's not anecdotal if it's in the mainstream media.

https://www.eurogamer.net/articles/digitalfoundry-2022-grand-theft-auto-5-ps5-tech-review

I'm not just talking about loading, but also installation. After downloading GTA5 on PS4, I discovered that a further installation period was required, which took over an hour (!).

That's Alex at DF.

Then general anecdotes in threads on the subject:

Took me almost 45 minutes to copy the MW patch the other night. Downloading it took less time.

https://screenrant.com/ps4-copying-update-file-slow-why-download-sony/

Many PS4 owners, while sitting through potentially hour-long copying sessions, likely wonder why the PS4 needs to copy updates in the first place.

Unvalidated here, but it does seem based on a common public narrative. So I'm wondering why some people aren't experiencing these? Is it really per game with some using delta patching? Or just that they've had background updates and haven't been aware?

 

[Deleted member 11852]

How often did these games patch in standby mode behind the scenes?

I have no idea. A lot would depend on the games, the particular HDD, if you were playing/had suspended the game, the install size, the patch size, the amount free space and whether or not the PS4 in question decides it needs to create a contiguous version of a case after patching. This feels like a lot a variables contributing to why people may - or may not - be not experiencing hour-long patch times.

All I can repeat is that, like Arwin, my experience of gaming patching games on PS4 was nowhere this long. But if you played games with a lot of patches, that were particularity bad, I can understand why this could be a take-away perception.

I watched a lot of games patch on my PS4/Pro and none were close to taking an hour to patch. I did not have some über expensive super-fast HDD in my PS4 and that's all I can really say.

 

[Shifty Geezer]

Regardless of particular factors affecting time, the fundamental patch process has to be the same. For me, the question is whether the copying process is as I believe it t ba and as common general ignorance reinforces, or whether it's not, whether some patches don't copy at all or not.

My personal experience that encourages me to accept the general consensus were the many Apex updates that came out of nowhere and, turning on the PS4, I was locked out of the game for 50 minutes, 10 minutes to download and 40 to 'copy'. Then getting the SSD, 'copying' took more like 10 minutes. For other games, these always update in the background of whatever I'm playing so I never noticed. That said, I do vaguely recall some patches seemingly quicker, as if only part of the folder was being copied.

I watched a lot of games patch on my PS4/Pro and none were close to taking an hour to patch. I did not have some über expensive super-fast HDD in my PS4 and that's all I can really say.

But none were 2-3 minutes? Because that's all you get on delta patch platforms like XB and PC. If you're anything like 20 minutes, there has to be more happening than replacing a few files.

In short, we have a great baseline on XBO. Sadly I've no idea what patch times are like on that because no-one talks about it! Clearly short enough that it's a non issue.

 

[Deleted member 11852]

But none were 2-3 minutes? Because that's all you get on delta patch platforms like XB and PC. If you're anything like 20 minutes, there has to be more happening than replacing a few files.

I definitely experienced PS4 games that patched in seconds. Some game patches were incredibly quick. Which ones games and patches? I can't remember.. this is a console I first owned nine years ago and have not used since November 2020.

The fundamental patch processes is replace old data with new, without creating crazy defragmented versions of the new version of the game,. There are lot of variables that can make that a very different experience for two different people.

 

[swaaye]

I think I've just never played any of the games that have super lengthy patch behavior. Or I've just forgotten lol

If I hadn't loaned out my PS4 I would do some experimenting.