*merge/rename* The Importance of an H.D.D. (e.g. caches, streaming etc)

[AlNom]

I think we all agree that HDD's are useful and a good thing, it's just a shame when it goes beyond to be (to some of us) an obtrusive problem.

Indeed! Taking for example, Halo:CE - I would have loved an option to cache all the levels given just how quickly they brought the player into the game after the files were copied (the PC version has one of the fastest loading times I've ever seen).

I didn't mind the 15-20 seconds it took for the levels to copy to the HDD, but for a game that I played almost all the time in co-op, it would have been nice to permanently cache all the levels to the HDD.

It's a shame more 360 games don't take advantage of this - they are certainly encouraged to use a HDD if present.

This certainly boggles my mind considering two of the best looking games, Gears of War and Mass Effect in particular, don't take advantage of the HDD when it is present. i.e. WHY NOT? Streaming is a much more difficult issue to tackle than caching to the hard drive - just take a look at Bungie's GDC presentation on loading. I would have thought that the constant spinning of the disc and the particular non-linear nature of Mass Effect would warrant the miniscule effort to make use of the HDD.

 

[babcat]

Can the CPU of a system such as the PS3 decompress textures and send them to the VRAM faster than the HDD?

 

[Arwin]

Can the CPU of a system such as the PS3 decompress textures and send them to the VRAM faster than the HDD?

The HDD can't do anything without any CPU. But apart from that, textures are typically loaded into the GPU already in compressed form. A Cell SPE could read a texture straight from the HDD, do some additional decompression and processing routines, and then set them in the XDR memory in a format that the GPU likes to deal with , and then the RSX can get the textures from there if necessary.

 

[babcat]

Arwin,

Would doing that mean being able to send more megabytes of textures to the VRAM per second?

 

[inefficient]

I look forward to learning more about this issue.
Could you name and link, some developers other than Capcom who have released games mandating an install to HDD?

A list of games installed to my personal PS3 currently:

Grand Turismo 5 Prologue: 5.5GB
Devil May Cry 4 - 4.8GB
Oblivion - 4.5GB
Hot Shots Golf 5 - 4GB
Heavenly Sword 2GB
Assasins Creed 1.3GB
Orange Box 450MB
Ratchet & Clank 419MB
Ninja Gaiden - 3.5GB (Optional)
Virtua Fighter 5 - 2.3GB - (Optional)


I was suprised even Resistance has a 411MB foot print now. Probably because of all those patches. But I don't have a single map pack for that game.

 

[slider]

God, that's an awfully big chunk of a 60gig HDD.

 

[inefficient]

Ratchet requires a fairly large install as well, I believe. Though a big difference with Capcom is that it doesn't take nearly as long.

Ratchet is only 420MB. See my post above.

Isn't that one of CAPCOM's real problems?: 5GB shouldn't take 20 minutes to copy, unless you're doing it very inefficiently.

5GB at 20min is about 33.3Mbit/s. Actually that is pretty damn slow now that I think about it...

It could be due to Sony's proprietary encrypted filesystem. I imagine they are using some sort of asymmetric encryption scheme. Maybe whatever method they are using is very slow for encryption but very fast at decryption. I also imagine that the 7th SPU is used for the encryption/decryption and that it is an OS function devs have access to via an API and not something they have direct control over.

 

[AntShaw]

It could be due to Sony's proprietary encrypted filesystem. I imagine they are using some sort of asymmetric encryption scheme. Maybe whatever method they are using is very slow for encryption but very fast at decryption. I also imagine that the 7th SPU is used for the encryption/decryption and that it is an OS function devs have access to via an API and not something they have direct control over.

How does that explain for the 3.7gb install for the Lost game?

 

[Arwin]

It could be due to Sony's proprietary encrypted filesystem. I imagine they are using some sort of asymmetric encryption scheme. Maybe whatever method they are using is very slow for encryption but very fast at decryption. I also imagine that the 7th SPU is used for the encryption/decryption and that it is an OS function devs have access to via an API and not something they have direct control over.

It's slow any way you look at it. In this very thread another install is mentioned of just under 4GB which takes 7 minutes. Normal transfer from 2x BluRay Should be about 76Mbit. That would be a one to one copy of one 5Gb block of data, no decompression involved, so in theory further optimisation should be possible, unless the bottleneck is actually the harddrive, which it shouldn't be (but you never know of course).

For now, looking at other games, Capcom seems currently very slow.

 

[AntShaw]

Is there clear labelling on the boxes?

For Lost: Via Domus it states:

'Minimum 3gb required for installation'

 

[joker454]

This certainly boggles my mind considering two of the best looking games, Gears of War and Mass Effect in particular, don't take advantage of the HDD when it is present. i.e. WHY NOT?

I think if you make hdd support optional then you effectively have to qa the game as two sku's, the hdd version and the non hdd version, if you are using the hdd in a significant way. So it makes the last few months of the dev cycle more complicated. Given that they have to get the game working with no hdd anyways (on 360), there is limited incentive to go that extra mile and support the hdd. Even more so when that thanksgiving launch date is looming near. It kinda sucks cuz Mass Effect could have soooo benefited from a hdd install option ;(

 

[Mmmkay]

There are a ton of disc seeks going on while DMC4 is installing.

 

[FutureCTO]

5GB at 20min is about 33.3Mbit/s. Actually that is pretty damn slow now that I think about it...

It could be due to Sony's proprietary encrypted filesystem. I imagine they are using some sort of asymmetric encryption scheme. Maybe whatever method they are using is very slow for encryption but very fast at decryption. I also imagine that the 7th SPU is used for the encryption/decryption and that it is an OS function devs have access to via an API and not something they have direct control over.

Sony's proprietary streaming encryption and file system are not a bottleneck to the HDD or Optical Drive.
I predicted the use of native encrypted processing running on Cell before April 2004.

People dismissed my analysis of
- Cell using on chip encryption (no one anywhere else ever predicted this)
- "Broadband Engine" being a single PE, not a 4 PE module
- PS3 using a single PE Cell
- the 64MB DRAM, not being on chip or eDRAM
- stating no way would it start in 65nm, 90nm more likely...
etc, etc, some things missed, some things hit, and some brain bowls landed on a different console.

What is a BroadBand Engine? Why call a processor that?

According to my interpretation all of the PE units are broadband engines.
This is the reason I haven't adapted to Paul's interpretation of the Rambus contract.

The buss, PU and APU of a PE are the ideal size for Public Key Encryption (CoDeC).
The PU will definitely be a fully featured 64bit Floating point processing unit.

IMO: Cell's goal is to create a native online environment, safe for commerce.
The Way a PE is designed eliminates a lot of overhead native to a secure network.
1024bit encryption needs to be accelerated for streaming applications.
Anyways you can learn a lot of this at http://www.MIPS.com

But to the point, there is in Fact a 1024bit data type.
And that it is very relevant to Cell's origin by design.

I confidently suggested encryption being on Cell and continued to follow any news that might relate.
Ultimately they chose a 512bit bus with 512KB L2, instead of the 1024bit bus & 1024KB L2 they were discussing in 2004.

I point this out to hoping lend creditable value to my opinion. Since security is not an issue for public discussion or information. The details of console security is not something to post about, but I can confidently say that the on the fly enCoDeCyption has very little effect on performance. It has no effect on optical drive bandwidth and a negligible effect on HDD bandwidth. So again, the biggest bottleneck to game data is the optical read speed. Everything else is faster. Even reading from GDDR3 directly to Cell!

 

[betan]

This certainly boggles my mind considering two of the best looking games, Gears of War and Mass Effect in particular, don't take advantage of the HDD when it is present. i.e. WHY NOT? Streaming is a much more difficult issue to tackle than caching to the hard drive - just take a look at Bungie's GDC presentation on loading. I would have thought that the constant spinning of the disc and the particular non-linear nature of Mass Effect would warrant the miniscule effort to make use of the HDD.

Mass Effect sure, but do you think Gears SP would benefit much from HD cache with its linear nature?
edit: for people how doesn't replay often.

There are a ton of disc seeks going on while DMC4 is installing.

How is this possible?
If that's the case, sure looks like lazy dev work
After all, how hard can it be to make install data sequential.
Sony certification deserves some blame as well.

BTW frequent seeks are the most reasonable explanation for so slow install of DMC4.

 

[Blade47167]

There are a ton of disc seeks going on while DMC4 is installing.

Maybe the assets don't copy over as one then decompress. Instead all files are separate and it has like a list of locations on the disk that they reside and it pulls from that. I remember that oblivion has duplicated data to make retrieval faster perhaps everything its copying over is also pulled from the disk at times depending where you are in the game. Seems stupid tho considering Blu-rays space if thats the case. Why not just have it all in one pack sitting on the center most area of the disk and be done with it?

 

[sevanig]

On a 25 gig Bluray Duplicate data makes sense.
Capcom should of just had one set of game data that is to be copied to the h.d.d. (set out in the preferred structure)
And another data set used for in-game streaming...

Well thats what I would of done

 

[AlNom]

Mass Effect sure, but do you think Gears SP would benefit much from HD cache with its linear nature?
edit: for people how doesn't replay often.

For Gears, I was thinking the advantage would be more on caching all the MP maps and related assets to the HDD. The campaign was fine for most of the time unless you died, in which case, an HDD cache might have been a lot more useful on Insane/Hardcore.

Is it a game breaker? Not really.... But when you die continuously on Insane and have to wait 15+ seconds just to return to a checkpoint..... Compare that to the Halo games' nearly instant "back-in-the-game" reloading. Best example is Halo 2's Sniper Alley. Sure the game had texture popping (which is probably more a function of spare CPU/GPU cycles for decompression), but at least you could reload the checkpoint nearly instantly when everything was cached. (Like dialing a number today versus back in the old days... 4.dededede 3. dedede 8 dededededededede 9 dededededededede... gotta spin more for the phone to figure out what it's dialing/loading!)

 

[inpHilltr8r]

There are a ton of disc seeks going on while DMC4 is installing.

I know it's installing a ton of files. I rather suspect all those seeks are back to the TOC.

 

[Bellic]

What size is the cache allowed for 360 games?

 

[Arwin]

I think it's 2GB, of which there are 3 slots reserved on a 20GB machine. Not sure though.