Prey demo on Xbox Live Marketplace
- Thread starter Cheezdoodles
- Start date
Jawed
Legend
If console games are already using multiple PAKs for performance (where one consolidated PAK is OK for PC), that implies that PAK itself is performing poorly off DVD. This seems to be primarily due to the quantity of seeks required when the game data is held in a consolidated PAK.Mordenkainen said:
What I'm interested in is, does the "tiled/streaming" nature of MT allow the DVD to support multiple-seeks (i.e. a consolidated texture file, instead of individual texture files) and streaming (mid-level loading) - stuff that's OK on PC with a hard disk. In effect, does the "virtual file system" of MT (versus that of PAK) reduce the impact of DVD seek time?
Can a game rely upon "just-in-time" textures off DVD, using MT
i.e. load enough textures to get the player into the level quickly, and load the rest as the player moves?With UE3 Engine's "streaming world", aren't we getting something similar? Isn't that answering the same problem? Does that support a streaming world off DVD?
Jawed
But that's nothing new. Every (disc-based) console developer does that sort of thing since the PS1; most SDKs even offer tools to optimize disc layout in such a fashion.Jawed said:
Seeks kill you, so you do everything to avoid them, even data duplication. All of that only becomes interesting difficult, when you run against your storage space limitations (i.e. think of something else, compress more, additional disc (which will also need to replicate a lot of the stuff from the 1st one)).
Jawed
Legend
Will MT make a difference when games are authored for loading off DVD?
For example, I'm guessing that the PAK format has a large seek overhead when loading textures off disk, which contributes to the need for textures to be duplicated across multiple PAKs, instead of a consolidated PAK. But I don't know if PAK has a significant seek overhead.
I'm merely guessing that the MT data structures shift the "seek overhead" to the graphics engine working with tiles in memory, with the CPU fetching tiles dumbly. I'm guessing that the way the scene is encoded into MT means that the CPU no longer fetches "individual textures", it fetches tiles. And I'm guessing that tiles, being more granular, require less seeks in total, compared to the CPU fetching textures off disk, one by one.
At the same time, the streamed nature of the tiling could make it easier for the CPU to allocate memory to textures. It's simpler to evict tiles (again, because they're granular) than for the CPU to decide which textures need to be in memory at a given time.
But hey, I'm guessing.
Jawed
For example, I'm guessing that the PAK format has a large seek overhead when loading textures off disk, which contributes to the need for textures to be duplicated across multiple PAKs, instead of a consolidated PAK. But I don't know if PAK has a significant seek overhead.
I'm merely guessing that the MT data structures shift the "seek overhead" to the graphics engine working with tiles in memory, with the CPU fetching tiles dumbly. I'm guessing that the way the scene is encoded into MT means that the CPU no longer fetches "individual textures", it fetches tiles. And I'm guessing that tiles, being more granular, require less seeks in total, compared to the CPU fetching textures off disk, one by one.
At the same time, the streamed nature of the tiling could make it easier for the CPU to allocate memory to textures. It's simpler to evict tiles (again, because they're granular) than for the CPU to decide which textures need to be in memory at a given time.
But hey, I'm guessing.
Jawed
Like Mordenkainen said, modren PAK files are just renamed ZIP files. Having to scower a big zip file for the data you want lakes longer than having the needed data for indvidual sections grouped down into smaller ZIP files, and having to work from a slower DVD drive rather than a faster HDD makes the the grouping more benifitial. Megatexture isn't doing anything to change that.
What kyleb said. Also, don't think of a megatexture as a collection of ordered tiles. You can't store locality in them because the player can be anywhere on it and request tiles in a very large range of orders. Exactly like a single PAK which doesn't store locality whereas how 3Drealms is doing Prey x360 they have many PAK files (with redundant data) so that they can store locality and improve seek time.
All the greater granularity you correctly mention in MT helps is with texture memory management. In fact, because of the fact texture memory consumption is always locked at 8mb video memory you may find yourself doing more DVD/HD seeks than if you had used regular textures and loaded a full 128mb batch of 'em during map loading time.
All the greater granularity you correctly mention in MT helps is with texture memory management. In fact, because of the fact texture memory consumption is always locked at 8mb video memory you may find yourself doing more DVD/HD seeks than if you had used regular textures and loaded a full 128mb batch of 'em during map loading time.
As a bit of an interesting aside, the access of two-dimensional data (e.g. MegaTexture tiles) stored on a 1D medium (e.g. disc) is a very common problem: It occurs in databases for example (albeit with higher dimensionality) and can be useful in image compression (nudge, wink).
Essentially you're looking for a mapping from N-dimensional space (e.g. 2D) to M-dimensional space (M < N, e.g. 1D) that tries to preserve some distance measure; meaning that if two tiles in the 2D version are close together, they also should be close together in the 1D version.
One can use space-filling curves for that, among them the Z-Curve (aka Morton order), the Hilbert Curve, or the Peano Curve for example.
Anyway, it's an interesting subject, especially if it reduces seeks...
Also, ZIP / PAK files can be easily read sequentially as they contain a "directory" which for each file in the archive contains the start of the compressed data from the beginning of the archive. Also, each file is compressed individually (i.e. no back- / forward-references across file-boundaries). This means that you can read files sequentially from an archive with no seeks (or other additional work ignoring the decompression of each file) if you keep its directory table in memory. I actually hacked a version of the unzip-library our game used to do this...
Essentially you're looking for a mapping from N-dimensional space (e.g. 2D) to M-dimensional space (M < N, e.g. 1D) that tries to preserve some distance measure; meaning that if two tiles in the 2D version are close together, they also should be close together in the 1D version.
One can use space-filling curves for that, among them the Z-Curve (aka Morton order), the Hilbert Curve, or the Peano Curve for example.
Anyway, it's an interesting subject, especially if it reduces seeks...
Also, ZIP / PAK files can be easily read sequentially as they contain a "directory" which for each file in the archive contains the start of the compressed data from the beginning of the archive. Also, each file is compressed individually (i.e. no back- / forward-references across file-boundaries). This means that you can read files sequentially from an archive with no seeks (or other additional work ignoring the decompression of each file) if you keep its directory table in memory. I actually hacked a version of the unzip-library our game used to do this...
Graphics cards also have the same problem. If you need to read 4 Texels from the texture to do bilinear filtering. You want all 4 samples next to each other in memory, which is impossible 100% of the time. At best all they can do is make it as optimal as possible.
Similar threads
