How long has ID's new engine/game been in development?
For the game, less than UT 2007 and longer than Crysis. For the engine, less than UE 3 and
less than CryEngine 2. Yes, they've been working on the game longer than the engine. id is trying a novel (for them - Valve partially did this with Half-Life 2) way to develop this next game. They concentrated on nailing the gameplay, game flow dynamics, interactions, story, whatever and now they're "throwing artists at it" to paraphrase Carmack.
Which is smart considering Graphics tech evolves at a much faster rate so it pays to only work on the bling-bling later in the development, unlike with D3 where most of the graphics features in the retail game were demonstrated 3 years before at MacWorld 2001. (And when the game shipped, every graphics programmer and their mother had already implemented normal maps + stencil shadows that D3 no longer looked "new").
Is this a property of megatexture and physically simulated, or one of tech in id Tech 5 that uses HDD?
It's one of the benefits of using unique texturing (which id calls MegaTexture). If you're using X
mb of textures for all surfaces you see at any given point, instead of X
number of textures like current engines do, then it doesn't matter if those pixels are white, red or black because they will need the same memory and polygons. Currently, if you want to add unique pixels you have to add another texture and more polygons to the scene. So performance-wise it doesn't matter if you have a mile-long brick wall with bricks that look exactly the same or instead paint that mile-long wall so that each brick is unique because the mb of texture memory used is the same with MT.
If this is the case I find it hard to see what (if any performance gains can be made using this approach?)..
A single texture means less polygons and less batches are required to process the same scene.
- And lastly any improvements in productivity any development team may gain from using this technology are fundamentally constrained by a number of factors:-
a) available tools and DCC apps to provide the artists/designers with enough flexibility to efficiently leverage the technology well..
id has a new set of editing tools which they call "id Studio" which deal with MT editing natively. ET: Quake Wars which uses 1st generation of MegaTexture also features improved editors with MT support (for terrains only on that game).
b) available knowledge/experience of staff of these tools, apps and technology in order to fully benefit from the productivity gains (over the team deciding to use alternative technology they may already be fully equipped and have the necessary experience to leverage really well)
This is a problem common to all workflow changes and is not exclusive of MT. If someone makes a engine using solely procedural textures you can be sure the artists will need training before hand.
Maybe I'm missing something but it doesn't seem to have that same "wow look at that! We need to put that good sh** into our next engine!" kind of appeal that Doom3 brought with it when it was first unveiled..
Changing your shadow implementation is not as disruptive as changing the way you manage your assets. Like you yourself said, MT needs dedicated tool support. It's not really something a programmer can whip out in an afternoon because he's going to be a lot more dependent on content for testing than say toggling between shadow volumes and shadow maps. Even the normalmapping implementation was mild compared to this: you only needed a shader and one normal map that any programmer can create. To implement MT you'll need:
An editor that allows painting huge textures in real time.
A dedicated shader that handles all the filtering and mip-map chain complexities.
A proprietary file format to store all this information (MT is more than graphics, it also handles physics interactions).
An efficient algo to stream pages into memory, fill dirty pages, flush, etc.
An efficient algo to decompress the huge texture (this is optional but even if you get all the above working correctly, unless you can somehow fit all the huge textures your game will use into a DVD (or two) it's not going to be practical).
Finally, to see and measure the benefits of reduced polygon usage you'll need new meshes.