Carmack demos new iD engine at WWDC keynote

[MrWibble]

I'm not usually part of the Carmack fan-club, but I think the mega-texture stuff is a really cool idea that could be extended to a lot of different applications.

It's not just a matter of a unique diffuse texture across all geometry, but also potentially a unique normal map and any other texel-based lookup you want to do.

While you certainly can use traditional textures and so forth, I think that authoring those becomes a lot more difficult and it complicates the rendering loop. Having one texture (or set of textures) across everything is a very clean idea, and that appeals to me.

One thing I don't get though, is if you're using this technique on more than just a heightfield (which seems to be evident from the demo shown and from what Carmack says about it) then how is the texture parameterised...?

AFAICS it still needs to be localised such that all the texture for your immediate surroundings has to be in a localised part of the mega-texture.

Perhaps it's less of a mega-texture like the one used for terrain right now, and instead it's some kind of mega-texture-atlas, which is automatically packed. That might make editing stuff a bit of a bitch though.

 

[Gabrobot]

One thing I don't get though, is if you're using this technique on more than just a heightfield (which seems to be evident from the demo shown and from what Carmack says about it) then how is the texture parameterised...?

AFAICS it still needs to be localised such that all the texture for your immediate surroundings has to be in a localised part of the mega-texture.

Perhaps it's less of a mega-texture like the one used for terrain right now, and instead it's some kind of mega-texture-atlas, which is automatically packed. That might make editing stuff a bit of a bitch though.

Well even Quake Wars isn't on a simple height field, there are places where the terrain is vertical (the terrain is just a model made in a modeling program and so isn't restricted to vertical displacement). Carmack talked about that sometimes causing problems in Quake Wars because he hadn't quite got it all worked out yet. Apparently messing around with the UV mapping typically solved those issues. I'm not sure specifically what the issues were though.

id Tech 5 obviously has it all worked out...and it's worth noting that it's used on literally everything including characters.

Edit: I found the interview where he talks about it (definitely worth a read, it's the most comprehensive look into Megatexture tech currently):

http://www.gamerwithin.com/?view=article&article=1319&cat=2

Q5: Does MegaTexturing technology bring any specific limitations with it?

Answer: No. There’s no limit to dynamically changing it. That’s one of the neat things about it – to the graphics engine, it looks like you’re just texturing on top of arbitrary geometry. You can move it around and all of that. With the technology in Enemy Territory: QUAKE Wars, there are some issues with deforming the texture coordinates too much. You’ll get areas that are blurred more than you would expect with a conventional texturing, and that’s something that’s fixed in my newer rev of technology.

There are some minor things you have to worry a little bit about. If you stretched up too steep a cliff slide, there would be some blurring involved there, even if you adjusted the texture coordinate somewhat. And you can crutch around that a little bit. That’s also a problem that’s been fixed by a newer rev of technology that we’ve got right now.

 

[nAo]

I still don't get what megatexture is, can someone explain it to me?
Carmack used to be quite open about his stuff..

 

[Gabrobot]

I still don't get what megatexture is, can someone explain it to me?
Carmack used to be quite open about his stuff..

Ok, so right now the maximum size texture you could possible use is 4096x4096. What Megatexture does is it allows for much higher res textures (Quake Wars uses a 32,000 x 32,000 texture on its terrain). It does this by streaming the data based off of what is actually needed (since you only need as much data as will show up in the screen pixels...so something really far away really doesn't need to be very high-res). As a result it actually ends up being more efficient then if you used traditional texturing.

So basically this can allow for 100% unique and super high-res textures on everything with no performance hit.

Here's Carmack's lengthy explanation:

Q1: What is MegaTexturing technology?

Answer: MegaTexture technology is something that addresses resource limitations in one particular aspect of graphics. The core idea of it is that when you start looking at outdoor rendering and how you want to do terrain and things in general, people almost always wind up with some kind of cross-fade blended approach where you tile your textures over and blend between them and add little bits of detail here and there. A really important thing to realize about just generally tiling textures, that we’re so used to accepting it in games, is that when you have one repeated pattern over a bunch of geometry, the texture tiling and repeating is really just a very, very specialized form of data compression where it’s allowing you to take a smaller amount of data and have it replicated over multiple surfaces, or multiple parts of the same surface in a game since you generally don’t have enough memory to be able to have the exact texture that you’d like everywhere.

The key point of that is what you really want to do is to be able to have as much texture as you want to use where you have something unique everywhere. Now normally, you just can’t get away with doing that, because if you allocate a 32,000 by 32,000 texture, the graphics curve can’t render directly from that. There’s not enough memory in the system to do that, and even when you have normal sized textures, games are always up against the limits of the graphics card memory, and system memory, and eventually you’ve got hard drive or DVD memory on there, but you wind up with a lot of different swapping schemes, where you’ll have a little low-res version of a texture, and then high res versions that you bring in at different times, and a lot of effort goes into trying to manage this one way or the other.

So when Splash Damage was starting on, really early with Enemy Territory: QUAKE Wars, they were looking at some of these different ways to render the outdoor scenes with different blends and things like that. And one of my early suggestions to them was that they consider looking at an approach where you just use one monumentally large texture, and that turned out to be 32,000 by 32,000. And I – rather then doing it by the conventional way that you would approach something like this (i.e. – chopping up the geometry into different pieces and mapping different textures on to there and incrementally swapping them for low res versus high res versions), just let them treat one uniform geometry mesh and have this effectively unbounded texture side on there, and use a more complicated fragment program to go ahead and pick out exactly what should be on there, just as if the graphics hardware and the system really did support such a huge texture.

In the end what this winds up getting us is the ability to create a great outdoor terrain texture that has far more complex interactions than anything that you would get with any kind of conventional rendering, where you’ve built it up out of pieces of lots of smaller textures on there, where they do some sophisticated things with growing grass up between bump maps. And then you can go back and do hand touch ups in a lot of different places to accent around features that are coming out of the surface. And this type of thing is, I’m very sure, going to become critically importance as we go forward into kind of next generation technologies on there. We’ve seen this over and over as we’ve gone through graphical technology improvements over the years, where there will be certain key elements that you start looking at in games that look really dated because they don’t have the capabilities that people are seeing in sort of the cutting edge things there. And this type of unique texturing over the coming generation of games, I think, is going to be one of those, where when people start looking back at a game that’s predominantly piled and doesn’t have that unique artist touched sense about all of the scenes, it’s going to look very previous generation.

 

[Richard]

Ok, so right now the maximum size texture you could possible use is 4096x4096.

The limit is now (at least) 8k x 8k, as mandated by the DX10 specs.

So basically this can allow for 100% unique and super high-res textures on everything with no performance hit.

You know, one thing I'm not completely clear on MT is whether we can have different texture resolutions on the same MT - or perhaps I should say, variable texel density. It makes sense to raise the texture resolution for objects that will be at the player's eye-level, the player can get real close to but not for those objects the player can never get to (those distant snow-capped mountains in the demo).

I still don't get what megatexture is, can someone explain it to me?
Carmack used to be quite open about his stuff..

I think you'll be wanting more gritty details about it. I think it's helpful to go back and read Carmack's first public thoughts on this:

http://www.bluesnews.com/cgi-bin/finger.pl?id=1&time=20000308010919

D3 also has a partial implementation of MegaTexture which you may want to mess with.

 

[Gabrobot]

You know, one thing I'm not completely clear on MT is whether we can have different texture resolutions on the same MT - or perhaps I should say, variable texel density. It makes sense to raise the texture resolution for objects that will be at the player's eye-level, the player can get real close to but not for those objects the player can never get to (those distant snow-capped mountains in the demo).

Well I imagine in the case of terrain where you really don't want to use a separate texture for the far away mountains you could just monkey with the UV map so that it's stretched out more on the far away hills. But I think generally you'd just use a different texture since there's no real reason for it to be seamless.

 

[ShootMyMonkey]

You know, one thing I'm not completely clear on MT is whether we can have different texture resolutions on the same MT - or perhaps I should say, variable texel density. It makes sense to raise the texture resolution for objects that will be at the player's eye-level, the player can get real close to but not for those objects the player can never get to (those distant snow-capped mountains in the demo).

For the most part, I get the impression that the answer to that question would be 'no.' At least not in the workflow of texture creation, anyway. It seems like the whole idea is just to completely forget about all of that and virtualize multiple texture segments into one gigantic working texture of a single pixel density. Now it could be that you can alter texture LOD bias for a segment by tagging geometry sections in such a way as to tell it "we'll never get the camera near this" so that the geometry and the corresponding MT stream block will build into their raw binary with lower-res textures.

That's sort of only loosely related to MT in that you'd have some tools in the level editing workflow that enables you to define properties of the stream block segments, and that segmentation of the virtual gigantic texture influences that segmentation. I'd imagine the texture artist really wouldn't have any knowledge of that when creating the texture, though (which is kind of the point, I guess).

 

[MrWibble]

Well even Quake Wars isn't on a simple height field, there are places where the terrain is vertical (the terrain is just a model made in a modeling program and so isn't restricted to vertical displacement). Carmack talked about that sometimes causing problems in Quake Wars because he hadn't quite got it all worked out yet. Apparently messing around with the UV mapping typically solved those issues. I'm not sure specifically what the issues were though.

id Tech 5 obviously has it all worked out...and it's worth noting that it's used on literally everything including characters.

Edit: I found the interview where he talks about it (definitely worth a read, it's the most comprehensive look into Megatexture tech currently):

http://www.gamerwithin.com/?view=article&article=1319&cat=2

Thanks for the link - it's interesting reading for me as I'm looking into this kind of problem now.

Certainly it seems that the existing implementation is essentially a plane - deformed more than a simple heightfield, but topologically the same. Mapping that is not a big problem. Whether they can use a modelling tool or not, they're almost certainly restricted to keeping the same topology - punching a hole through from one cliff to another would not be possible.

However the claim that he's solved the mapping issues for other types of geometry (he doesn't say it's entirely arbitrary, but that seems to be the implication) would be pretty impressive. I'm very interested to know what kind of mapping he's applying to things to make it all work for non-simple surfaces.

Does it really provide a good solution for a surface of non-zero genus? Does it do it unaided by an artist? How does it ensure locality of texture samples for use in a streaming technique like MT?

I'm also intriguided to know why you'd want MT on a character, where generally you're not going to fly slowly around it paging in high-resolution bits of texture. Unless he's just talking about the mapping itself, which would certainly be useful.

Typically Carmack seems to employ existing techniques (albeit he's sometimes the first to do so in a game engine rather than some offline process) rather than come up with brand new stuff himself. So is he using some existing paramaterisation? I've tried a few solutions recently and not really found one that would be good enough for the purpose.

 

[nelg]

Those look like utter crap. I'm sorry, but they simply scream "Carmack" to me. What does that mean? They look like plastic, have a huge lack of details and I hate it. Even for "early" tech shots they're horrible.

The first thing that came to my mind was blow torch and wax museum.

 

[MrWibble]

Actually after an afternoon of thinking about this and going through a pile of papers I think I might be barking up the wrong tree with the parameterization stuff... I can see an alternative approach to the texturing that might just work.

Right, I'm off to write a big pile of code

 

[MfA]

I'm also intriguided to know why you'd want MT on a character, where generally you're not going to fly slowly around it paging in high-resolution bits of texture.

Would it be useful to be able to only store the highest resolution mipmaps for certain areas of the texture when mapping causes those to get stretched?

 

[nAo]

Would it not be useful to be able to only store the highest resolution mipmaps for certain areas of the texture when mapping causes those to get stretched?

But you can't never guarantee you're not going to minify those textures and savings wouldn't probably that big anyway, imho it's not worth the hassle.

 

[MfA]

But you can't never

Hmm?

minify

Magnification you mean? Could you simply clamp the miplevel based on location inside the texture? (I'm assuming he simply uses texld with coordinates he calculated using some block based index texture, it's the only way I see it working when only part of the original texture is ever loaded.) I guess if using anisotropic filtering you'd have to find a way to switch to bilinear filtering when it happens, since the hardware won't be switching to the magnification filter automatically anymore.

This would be handy regardless for MT, since you can show the low resolution mipmaps while streaming in the high resolution ones.

 

[nAo]

Hmm?
Magnification you mean?

No, mignification I meant
No matter how stretched your texture is, you can always run the risk of mignifying it if you don't store the lowest mip map levels, I'm concerned about aliasing/performance.

Could you simply clamp the miplevel based on location inside the texture?

You can do that easily if you wanna compute texturing LOD within your shader but sampling a texture that way is not exactly fast with current hw (dunno about R600..)

I guess if using anisotropic filtering you'd have to find a way to switch to bilinear filtering when it happens, since the hardware won't be switching to the magnification filter automatically anymore.

Yep, how you can solve that I don't know since we don't have that level of control at the moment.

 

[MfA]

You can do that easily if you wanna compute texturing LOD within your shader but sampling a texture that way is not exactly fast with current hw (dunno about R600..)

Even if you just bias the LOD? (With texldb.)

Yep, how you can solve that I don't know since we don't have that level of control at the moment.

If all else fails just have two samplers and flow control.

 

[nAo]

Even if you just bias the LOD? (With texldb.)

I tested it and it takes 4 cycles on NV4x/G7x hw and AFAIK it's slower than a normal fetch on G80 as well

 

[MfA]

How fast is texld with a variable as texture coordinate?

 

[nAo]

How fast is texld with a variable as texture coordinate?

4 cycles

 

[MfA]

AFAICS with MT you have to compute texture coordinates in the shader anyway since it's only ever partly loaded, only loading blocks currently visible into free parts of the actual texture.

So how about this, you simply always store the highest resolution mip for a block at level 0 ... but you apply a bias based on the block you are working with. In some cases you will need to add borders to the blocks for filtering, but apart from that it should work ... and because it's storing the highest resolution at level 0 magnification will still kick in without any more trouble.

 

[Gabrobot]

Actually after an afternoon of thinking about this and going through a pile of papers I think I might be barking up the wrong tree with the parameterization stuff... I can see an alternative approach to the texturing that might just work.

Right, I'm off to write a big pile of code

If you haven't already, you might want to check out the MegaTexture tech in Doom 3...I believe there are some console commands that may shed some light on it, as well as just the creation command itself. Or even just walking around and looking at the texture may reveal something about how it works. To get it working you just need this:

http://doom3.filefront.com/file/Megatexture_Technology_Mod;72779