Carmack on Ray Tracing & Rasterization

[Shaderhacker]

GPUs wouldn't help us with raytracing. Arnold, the renderer we use, does not support it and we'd prefer it to remain that way.

You are in good hands!! Arnold rocks! And I hope we can get it in-house eventually.

BTW as far as I know only a few VFX studios use GPUs for significant rendering tasks, for example Weta has them calculate their spherical harmonics based lighting. The resulting intermediate datasets are then fed to PRMan running on regular CPUs.
The current trend is to get rid of such intermediate steps, pointcloud data and shadow maps and such, because it takes a lot of additional artist time to manage these. It's easier and cheaper to just raytrace and buy more CPUs.

Yep. Brute force is way better than cluttering up the pipeline with overhead. We are so sick of point clouds. I feel ya.

-M

 

[Shaderhacker]

I understand that, but from the context of Carmack's comments, it's about knowing where raytracing on highly parallel compute hardware like GPUs is. Even the best CPU still has a tenth of peak single GPU FLOP performance.

You are right. But the GPU is hampered by memory - which automatically disqualifies it.

 

[Laa-Yosh]

You are in good hands!! Arnold rocks!

I know, we've been on it since like early 2010, even contributed to some of the development effort. One of our dev guys actually left to Madrid to work on the renderer full time

 

[Laa-Yosh]

Try using GI with fur in a RT. Or try motion blur with glossy reflections.

Oh, we do all that... we're now doing long hair and beards too Not so many glossy surfaces though, that'll be the next project I guess.

 

[MfA]

Also, shadow buffers require you to manually adjust bias for the shadows, if you want to avoid shadows disappearing or bleeding through - raytracing doesn't require that either.

Not really inherent in forward projection, aka rasterization, hard shadows with an irregular z-buffer can be identical to hard shadows with raytracing.

 

[anexanhume]

You are right. But the GPU is hampered by memory - which automatically disqualifies it.

For now. Eventually available has to outpace target resolutions or main memory will become coherent to GPUs, no?

 

[Shifty Geezer]

GPUs wouldn't help us with raytracing. Arnold, the renderer we use, does not support it and we'd prefer it to remain that way.

Cell missed its calling.

 

[Shaderhacker]

For now. Eventually available has to outpace target resolutions or main memory will become coherent to GPUs, no?

I hope so. We need at least 24G boards. I'm not sure how they would cache textures as well as hold complex dynamic data structures that update at least 30fps.

It'll be a while.

We don't even have enough bandwidth for drawing hair primitives. It's still too expensive to render hair the proper way.

-M

 

[Laa-Yosh]

Not really inherent in forward projection, aka rasterization, hard shadows with an irregular z-buffer can be identical to hard shadows with raytracing.

I can only rely on my experiences with PRMan's shadow maps, those were a b****... min distance, max distance, mid distance, bias, resolution, blur... And never enough memory, so I ended up animating the shadow camera to follow the action, and so on. Man, I hated it

 

[Billy Idol]

Thanks Laa-Yosh for the info. How much is the maximum or average memory consumption while raytracing?

I am a bit surprised about the hardware you use. Two 6-core Xeon seems very very few resources? Does Arnold scale to much higher number of cores...say 100? Or what about using a Xeon Phi instead...maybe depends on memory usage?

 

[Laa-Yosh]

I have no idea about memory usage statistics, but the new 12-core nodes have 64GB in them.

As for using this number of CPUs and cores, again we have standard server racks and these are I think 2U machines. Not sure how many cores you can put in at that size.

There are also other issues about heat, and weight per square meter - again, the office building is a restored one and more than 150 years old, and the render nodes are on the first floor. We need to keep them distributed a bit and the aircon system has some funny issues during the summer when a certain tree releases these little flying thingies that tend to get sucked in and stop the airflow...

Then there's the network infrastructure and the storage and the various specifics and access patterns for the different type of machines and jobs... And I'm also completely at a loss about the cost and other efficiency issues but I'm sure our sysadmins have done their math and all. And we're still a relatively small shop not involved in VFX work, so our infrastructure can and has to be more simple and cheaper...

 

[Billy Idol]

Thanks for the infos!

 

[Laa-Yosh]

You're welcome!

Also, I've only included the hardware info to give some context to those rendering times. But also keep in mind the majority of our nodes are older and not as fast as these new 12-core systems, so the average frame times have to be judged in light of that.

 

[Andrew Lauritzen]

Also, shadow buffers require you to manually adjust bias for the shadows, if you want to avoid shadows disappearing or bleeding through - raytracing doesn't require that either.

That's not totally true on either count... filtered shadows can avoid the filter-size-based bias and thus only need a small precision delta to avoid numeric/self-shadow issues. And raytracing needs the same bias away from the polygon, so it's not really that different. Ray traced hard shadows can of course avoid some forms of aliasing that are difficult for shadow maps, but at a severe performance cost, similar to IZB. You pay for the hard cases either in performance or quality, there's no free lunch. Also I'll note that aliasing due to single-ray-per-pixel shadows is far worse in practice, and filtered shadow maps are a much more efficient way to solve that form.

I think for hard shadows shadow maps of sufficient resolution + cascades + some decent filtering will still win over ray tracing. For plausible soft shadows everything based on a single shadow map looks like crap, but conversely ray tracing requires way too many rays. Sigh... maybe stochastic rasterization, but it's just an expensive problem it seems.

Obviously any time you need truly incoherent rays you need ray tracing. The issue is if you have so little coherence you normally need a lot of rays to avoid noise. And filtering games can be played with either technique so if you're not looking for a physically correct result (which games are not), it's a bit of a wash there too.

The more interesting question to me is not whether ray tracing triangles is interesting, but rather whether ray tracing filtered/volumetric data structures is plausible. Adding just incoherent rays but requiring super-sampling is ok, but not super-compelling TBH, but adding constant-time volumetric lookups (a la. frustum or cone tracing) is definitely interesting...

I understand that, but from the context of Carmack's comments, it's about knowing where raytracing on highly parallel compute hardware like GPUs is. Even the best CPU still has a tenth of peak single GPU FLOP performance.

Sure, but IMHO "interesting" ray tracing (i.e. incoherent) is about handling divergence and reconvergence, not FLOPS (although you still need some). This is why IMGs chip is much more about scheduling than ALUs - for good reason.

 

[Laa-Yosh]

Please understand that I'm no graphics programmer - when I talk about issues with shadow maps, it's from the user side, handling lighting technical issues for a few shots on a project that happened about 6 years ago. Also, it's not a realtime application, so it's about far higher quality and somewhat different circumstances.

 

[Andrew Lauritzen]

Please understand that I'm no graphics programmer - when I talk about issues with shadow maps, it's from the user side, handling lighting technical issues for a few shots on a project that happened about 6 years ago. Also, it's not a realtime application, so it's about far higher quality and somewhat different circumstances.

Yeah no issues, and your statement is still generally correct that shadow maps require more cleverness and more tweaking (although they've gotten more automatic in the past few years, at least in the real-time space). Just wanted to clarify a few things since it's an area I'm very familiar with

 

[faizanali]

Wow, I can't imagine that day when we'll actually be playing games ray traced in 60FPS.

 

[proffesso]

Oookay, some info on render times as promised...

First of all everything is fully raytraced, but not too many reflective surfaces and almost no refractions are visible in the usual type of content.
We're using area lights and global illumination, not sure about how many diffuse bounces are included though.
We have very few things to adjust with regards to quality, which is actually a welcome thing as it makes optimizing renders an easier task. It's also safe to say that even our preview settings look significantly better than the average game; however both motion blur and GI and shadows are quite noisy, so we have to take a significant hit for acceptable image quality.

Now the problem is that each scene is separated into various layers for a bunch of reasons. Sometimes it's impossible to render everything together because it'd cause memory problems and so on. It's also faster to re-render just an individual element if something is wrong with the asset or the animation.
The other reason is that we optimize a lot - for example if we render a character with metal pieces on his clothing, those would have to reflect the entire environment. But for this layer we switch all the other objects to a lower detail version with simplified shaders if the reflection is a bit glossy and distorted, because noone would see the difference.

Averages for a randomly selected shot's elements are:
- main character 40 minutes per frame
- various hair eyebrow fur layers etc. 3-7 minutes
- various set elements/dressings 5-7 minutes
- volumetric smoke elements 4-20 minutes
Some elements haven't been completed and rendered at preview settings:
- secondary characters in background 5-8 minutes
- main set 20 minutes
Altogether 32 layers for this shot, all at 1280x544 resolution.

As suggested by the above, moving from preview to production settings increases render times about 500%. So we can say that the combined rendering time for all elements will add up to a few hours.

The 2D compositing job that combines these elements together and adds various post processing effects renders in about 3 minutes per frame. These include stuff like color correction, tone mapping, relighting on both individual elements and the final image, and even crazy little tweaks like adjusting color and transparency for hair and eyelashes in some cases.

We have a few dozen render machines purchased during the past years so they're not all the same; the fastest ones have two 6-core Xeon CPUs. Such a machine may render two frames at once AFAIK.

I guess this can help puts the requirements of raytracing into perspective.

the current trend with arnold is to try and render as much as possible in a single pass. splitting out passes into a lot of elements is old style that was done mostly for memory concerns. these days a objectid layer will take care of all the old style pass info (for cc).

Memory wise, raytracing isnt bad. its mostly geometry that chews memory. but even those are only called when in camera. render machines with 12gb ram are on the highside when it comes to farms. though I suspect that will change now ram is cheap. so any new boxes may have 24 - 32gb or so. some compositing boxes will have 32 - 64gb, but thats because those things keep multiple frames/layers in memory.

those dual cpu machines will generally just crunch 1 frame. rendering multiple frames at once on a single box means doubling the required ram, disk and network requirements. hence, rarely, if ever, done.

1280x544? im guessing you guys are doing cutscenes for a 360 title?

 

[Shaderhacker]

I can only rely on my experiences with PRMan's shadow maps, those were a b****... min distance, max distance, mid distance, bias, resolution, blur... And never enough memory, so I ended up animating the shadow camera to follow the action, and so on. Man, I hated it

Yeap. I feel your pain. We had the same problems. Very hard to light a whole scene with the scene is so large and your shadow map needed to be huge!

 

[Shaderhacker]

Also I'll note that aliasing due to single-ray-per-pixel shadows is far worse in practice, and filtered shadow maps are a much more efficient way to solve that form.

No one would ever only cast 1 shadow ray in ray-tracing. That's just asking for trouble.

I think for hard shadows shadow maps of sufficient resolution + cascades + some decent filtering will still win over ray tracing.

The problem that Laa-Loosh and I run into is tryign to create a shadow map that will encompass the whole scene. You will *not* be able to compute a high enough shadow resolution in a reasonable amount of time or memory footprint. Also, shadow maps are pretty time-consuming to compute when hair or semi-transparent geometry is involed where you need to create deep shadow maps.

We had big problems with deep shadow maps when trying to render smoke in a Road Runner short where he created a long trail where the sun light cone just could not step through the volume with enough resolution to avoid the stair-stepping aliasing in the smoke (self-shadowing). We eventually went with Mantra and just ray-traced the shadows.


-M