HDR overused?
- Thread starter speedsix
- Start date
Shifty Geezer said:
All of which sounds like it'd be more work to implement and more prone to issues than current methods of analysing the HDR values to work out their rough distribution and find a mapping from that.
Also although people state "photo realism" as an aim, really I think they're after "realism" - which is less a case of simulating how a camera works and more a case (IMO) of simulating how the eye would percieve an image. And if you don't agree, we're going to have to start putting lens flares back in
Anyway, it's entirely possible that some games *will* try techniques like the one you suggest - taking a more scripted or focal-point based approach to the mapping. It may be that it creates a noticeably different style to graphics that differentiates one title against another. As I've said, we're still going through a process of seeing what works and what doesn't. One thing I'm sure of is that without HDR and this kind of experimentation, we'd be stuck with everything looking much the same as it does now.
Lens flare was funny. An artefact lens makers took great measures to try to reduce, CGs went throwing in with abundance. Personally I think photorealism is a better target than...occular realism. Occular realism doesn't have much by way of DOF for starters, whereas that's an important aspect to artistic imagery. Ultimately it doesn't matter which approach is taken, as long as what the games chuck out looks as good as the TV programmes and movies we watch on the same display hardware!MrWibble said:Also although people state "photo realism" as an aim, really I think they're after "realism" - which is less a case of simulating how a camera works and more a case (IMO) of simulating how the eye would percieve an image. And if you don't agree, we're going to have to start putting lens flares back in
Shifty Geezer said:
Your eye doesn't have DOF? Mine seems to do ok.
One of my personal bug-bears is overdone DOF in graphics. It's fine for giving hints as to exactly what the player should be looking at, especially in a cut-scene - but in a scene where I might want to choose to look at something in the background I find it annoying if the background is blurry but the foreground that I'm not so interested in is not.
I'm not sure what we can do about it, until we get an eye-toy like system that can work out what bit of the screen I'm actually trying to focus on...
DOF in eyes is a bit awkward because when not focussing on an object, that object is seen in double vision. DOF is there but it's not particularly noticeable, not to the degree that it's used in much fine-art photograhy, in part due to the relatively short relative focal length of the eye. If you're using a camera with a longer focal length, DOF is normally very pronounced, so a camera situated a long way off and zooming in on the action should have it. Normal FPSs or first/near-first person perspectives shouldn't. It's one of those things that a professional cameraman would be able to contribute a lot to I imagine, whereas your average game developer is more likely to go with 'how cool is this?!' DOF in spades! The DOF in Kingdom Hearts was pretty tastefully used IIRC, focussing on the point of interest. I expect the next Team ICO game to produce some excellently directed camerawork, but I hope we don't have to wait two to three years for the first truly film-like virtual photography. DOF used based on where the eye is looking is an interesting idea. Not sure how well that would work but it's probably necessary for FPS if they're to have DOF (unless it's a FPS for a myopic soldier ). For spectators I guess they'd end up looking where you were, following the focussed area, which would be pretty weird!
). For spectators I guess they'd end up looking where you were, following the focussed area, which would be pretty weird!Mintmaster
Veteran
Like I said earlier, you have the previous frame's average luminance for that. Neither a camera nor an eye adapts instantaneously.MrWibble said:
Let's do an example. Take Xenos' 0-32 range, and your goal is to map the average luminance to the scene to a value of 0.25. This way you can store values 128x brighter or 64x darker than the average.
When the scene starts, assume the average is 1, so your scale factor is 0.25. Now suppose you have raw values ranging from 1000 to 10M. You render the scene, and everything gets clamped to 32, so your average is 32. Next frame, you use this to knowledge and think the scale factor needs to be 128 times smaller to get the average down to 0.25. So you're scale factor is now 0.0002. This time, 1000 gets mapped to 2, but most values will still saturate at 32. So lets say your average turns out to be ~25. You need another factor of 100, so you update the scale factor to be 2*10^-6. Next frame, the brightest values of 10M are mapped to 20, and your dimmest are mapped to 1/1000 (slight compression down there, but as I said before, it doesn't matter). Your average should be close to your target now, after just three frames and that horrible initial guess.
As the scene changes, this scheme automatically adapts to the lighting. It adjusts at two orders of magnitude per frame, which is way faster than a camera and eye, so you'd probably want to slow it down anyway.
You really don't lose anything by doing it this way, as you still know what the raw values are. You don't need an uber-high range format. Dynamic range of a few thousand is enough. In a real camera, even if your scene has areas with luminance of 0.001 and 1M simultaneously, your film won't give you anything outside of plus/minus two orders magnitude away from the "midpoint". If most of the interesting stuff of that scene is near 1000, your film will capture "fully bright" for any light value from 100,000 to 1M, and "fully dark" for anything between 0.001 and 10.
(Apologies for not knowing the correct photographic term for all these concepts I'm putting in quotes.)
As for the human element of post-production and editing, I fail to see how a higher precision format solves that problem. You can use the above method for spot averages instead of full scene averages. Like I said, you do know the raw values provided instantaneous contrast isn't really high, and even then you have more than enough information to decide how to tone map it.
Last edited by a moderator:
Mintmaster said:
(I don't suggest a human element can be replaced with more precision - the two are seperate issues for me. However *without* enough precision in the image, I feel we'll lose quality in the image after mapping it, and we don't quite have enough information to map it correctly in the first place.)
Well your method here is just the first part of most tone mapping systems - working out what the "target" luminance is, possibly taking into account previous results. Generally speaking we average the logarithms of the luminance values and we average with a previous result based on a time-step, rather than simply throwing in a previous frame. This stuff is bread and butter to a modern GPU, so there's not a huge amount of worry about in that step.
I'd be pretty worried about doing any kind of statistical analysis on a range of values that have already been very harshly clamped. You seem to suggest that the range has both an upper and lower clamp - but if that's the case, you need to both offet *and* scale the range to get it to fit... how would you know which is necessary when you've thrown most of the distribution information away? Generally speaking you'd assume a lower bound of zero, and only worry about the scale, but if you're seriously suggesting just linearly scaling to fit that's going to look horrible. The optical system doesn't work that way, and neither does film, AFAIK.
Furthermore, you're going to have a hell of a problem if your brightest spot is indeed something like the sun - your method will see a very high average and even after adjusting, you're going to get a bright oversaturated area where the sun roughly is, and black on the rest of the image.
There are good reasons we fit luminance to a curve, and so I have no idea why you seem to think we shouldn't bother. My original point was that we're all still experimenting with ways to do that better - it's not so much a performance issue (though some of the non-realtime methods would be if we tried them) as a quality issue. I really don't see your method improving quality. Or maybe I'm just not following what it is you're actually trying to say.
If what you're essentially arguing about is that we don't need more accuracy in the framebuffer, then from my point of view at least, you've done nothing to convince me (you still haven't even touched on the issue I raised about rendering the image in the first place, which a photographer of course doesn't need to worry about). But I'm confused - you say a range of "a few thousand is enough" - but that's a lot more than we have in a traditional LDR buffer. Are you just saying we should use integer 16-bit instead of FP16? We certainly could do that, but again I don't see the advantage. You could probably build a slightly simpler bit of hardware, but I think only the ROP stage is actually going to be affected anyway, and we've already got most of the way to that bit being float already. And all you're really doing is shifting the precision around a bit, without really addressing whether or not that's necessary.
We could test that of course - take a bunch of real HDR imagery (with at least fp32 accuracy) and map it to both FP16 and Int16, then compare the results to see which is more accurate... My gut feeling is that int16 will come off worse, but I really have no basis for that other than instinct and experience.
If you're just suggesting an alternative method of tone-mapping then you're welcome to try it out and post some results - however I think this kind of simplistic tone-mapping has already been well explored and already discarded as producing unrealistic results which don't look good. Certainly my own tests show that you really do need to spend a bit of time getting the mapping right and it's pretty essential to be doing more than mapping the visible values linearly to the output range.
So I'm not sure what exactly you're trying to fix. You've acknowledged that we do need better than LDR precision, you've suggested a more basic solution to a couple of problems than we already have, which gets us nowhere in particular, and you've glossed over some of the thornier issues anyway.
Not a convincing argument to me, sorry - I'm really just not sure what it is you're trying to say...
overclocked
Veteran
Anyone knows the type of HDR PGR3 is using in "view our cars" ? It must be some mistake by bizarre there cause in certain angles and spaces the Lightning is way off.
mckmas8808 said:
Artistic; for example, the very reason people compare the Doom 3 engine to Source.
But for personal preference, I'd rather play a game with style and content, rather than something where its obvious dev's spent 90% of their time trying to get lighting right
Don't get them wrong though, all technicalities aside, HDR libraries are probably saving devs alot of development time, and that I can't argue with...no matter how bad it might look in the end.
A176 said:Artistic; for example, the very reason people compare the Doom 3 engine to Source.
But for personal preference, I'd rather play a game with style and content, rather than something where its obvious dev's spent 90% of their time trying to get lighting right
Don't get them wrong though, all technicalities aside, HDR libraries are probably saving devs alot of development time, and that I can't argue with...no matter how bad it might look in the end.
I wouldn't say it's saving us any time - if anything it takes longer...
Mintmaster
Veteran
MrWibble, a lot of your arguments don't make sense at all, and you're completely avoiding the factual evidence I have that my method works: A real life camera. You also need to understand that one still has to tonemap the output (i.e. develop the film).
I really don't know what you're going on about here. Give me an example of how this system fails. If your tone-mapping operator needs more than 4 orders of magnitude in its domain to go from white to black, it's going to completely wash out the constrast between a pixel twice the brightness of another. I've seen papers on local constrast operators, but that's going beyond a camera's capability, and I don't think range would be a problem anyway.
As an example of why we need this change of thought, I heard nAo talking about pushing the limits of FP16/NAO32, where he sees dithering/banding in the darkest scenes of Heavenly Sword. There's no reason to render the absolute color value in the framebuffer. The previous frame's exposure is a good estimate of the next frame's exposure, so use this value to assign more reasonable framebuffer values for the scene.
Another reason is that I don't think we need to move towards 64-bpp rendering. 32-bpp is enough if the format is right. FP10 is great, and shared exponent would be even better. FP16 is overkill for a framebuffer.
In laboratory testing, a range of 10,000:1 is a lot greater than the active domain of a film's/CCD's response curve. (8-bits is enough for the range of the curve, but we don't have a choice there anyway due to monitor technology.) For photorealism, it's enough. Period.MrWibble said:
You'll notice that the clamping I'm talking about is in terms of "order of magnitude", hence mathematically speaking it's a logarithmic clamp. A shift on a logarithmic scale is a scale in the linear world. The clamping itself happens automatically when you force a value into a finite precision format. I'm just talking about dynamic range, and it's no different whether you're in graphics, instrumentation, electronics, or whatever. Saturation value divided by noise floor.
Umm, and how do you think an optical system works? The iris scales the light your camera lets in. The shutter can be open for different amounts of time, again just a linear scale. The light hitting the film's surface is directly proportional to the light emitted by a point in the scene. Period. If it was non-linear, or even had an offset, you'd win the Nobel Prize.
I really don't know what you're going on about here. Give me an example of how this system fails. If your tone-mapping operator needs more than 4 orders of magnitude in its domain to go from white to black, it's going to completely wash out the constrast between a pixel twice the brightness of another. I've seen papers on local constrast operators, but that's going beyond a camera's capability, and I don't think range would be a problem anyway.
Nope, you got it backwards. The sun will be clamped to an effective value of 32/scale, so it's going to give you a lower average luminance. If 1% of the previous example had a luminance of 1B instead of 10M, it would get clamped to 32 in the framebuffer, translating into a value of 10M. The computed average doesn't change, and so dark details retain their contrast. What will happen is that if the sun is more than 100 times the average luminance, it won't affect the scale factor, so the rest of your scene retains the detail.
No, I was just talking about Xenos' FP10 format. I don't think 8-bit is enough, though zeckensack does. 10-bit integer (1000:1) may get you decent results, but it's borderline IMO. I was really disappointed when I found out ATI's X1K series didn't have FP10.
Okay, now I know for sure you don't know what I'm talking about. All this talk has nothing to do with tone mapping. For tone-mapping the only suggestion I made was to look at the film response curve, and that most certainly isn't linear. You still have to tonemap this framebuffer into the final results you see on screen.
As an example of why we need this change of thought, I heard nAo talking about pushing the limits of FP16/NAO32, where he sees dithering/banding in the darkest scenes of Heavenly Sword. There's no reason to render the absolute color value in the framebuffer. The previous frame's exposure is a good estimate of the next frame's exposure, so use this value to assign more reasonable framebuffer values for the scene.
Another reason is that I don't think we need to move towards 64-bpp rendering. 32-bpp is enough if the format is right. FP10 is great, and shared exponent would be even better. FP16 is overkill for a framebuffer.
Mintmaster said:
So you're just arguing for less precision being necessary... ok - well my point in that regard is that representing a single image does not require the same dynamic range as rendering it.
What would happen in your scheme if a scene contained a very bright light-source (such as the sun) which was obscured by a slightly translucent surface?
The "correct" result would correctly attenuate the lighting - the result will quite possible be relatively dark - perhaps the image contrast will be no more than 100:1, but the lightsource will be clearly visible through the surface. For the sun, if you try to view it normally (not that I'd recommend that) you don't see the sun very distinctly because anywhere near the sun you still get so much light bouncing into view that the whole sky is bright. However viewed through a dark surface, you'll see the sun clearly defined.
With your scheme, after the system settles down, the average luminance will be relatively dark and your system will end up setting the clamp such that only a fairly low-end part of the possible range is represented and anything bright is clamped. Fine if we just wanted to represent that final image - but rendering it still involves rendering that bright lightsource first, and that will get clamped, way lower than is going to be useful.
You'd still possibly see the shape of the lightsource, but any attenuation will be lost and you'd get a big area of flat colour rather than a nicely graduated result. In my example you might not see much at all - the whole sky is probably brighter than the attenuated result, let alone the bit the sun is in - clamping it even a fair amount brighter than the average will give you a flat colour across the whole sky, the sun will be lost entirely, and when attenuated it's probably not bright enough to really show through at all anymore.
Because the result is actually now *different* to the first attempt (not more or less precise - actually quite different in distribution) you'll get a new average - which will be even lower, because the clamp means the brightest values got thrown away, and everything will get gradually worse until you can't actually see the light through the surface at all...
The only solution I can see to this would be to have an additional buffer to store the maximum luminances into, and use that to calculate the necessary range... but unless we can change the hardware to do that for us, we've just allocated a 2nd render target with the same resolution as the first anway, so we might as well have just used more precision...
Mintmaster
Veteran
No need for a long explanation there, MrWibbles, as I understood after the first sentence. That's one of the extreme cases which does show clamping errors, and will give an incorrect output. (zeckensack: This is why more than 8-bit is needed. Even if you knew the perfect scale factor, you can't render a bright object through a tinted surface correctly)
I think there are a few things that could mitigate any errors:
1) 8,192:1 does still have some room to play with. Going back to my example, you can target a lower average value (say 0.1 instead of 0.25) to increase the room on the upper end. 320x the average is a lot, and your tone map shouldn't need that full range.
2) HL2, which can't do any HDR at all through a window, manages okay. This will be much less of a problem.
3) You can special case this with the alpha buffer. Even though you only have 2-bits in a 10-10-10-2 format, you can flag where super-bright objects are. The flag can stand for an additional factor of 100 or something for the sun. In practice, I suspect only a few super intense sources would need this.
4) I think it's a hard artifact to visually identify as incorrect. #1 and #3 should take care of most of it.
One more thing:
I think there are a few things that could mitigate any errors:
1) 8,192:1 does still have some room to play with. Going back to my example, you can target a lower average value (say 0.1 instead of 0.25) to increase the room on the upper end. 320x the average is a lot, and your tone map shouldn't need that full range.
2) HL2, which can't do any HDR at all through a window, manages okay. This will be much less of a problem.
3) You can special case this with the alpha buffer. Even though you only have 2-bits in a 10-10-10-2 format, you can flag where super-bright objects are. The flag can stand for an additional factor of 100 or something for the sun. In practice, I suspect only a few super intense sources would need this.
4) I think it's a hard artifact to visually identify as incorrect. #1 and #3 should take care of most of it.
One more thing:
Your line of reasoning here is similar to the previous post where you said everything would be dark if there was a bright light source. You should remember that the calculated average has an inverse feedback to the scale factor. If the average is lower than your target, then the scale becomes larger, thus brightening the image.
Mintmaster said:
Or we could just use pixels with more range and precision and not worry about special case hacks...
If we're stuck with some hardware that doesn't have FP16 or above, or (like pretty much anything we have right now) where FP16 has some other penalty we don't want to pay, then we'll need to work around with a lower precision (I actually really like Nao's solution, though clearly that has other downsides we're still looking into). However if we're talking about where things are going in the near future, I really hope we'll just have standard support for HDR pixels and texels rather than figuring out how to squeeze things into LDR and special-casing all the stuff that doesn't work.
Perhaps FP16 and above aren't strictly necessary in every case, but it certainly represents a reasonably easy and general-purpose solution to many problems, so why not go that way? It's maybe not elegant, but it'll work, and we can probably then find other things to do with the range.
Yes, you're right - the problem then is that you could get quite a bit of oscillation happening, rather than just a gradual interpolation towards a stable value. That can of course be compensated for to some degree, but it would still worry me to have a scheme based on feedback from a "broken" result. Especially where this feedback is happening right in front of someone at 60fps...
Speaking of which, is there direct support for HDR textures on modern GPUs or are they still 8bpp only and FP16 etc is only for framebuffers? If there is, what HDR image formats are supported?MrWibble said:
