When will we get double precision?

[K.I.L.E.R]

I realise that performance is a big issue but what about times when the bigger issue is numerical precision?

When will we see the minimum of double precision introduced in video cards for both fragment and vertex operations? When will we see double precision as a minimum standard?

 

[Zengar]

I thought Nvidia was going to introduce double presision hardware in second half of 2007? Why do you need it anyway? For GPGPU? I hardly see why numerical presision is a problem for games/graphics.

 

[_xxx_]

November, according to the current rumors.

 

[Nick]

I realise that performance is a big issue but what about times when the bigger issue is numerical precision?

When would that be?

When will we see the minimum of double precision introduced in video cards for both fragment and vertex operations? When will we see double precision as a minimum standard?

For pixels and vertices single precision is really enough. If you have problems with numerical precision it's very likely that you're doing something wrong. It can be tricky to fix but double precision is often the wrong answer.

I don't think there's any consumer GPGPU application which requires double precision. Only for scientific purposes I can imagine some uses. But for that they would still often prefer a reliable CPU over a GPU.

 

[tabs]

November, according to the current rumors.

That would be for the new Tesla line though, rather than the home models, I believe.

 

[Tim Murray]

That would be for the new Tesla line though, rather than the home models, I believe.

No ETA on DP for consumer models, mainly because there's not really any need.

 

[Andrew Lauritzen]

For pixels and vertices single precision is really enough. If you have problems with numerical precision it's very likely that you're doing something wrong. It can be tricky to fix but double precision is often the wrong answer.

That's a bit narrow-minded although unfortunately a common misconception. The example I usually give is summed-area tables which require quite a lot of precision and there are few - if any - ways to avoid that. There are many other algorithms that *could* be implemented with more precision but naturally we don't use them now because we don't have double precision

In GPGPU work - particularly scientific - lacking double precision is pretty crippling for many people. Even if *most* operations use single-precision, often there are a few operations that need high-precision. Thus it's not particularly bad if doubles are a bit slower, but they really need to be available at least.

That said both AMD and NVIDIA have said "by the end of the year", and I fully expect that to be the case. I'm sad that NVIDIA has elected to only expose doubles in their HPC line (particularly since they exist in HLSL, etc. already), but I can understand that they're grasping at straws for a way to differentiate their HPC parts

 

[_xxx_]

That would be for the new Tesla line though, rather than the home models, I believe.

G92 rumors, not Tesla.

 

[Arun]

but I can understand that they're grasping at straws for a way to differentiate their HPC parts

Well, I think I've said it before, but I firmly believe that HPC parts should not differentiate themselves from consumer parts based on feature-sets except in terms of reliability; rather, limiting the performance of HPC-oriented features seems like a much more elegant approach to me.

In the case of FP64, the solution in my mind is to: a) Make it take more clock cycles than on HPC solutions, in every ALU. This should not be done at the driver-level. b) Artificially limit how many multiprocessors can be working on programs with FP64 at a given time. This should be driver-configurable, so that if a game suddenly had a good reason to use more than that, it could be changed.

While the following point in my CUDA Introduction article was not aimed specifically at FP64 capabilities, but rather at GPGPU and CUDA in general, I think it illustrates the problem quite elegantly anyway:

Finally, we would like to take this opportunity to urge NVIDIA (and AMD) not to artificially restrict GPGPU development on consumer cards. While we fully understand the push for higher average selling prices (ASPs) in the GPGPU market, we would like to remind point out that emerging markets require developer innovation. Allowing anyone with a some coding skills and a cool idea to experiment on real hardware can be a key catalyst for short-term and long-term innovation.

At this point, it's really up to NVIDIA and AMD to do this however, rather than do things 'naively' and prevent long-term developer innovation.

 

[Nick]

That's a bit narrow-minded although unfortunately a common misconception.

I said "very likely" and "often". It's quite common that people want increased precision when faced with a precision problem, while there are other (and better) solutions. I'm not excluding that there are some very specific algorithms that demand double precision, especially for scientific GPGPU applications.

The example I usually give is summed-area tables which require quite a lot of precision...

Good example from an algorithmic point of view, but what's the practical value for consumer applications? We already have mipmaps and anisotropic filtering. Summed-area tables would require gigantic storage capacity and bandwidth. So I don't think double-precision support will suddenly make summed-area tables a popular technique.

Anyway, I definitely welcome support for double precision. I just don't think it's The Next Big Thing.

 

[Nick]

At this point, it's really up to NVIDIA and AMD to do this however, rather than do things 'naively' and prevent long-term developer innovation.

Lots of innovation comes from the academic world, where price nor performance really forms a limitation. Current consumer applications still struggle making innovations from many years ago practical. And I'm pretty sure that any cool idea can be illustrated with single-precision, even though it asks for double-precision for practical situations. So I don't think IHV's are putting a brake on innovation. And let's face it, what is innovation worth for the consumer market if only the latest high-end can run it? So I don't think there's anything wrong with keeping it out of the consumer market until it's really ready for it.

I wouldn't be surprised if only the next-generation high-end chips support double-precision anyway. It makes no sense to make mid-end chips more expensive, slower and hotter while there's no application that uses it, for at least another three years...

 

[Andrew Lauritzen]

Well, I think I've said it before, but I firmly believe that HPC parts should not differentiate themselves from consumer parts based on feature-sets except in terms of reliability; rather, limiting the performance of HPC-oriented features seems like a much more elegant approach to me.

I agree 100%, but everyone that I've spoken to at NVIDIA seems to use the feature-differentiation and "consumer GPUs don't need it" arguments, which are certainly irritating to me as a researcher.

Good example from an algorithmic point of view, but what's the practical value for consumer applications?

One example is this However SAT's have been used in quite a few academic papers over the past few years for stuff like depth of field and even lighting integral evaluation (see this years I3D posters - may be posted @ ATI/AMD's developer site).

 

[Nick]

One example is this However SAT's have been used in quite a few academic papers over the past few years for stuff like depth of field and even lighting integral evaluation (see this years I3D posters - may be posted @ ATI/AMD's developer site).

It's an awesome technique but I believe it proves my point. Mid-end chips would not be able to run this adequately in an actual game. So there's little point in equipping next-generation consumer cards with double precision yet. It will take a few more years before it becomes practical. Furthermore, single precision was not a limitation for your innovation. It would be better with double precision, but you've been able to fully proven your idea without it. Now we have to wait on Moore's law to make it interesting for the consumer market.

 

[Andrew Lauritzen]

So there's little point in equipping next-generation consumer cards with double precision yet. It will take a few more years before it becomes practical.

That's true, but it would be nice for researchers and developers to have the features available on a graphics card to work with *now* so that future applications can target doubles, etc. By my understanding, NVIDIA plans to have doubles only on HPC parts and not even Quattro's (tell me I'm wrong?) so no doubles for graphics at all!

 

[Blazkowicz]

G92 rumors, not Tesla.

G92 will naturally be used in home, Tesla and Quadro cards. which are about discriminating on features and drivers based on price, in a similar way to windows server vs windows for workstation.

 

[CarstenS]

Any chance to make this available cheaply by using 4 cycles instead of, say, 2?

 

[Nick]

That's true, but it would be nice for researchers and developers to have the features available on a graphics card to work with *now* so that future applications can target doubles, etc. By my understanding, NVIDIA plans to have doubles only on HPC parts and not even Quattro's (tell me I'm wrong?) so no doubles for graphics at all!

Ok, I've been assuming that there would be at least one graphics card with support for double precision. If that's not the case then I fully understand your gripes.

But what are the chances that they develop an entire chip+card just for HPC, without graphics functionality, and not even use that chip for Quadro's? There's probably some demand for double precision from the non-real-time rendering market as well.

In any case I can't imagine that you'd have to wait several more years.

By the way, does your application require double precision for generating the SAT or for sampling? If it's just the former have you considered doing it on the CPU? SSE2 is really quite fast for double precision.

 

[Farhan]

Any chance to make this available cheaply by using 4 cycles instead of, say, 2?

My guess is it would be more than 4 cycles if they wanted to minimize the hardware required. Maybe more like 8.

 

[Tim Murray]

Through what API can you get double precision for shaders?

 

[rendezvous]

Through what API can you get double precision for shaders?

Cg definitely supports double. It is treated as float in current profiles but that doesn't prevent any future profile to treat double with full precision.