NVIDIA CUDA 1.0 released

[B3D News]

NVIDIA has released version 1.0 of its CUDA programming framework, with a number of new features including asynchronous kernel calls and 64-bit Linux support.

Read the full news item (http://www.beyond3d.com/content/news/304)

[Tim Murray]

I was all excited about writing global mutexes too, until John Stone told us that it wasn't supported by G80. :(

[Geo]

You need to see this as an opportunity. "But, John, I'm going to need you guys to give me an 8600 then so we can cover it the way it deserves. . . "

[Tim Murray]

You need to see this as an opportunity. "But, John, I'm going to need you guys to give me an 8600 then so we can cover it the way it deserves. . . "
Er, John Stone is the guy at UIUC who wrote all of their CUDA stuff (and probably knows more about making CUDA apps go fast than anyone who's not on the CUDA team proper). :p

[nutball]

Awesome news! Slightly worrying that two parts from the same family of hardware can have differing functionality like that though -- going to make for some compatibility-issues-from-hell situations. Oh well, that's progress I suppose!

[Tim Murray]

Awesome news! Slightly worrying that two parts from the same family of hardware can have differing functionality like that though -- going to make for some compatibility-issues-from-hell situations. Oh well, that's progress I suppose!
I really don't think it will. Like I said, the only thing that is different right now is the support for atomic functions, and I still can't really figure out why you'd ever want to use them. Performance is probably completely abysmal, for one, and I would imagine that you could do all of it on the CPU much faster.

[Geo]

And yet they added them to the more recent part. That says something.

[silent_guy]

I really don't think it will. Like I said, the only thing that is different right now is the support for atomic functions, and I still can't really figure out why you'd ever want to use them. Performance is probably completely abysmal, for one, and I would imagine that you could do all of it on the CPU much faster.

At this moment, the only way for multiple blocks to interact with eachother is by using really ugly hacks. With atomic functions, it's much easier and you can now implement semaphores and polling loops to align all blocks and restart calculating without the overhead of the CPU having to reissue a kernel.

If you only use __syncthreads intra-warp and 1 atomic function per warp for inter-warp synchronization, then maybe performance won't be all that bad?

I had a quick look at the SDK this morning and grepped for 'atomic': they have the histogram64 example where they use atomics on a 1.1 shader and reduction on a 1.0 shader. It would be nice if someone with a 8600 could try both and compare the execution speeds.

[Tim Murray]

At this moment, the only way for multiple blocks to interact with eachother is by using really ugly hacks. With atomic functions, it's much easier and you can now implement semaphores and polling loops to align all blocks and restart calculating without the overhead of the CPU having to reissue a kernel.

I'm just not convinced that it's going to be faster than just using the CPU to perform global synchronization. I also wonder how it's implemented (whether it costs two memory operations or what).

[silent_guy]

I'm just not convinced that it's going to be faster than just using the CPU to perform global synchronization. I also wonder how it's implemented (whether it costs two memory operations or what).

I assume you're hinting at using L2 caches in the ROPs to prevent full external memory round trips?

As for being faster or not than CPU based synchronization: It will probably depend on the amount of warps in play? For a smaller number, atomic operations will definitely have a lower overhead than a CPU relauch (PCIe latency etc.)
For a large number, atomic ops may have too many collisions and eventually CPU overhead will be smaller. My feeling is that you should be able to go pretty far with with atomics, before you hit a wall, by having multiple synchronization stages.

Anyway, it's definitely nice to have the option. Since I just want to play around with it, absolute speed is not my top concern: I may buy an 8600 instead of an 8800 just for this feature.

[armchair_architect]

I'm just not convinced that it's going to be faster than just using the CPU to perform global synchronization. I also wonder how it's implemented (whether it costs two memory operations or what).

No idea if this is how they've implemented it of course, but in the graphics pipeline, the z/stencil tests and color blending are all atomic RMW operations. So this is very similar to something they've optimized heavily before.