[MfA]

Would be nice ... and almost free as far as transistors is concerned.

[Rayne]

Originally Posted by rpg.314

Can we have some more details please for that?

I barely remember anything, because it was year 2006/7, but i remember that with a single buffer for the DMA transfer, the SPE was idle most of the time. But, with multiple buffers and overlapping the computation on one buffer with the data transfer in other, the results were better. This is a must for any Cell app imho. I also remember problems trying to multi-thread my code because there was shared data across the threads.

Originally Posted by rpg.314

It's really a shame that SSEx isn't orthogonal. Why does intel have to design every ISA (save lrbni) like they are brain dead or something? For your problem, may be loading the four values into a aligned float[4] array and then a load may be faster

Yeah, i did something like that, but with the suffle instructions & several registers.

Originally Posted by rpg.314

That would be fun. They do have an optimized 4x4 transpose macro available in if you use the intrinsics.

I remember using the _MM_TRANSPOSE4_PS macro, but it needed the 8 XMM registers, and sometimes, you are using some registers to store some 'previously suffled' data

Originally Posted by rpg.314

That would make a lot of effort put into exploiting memory hiearchy useless.

Don't shoot the messenger

[nAo]

Originally Posted by TimothyFarrar

How about banked vs cached memory in terms of vector scatter/gather. Clearly read only vector gather can be covered by the texture units, so I'm talking about R/W vector scatter/gather only.

Cache memories are banked as well so I am not sure the premise is entirely correct, though I still get what you are pointing at
Also gather done via texture unit doesn't sound like a good idea if you are going to re-use your data multiple times.

A LRB hyperthread doing scatter/gather can only access one L1 line per clock, so if you actually do a scatter/gather which doesn't simply load from the same vector sized cache line, then performance suffers proportional to the number of L1 lines hit. In my eyes, this vastly marginalizes the usefulness of scatter/gather.

This is a scalable approach, which can get faster in the future. Doesn't nvidia operate in a very similar way for (uncached) global memory accesses?
Unfortunately scratchpad memory based programming models don't scale so nicely.

One can compare and contrast to current NVidia hardware where scatter/gather to random memory locations runs at full speed as long as each lane of the vector accesses a separate bank of memory. This opens up a lot more flexibility with scatter/gather compared to the LRB approach in terms of keeping up ALU throughput.

It certainly fast, but I wouldn't call gather/scatter from a minuscule memory that you have to manange on your own 'flexibile'

[MrGaribaldi]

Originally Posted by Jawed

It's in the AMD presentation. Firmly stuck in CPU land.

But why is that? They don't give any reasons in the slides, although it looks like the ME is taking "too long".

Nvidia has a nice CUDA sample for doing DCT on the GPU with 2 different kernels that are enjoyably fast of themselves, and can be further tweaked, so I don't see why you couldn't offload it to the GPU as well.

Granted, I've only played around with doing MJPEG compression on the Cell and GPU, so could very well be that I'm missing some of picture for doing .x264 encoding...

[rpg.314]

Blockwise DCT should be very nicely parallelizable. Also cuda helps as it exposes the dedicated video decode hardware, alteast on windows, so only the last lossless compression needs to be done on the CPU. ME and DCT can be both on GPU.

[rpg.314]

This is a scalable approach, which can get faster in the future. Doesn't nvidia operate in a very similar way for (uncached) global memory accesses?
Unfortunately scratchpad memory based programming models don't scale so nicely.

I'd like a detailed explanation on this please. It seems to me that you are implying that the hardware managed caches (with s/w hints like on lrb if need be) scale better than purely software managed caches like on gpu's.

[Jawed]

Originally Posted by MrGaribaldi

But why is that? They don't give any reasons in the slides, although it looks like the ME is taking "too long".

Yeah, shame there's nothing more detailed.

Is there an NVidia presentation on the details of h.264 encoding? Anyone else done one for CUDA encoding?

Nvidia has a nice CUDA sample for doing DCT on the GPU with 2 different kernels that are enjoyably fast of themselves, and can be further tweaked, so I don't see why you couldn't offload it to the GPU as well.

There's a CAL sample for DCT too.

Granted, I've only played around with doing MJPEG compression on the Cell and GPU, so could very well be that I'm missing some of picture for doing .x264 encoding...

I don't understand the h.264 encoding pipeline at all well, so I don't know how realistic ME and DCT both on the GPU is.

Jawed

[TimothyFarrar]

This is a scalable approach, which can get faster in the future. Doesn't nvidia operate in a very similar way for (uncached) global memory accesses?

Not to my knowledge, global accesses (ie uncached) are also banked addressed (just like shared memory) with CUDA. I'd argue that in terms of bandwidth limited cases, that having the extra 16x addressing capacity (16 bank addresses, vs 1 cacheline) per global access can be quite a performance benefit (well assuming the programmer can make use of it).

[trinibwoy]

Global memory accesses don't seem to be banked. Coalescing is contingent on all required addresses being present in the same contiguous memory segment. However the segment size can be either 32, 64 or 128 bytes. So global memory access works similar to the LRB cache where there is one read per segment (cache line).

[nAo]

Originally Posted by TimothyFarrar

Not to my knowledge, global accesses (ie uncached) are also banked addressed (just like shared memory) with CUDA. I'd argue that in terms of bandwidth limited cases, that having the extra 16x addressing capacity (16 bank addresses, vs 1 cacheline) per global access can be quite a performance benefit (well assuming the programmer can make use of it).

Just checked CUDA docs and global memory accesses are "simply" coalesced, which sounds similar to what LRB does (caching aside..).
Also this cache vs bank memory stuff doesn't make any sense, cache memories *are* banked.

[pcchen]

Originally Posted by nAo

Just checked CUDA docs and global memory accesses are "simply" coalesced, which sounds similar to what LRB does (caching aside..).

Yes, global memory access is basically just following the memory controller pattern. However, in the case of GT200, it seems that there's some sort of a reorder buffer between the memory controller and the ALU, so the coalescing rules are much more relaxed than G80.

[nAo]

Originally Posted by pcchen

Yes, global memory access is basically just following the memory controller pattern. However, in the case of GT200, it seems that there's some sort of a reorder buffer between the memory controller and the ALU, so the coalescing rules are much more relaxed than G80.

Yep, that's pretty cool.

[Jawed]

An example of the effect of the improved memory controller in GT200:

http://dl.getdropbox.com/u/484203/Le...lutionSoup.pdf

Jawed

[rpg.314]

Global memory accesses don't seem to be banked. Coalescing is contingent on all required addresses being present in the same contiguous memory segment. However the segment size can be either 32, 64 or 128 bytes. So global memory access works similar to the LRB cache where there is one read per segment (cache line).

If 64 threads issue contigous reads, each thread reading 16 bytes, then you have fetched 1k data from memory. Probably Timothy is referring to the fact that that 1K will be fetched by different memeory controllers and then merged together iin the register file, hence the banking. Elements of this technique re already there in CPU's with multi channel memory controllers, so I suppose it is an issue of semantics.

[Simon F]

Originally Posted by pcchen

For example, I am not sure if CABAC can be done fast on a GPU.

It's damned difficult to do fast on anything

FWIW Encoding is probably easier than decoding which is ironic as the latter is surely going to be more common.

[rpg.314]

For decoding we already have dedicated hw in gpu's today. So transcoders most likely will be taking advantage of it.

[T.B.]

Originally Posted by nAo

Yep, that's pretty cool.

It really simplifies the code in many cases, which can give you a nice speed-up. For my biggest algorithm, I got some 30% just by removing the ungodly-complex-GF8-coalescing-code with a very straight-forward partially coalesced one.

Staying on topic, I'd say having a little bit of smarts in the way you process memory accesses is often quite useful, especially if you are not on a fixed platform. Cell suffers from not having this, but makes up for it with 6 cycles latency. It would be interesting to know how much the GT200's reorder-buffer costs in terms of latency.

[rpg.314]

I suspect it is about 50-100 cycles. In initial CUDA docs they said that 400-600 clock cycle latency should be expected. But when volkov published his benchmarks, he found more like 500-700 clocks on GT200. This is a rather crude estimate, I admit.

[T.B.]

Originally Posted by rpg.314

I suspect it is about 50-100 cycles.

That sounds waaaaay too high for me. Like an order of magnitude too high.
OK, the reorder is at base-clock, not shader clock, which I assume you meant, so that brings it down a good bit.
Still, I may be totally off here, but even 20cy is a long time.

[rpg.314]

The numbers there are definitely in terms of shader clocks. However, considering the logic involved, 100 cycles is indeed high. However, it may well be the case that nv wanted to minimize the area used (as it is CUDA specific) so used a smaller coalescer many times to reduce an already bloated die. Like I said, it is a crude estimate.

[CouldntResist]

Only Arm and Core will prevail. Other architectures (nvidia, amd, larrabee, itanium, cell) will be assimilated or obsoleted.

Resulting polarity on the industrial scene, will be seed of epic conflict within human civilisation, destined to last for eons. Eventually spreading over entire galaxy, the conflict will outlast biology based life form of humanity. Our cosciousnesses, now encased in machine shells, will be occupied with the ultimate goal: complete annihilation of the opponent, before the Heat Death of Universe happens...

[T.B.]

Originally Posted by CouldntResist

Only Arm and Core will prevail... Eventually spreading over entire galaxy...

Galactic Arm vs. Galactic Core? Hmm....

[TimothyFarrar]

Originally Posted by pcchen

Yes, global memory access is basically just following the memory controller pattern. However, in the case of GT200, it seems that there's some sort of a reorder buffer between the memory controller and the ALU, so the coalescing rules are much more relaxed than G80.

Oops, that's what I get for a quick post in the local Apple Store this weekend. I was referring to the ability of the GT2xx series to reduce access size. Specifically the ability for the hardware to reduce global access requests from 128 byte segments to 64 byte segments or 32 byte segments. My mistaken 16x addressing factor is really a 4x addressing factor with the reduction from a 128 byte segment to four 32 byte segments.

The 32 byte segment turns out to only be 1/2 the size of a LRB cache line. Marco's point is indeed a good one for global memory accesses.

[trinibwoy]

Originally Posted by nAo

Just checked CUDA docs and global memory accesses are "simply" coalesced, which sounds similar to what LRB does (caching aside..).
Also this cache vs bank memory stuff doesn't make any sense, cache memories *are* banked.

Sure, but with the shared memory stuff a single read can pull data from all banks simultaneously at arbitrary offsets within each bank. Aren't single-ported cache reads rigidly limited to pulling a single contiguous cache line at once?

[Jawed]

Originally Posted by trinibwoy

Sure, but with the shared memory stuff a single read can pull data from all banks simultaneously at arbitrary offsets within each bank.

What algorithms need to do this? Why? When are these offsets arbitrary, but not random?

Jawed