Adding full random memory of a CPU

Actually, I would disagree a little with that. VPU's are no worse for random accesses than CPU's are, but the definition of 'random' is different for the two.

There are fundamental limits here. If accesses are truly utterly random, it doesn't matter how much cache or effort you spend in 'intelligent' banking, you'll end up on average with one page break per Nth memory access where N is your number of banks (that is, assuming that the 'cache' is much smaller than the memory size). Your effective burst lengths will be limited to your 'struct size' (if you see what I mean) and/or you may also incur an overfetch cost, if you do not make use of all the data in a cache line. These are impossible to get around. (Usually cache lines and burst lengths are tuned to be reasonably similar, so overfetch and bank switches are the key costs).

In order to improve things, you choose to optimise looking for coherence in your input data and optimise for certain classes of pseudorandom access pattern.

At the moment, VPU's and CPU's assume different kinds of coherence. CPU's use LRU caches, which one could view as assuming temporal coherence. VPU's make less use of these and more of latency compensation 'caches', prefetching, FIFOs and arranging data in memory to avoid bank switches - basically, we assume spatial coherence. Of course, there is some overlap: we have some LRU caches, and CPU's are now doing things like automatic prefetch in order to get spatial coherence on large arrays.

Neither has any optimisation for overfetch other than assuming that a particular bit of data will all get used before it's discarded (and, given the cache line / burst length relation, it seems unlikely that there is much in this area, at least while we use SDRAM-type technology).

Coherence optimisations are fundamental cost/performance tradeoffs. It would not be hard to add large LRU caches in a VPU, but it will cost area. That cost would be expended if there was a convincing business model behind the decision, but there already is a highly expensive, high performance, general purpose CPU in the device... I mean, what else is it going to do if the VPU ends up doing everything?

 

Interesting - my question would be, do these large LRU caches support the maximum throughput of the VPU device per-clock, in terms of reads and writes, and if so then does it have an effect on the cost of the silicon against, say, the equivalent sized CPU cache? To support full-speed, fully random access on current hardware you could potentially need to read and write multiple pixels in a single clock to unrelated memory locations in the cache. Of course, if the cache is allowed to throttle the performance of the overall system this would seem to be less of a problem, but it introduces a potential bottleneck.

This also seems a rather different case to the CPU cache situation, where typically you would not expect to generate a large number of writes and reads to unrelated locations in the space of a single clock, or even a small number of clocks. This is why my instinct tells me that it's not as simple as slamming a large cache on the back end of a VPU.

My immediate concern would be - do you end up in one or more of these cases:

- The VPU system goes relatively slowly with random access anyway, so it wasn't worth the extra area for the cache, and a CPU ends up just as quick for the algorithm in question.
- The cache slows down the maximum throughput of the device when rendering simple pixels to consecutive memory locations, and thus impacts the basic performance of the device when it's working on its most standard function.
- The cache can't support the full speed transactions of the VPU per-clock and thus is bypassed in some way for simple rendering, effectively becoming a big chunk of wasted area in this case (ie. when rendering typical non random-access applications).

Maybe I'm wrong here and I'm inventing issues where none exist - I must admit to not having thought the problem through in too much detail.

 

What ATI employee was being defensive here? Alright so it's pretty obvious you mean me, but I don't get WTF you're talking about.

I added a simple opinion on the topic, without trying to defend or even relate to any current, former or future hardware or ATI business, competitors or market situation. It was a comment entirely on the technical merits of such functionality. In what way does IHV employees shutting up raise the standard of a technical discussion? Do you have a personal problem with me or what?

 

Rev, I think you're getting a bit out of line...

But then don't expect anyone else to stay polite when you're telling them (indirectly) to STFU.

I think you're mistaking simple, factual explanations for defensiveness.

Disagree.

 

I didn't think it through in that kind of detail either

I'd have said ordering and coherence might be a bigger issue than the second of those. The first I already covered. The third is simply a cost/benefit issue, if there are scenarios where it helps significantly then the cost might be worthwhile even if it isn't worth it for 'rendering' cases.

 

What marketing pieces?

 

You know how those ATI guys bedazzle us with technical mumbo jumbo and then slip in “ Act now and we will include free shipping. Quantities are limited, operators are standing by." Sleazy bastards.