mm... Thanks. So for shaders & tex ops, does it come down to higher utilization (fewer gaps) with the lower latency?
Maybe I'll move this...
The VMEM documentation shows that this can be complicated.
One thing of note that I was curious about when GCN was announced: vector memory operations monopolize the vector memory path. While it is true
It breaks a vector memory operation's execution into 4 stages once a wavefront reaches the point of issuing it:
1) Issue: send operands for this instruction down three parallel buses. Command (type of access, texture and sampler data if necessary), Address (number of coordinates), Data (if writing)
Whichever takes longest determines the cycle count, with buffer accesses being simplest and fastest.
2) Address computation and data request
Depending on the type of request, particularly the kind of filtering, multiple addresses are computed and sent. A memory instruction can sit many cycles here spitting out addresses for the L1.
More complex cases like anisotropy can actually be a bottleneck here.
3) The requests go to the L1, and it starts collecting the necessary data. This is subject to what happens to what the rest of the system recognizes as memory requests.
This would be the stage that is heavily influenced by memory latency. The cache is described as being non-blocking, so L1 is able to overlap latencies by keeping multiple accesses in flight. This means that if there are one or more misses, the latency of a miss can be overlapped by other hits or even other misses. This, and measures to increase locality can cause the more complex accesses to not scale to insanely high latencies while also making the issue and address generation latencies a measurable portion of overall latency that memory has no influence over.
4) When the L1 gathers the data, it submits the data to the path responsible for filtering and conversion, and this then returns to the shader. This is subject to the filtering/blend rates, which neither DRAM nor ESRAM can influence.
I think, going by this, that the 125 or so cycles saved matters more for accesses that have very low issue overhead and filtering/conversion which means buffers and possibly compute in particular. The next requirement is something like a producer/consumer relationship or poor enough locality.
This may lend itself to compute, and perhaps the ROPs could find this useful.
This might reduce the necessary level of occupancy to hide memory latency somewhat. Various matrix multiple kernels already favor heftier contexts and perversely lower occupancy in order for better cache blocking / register use / data share use, so perhaps more aggressive measures can be taken without being hindered by reduced latency hiding.
I saw somewhere, but cannot remember where, about divvying up the workload so that at least for some scenarios texturing can be satisfied by DRAM because it is latency tolerant and not the heaviest bandwidth consumer.
ROPs burn serious bandwidth to hide their limited tolerance for latency, so there may be a benefit where reduced latency also means the ROPs can use less bandwidth or be better utilized.
In the case of ROPs, it might be nice to know if their latency is different, like if they can bypass even more with the ESRAM than the vector path can.
