More insight
TG should stand for Thread Group. So it's basically a resource descriptor for a linked group of wavefronts. "IA" is still unknown.
And I'm no longer sure if it's actually the CS Dispatcher which is holding track of commands in flight, but instead the "Draw Engine". So the CS Dispatcher is not keeping track of anything, it's only the Draw Engine which ensures that no grid is scheduled twice. There appears to be a queue in between the CS Dispatcher, and the CS resource allocator in the SPI block,. But it also means that (difficult to allocate)) thread groups already scheduled, can only be jumped by compute commands handled by a different ACE. Assuming that GCN 1.1 and 1.2 still only have a single CS Dispatcher per ACE.
The following
is just a guess, but I assume the Draw Engine works as follows in GCN 1.1:
It holds 8 unique pointers, one on each ring buffer. If one of these may be increased, it knows that it can fetch another command. In addition, it has another 64 execution slots for compute commands in flight.
When the next command is a barrier end, it will check whether the condition is met and skip this queue otherwise.
When the next command is a fence, it will check whether reference counter is zero and the condition is met and skip this queue otherwise.
When it is a regular compute command, it will check for free execution slots., assign the command, and probably increase a reference counter for that queue, and skip this queue otherwise.
When it is a barrier start, it will check if the reference counter is zero, and then trigger the transition, and skip this queue otherwise.
Whenever the attached CS Distributor is idle, it will evict the previously active compute command and reduce the reference counter on the corresponding queue. It will then continue to flag the next occupied slot as active and to commit it to the CS distributor.
So much for 1.1 behavior at least. I don't even think any more that it is actually command list aware. It's just doing a good job at pipelining. And it's simple. So simple that it might actually be full ASIC.
The HWS on 1.2 is slightly more complex. It is command list aware.
Question would be: How?
That's actually not looking like ring buffers any more, as skipping or offloading simply wouldn't be possible with ring buffers, at least not without copy. So it's probably one additional level of indirection, with the ring buffers only containing references to the actual command list. And a single side port per Draw Engine to hand over command lists to the other paired Draw Engine in case all 8 handles from the 1.1 Draw Engine are already occupied.
Not a big change actually, just one tiny indirection. But it's unlocking even more potential when dealing with undersized command lists, with up to 16 command lists from a single queue in flight concurrently, when I'm not mistaken.
GCN 1.1 only achieved a single active command list per queue, and the rest was just efficient pipelining.
Now, I'm wondering once again where Nvidia is failing?
Implicit fences between command lists possibly? Sounds like there is some batched cleanup performed on the end of each command list. That would of course block pipelining commands from subsequent command list, and explain why GCN is running circles around Maxwell when the command lists are too short. I wonder if Maxwell wouldn't actually profit from using multiple software queues as well. If there really is just an implicit barrier, it shouldn't have any impact on concurrent queues.
For once, I'm going to assume that they did not lie about the changes to async compute capabilities on Maxwell v2, that they actually do have multiple, concurrent compute queues available and that they are working.
Such an implicit barrier inside each single queue would still explain all anomalies we have seen so far. And AFAIK all "real" benchmarks so far only used a single compute queue as well, so they would naturally all run into this tiny catch when the workload is degenerated.
