For GP100, my understanding is that it was done mainly for FP64. Of course it's nice to have more registers, but they and more importantly their connection fabric to the ALUs themselves increase (of course) up space, power and complexity.
AMD: Speculation, Rumors, and Discussion (Archive)
- Thread starter iMacmatician
- Start date
- Status
For GP100, my understanding is that it was done mainly for FP64. Of course it's nice to have more registers, but they and more importantly their connection fabric to the ALUs themselves increase (of course) up space, power and complexity.
Does AMD buffer operands for the entire wavefront first or are the reads done on the fly?
I didn't think it was either; I was just spinning on what previous poster posted.
Example.
GPU A CU takes instructions from a single wave. It can start a new instruction from the same wave every cycle (and has low enough instruction latency / reordering to keep the pipelines mostly filled). Let's say it fails to find an independent instruction 20% of the time. Result = 80% ALU utilization, IPC (of a wave) = 0.8.
GPU B CU on the other hand can start a new instruction every 4 cycles from a single wave. It takes instructions (round robin) from 4 waves to guarantee 100% ALU unit utilization (when not waiting for memory). Result = 100% ALU utilization, IPC (of a wave) = 0.25.
GPU B needs to have register space for 4 waves per CU, as it runs them all at the same time. GPU B thus need 4x larger register files to operate efficiently. Low IPC (from wave's point of view) is a trade-off, as it means that every single wave is executing for a longer time, reserving registers. Also running more waves concurrently tends to trash caches more, unless caches are also bigger (and have more associativity).
CU can run as many waves as there are registers (or LDS if that is the bottleneck). If shader is complex, it requires high amount of registers. If not enough waves can be concurrently running, there is no memory latency hiding. Utilization issues are caused by memory stalls. All waves on the same CU are waiting simultaneously for memory. GPU doesn't issue any instructions to the SIMD.
I have been wondering why GPUs haven't already switched to two-tier register files. Most of the registers are not used right now, but are instead just storing some loaded data for later use (compiler statically does this to hide memory/cache latency as there is no OoO and no register renaming). A big low latency register file near the execution units is expensive and consumes a lot of power. Why not keep "storage" registers that are not used right now a bit further away from the execution units. This way you should be able keep the main register file smaller and its latency lower (and distance to main register file smaller -> power savings).
Nvidia did a move into this direction with Maxwell, when they introduced operand cache (basically a tiny L0 register file). It lowers the register file power consumption. I am wondering whether this change was one of the factors allowing them to double registers per thread in Pascal, now that the new process gives them the die space to spend for more registers. If AMD increases their register file size in Polaris, I am curious to see their approach. A full blown 2-tier register file would be very interesting.
Interesting read about this topic:
Nvidia did a move into this direction with Maxwell, when they introduced operand cache (basically a tiny L0 register file). It lowers the register file power consumption. I am wondering whether this change was one of the factors allowing them to double registers per thread in Pascal, now that the new process gives them the die space to spend for more registers. If AMD increases their register file size in Polaris, I am curious to see their approach. A full blown 2-tier register file would be very interesting.
Interesting read about this topic:
Jawed
Legend
A general observation about CUs: the hardware has a register file (or several), a shared memory and a L1 data cache - usually in that order of descending size.
NVidia unified L1 and shared memory. Registers stand as the odd one out.
Larrabee architecture uses L1 for all three roles: L1, work item state and shared memory. It then has a small register file, AVX style (and, obviously, an x86 RF too). An operand cache, if you like.
NVidia unified L1 and shared memory. Registers stand as the odd one out.
Larrabee architecture uses L1 for all three roles: L1, work item state and shared memory. It then has a small register file, AVX style (and, obviously, an x86 RF too). An operand cache, if you like.
I was under the impression they unified Texture Cache and L1 with shared memory still being separate. Do I remember incorrectly?
http://docs.nvidia.com/cuda/kepler-tuning-guide/#shared-memory-capacity
Since Fermi, Nvidia has had configurable 64 KB memory pool per SM. You can configure L1 / shared memory split. Options: 16/48, 32/32 and 48/16.
Yep. Maxwell merged the data L1 with the texture cache.
Completely agreed. Specially when you think that you don't hype your partners showing them a "high level views"? Didn't Anandtech already showed that months ago? You hype your partners sending them an email saying "our new chip will cost x amount of money and the reference card will cost x amount of money, so we expect the profit from every card to increase x amount of money". In my opinion that is the language they talk and/or understand, how much money that can make, and not the "new tech we will introduce" Which again...they already talk about....
This is even worse because of the timing. They are tweeting in just after the Nvidia presentation, so people are waiting for AMD response. Also they place the event right after Nvidia NDA...again people will be waiting AMD response, and AMD saying "Hey we have an announcement to make!" and then "oh no, that wasn't meant for you, it was meant for our partners" like really?
Someone in AMD marketing department needs to go home.
Do you know they haven't for a fact? This paper: http://dl.acm.org/citation.cfm?id=2155675 came out quite a few years ago, for example.
I am aware of that. You cannot, however, do that with Maxwell any more. It's open for debate if that's a hard limit or only imposed via software restrictions. I would guess that for power reasons, simpler hardware mostly is more efficient and tend towards the first option: hard limit.
This paper describes a compile time (static) technique. You should see it clearly in the GPU shader microcode. CodeXL shows GCN Gen1-3 microcode, and it doesn't have any hints to tiered register file. Neither does AMDs ISA documents or AMDs open source linux drivers. Same is true for Intel's open source linux drivers and ISA documents.
Are there any public low level ISA documents for modern Nvidia GPUs? PTX is a virtual higher level ISA and it doesn't expose low level stuff like tiered register file or operand reuse. Are there any open source drivers available?
Maxas. It's not official, but it's public and it's pretty well documented.
https://github.com/NervanaSystems/maxas
OpenGL guy
Veteran
I assume you mean that GPU A is supposed to be Nvidia Maxwell or Pascal. You should note that Maxwell takes at least 4 warps per SMM to get peak ALU rate since there are 4 vector units per SMM. A single warp per SMM can only harness, at best, 1/4 of the SMM's ALU horsepower, and at worst 1/24th.
- Status
