Thus it's very hard to make the CPU and GPU run in lock step without stalling either of them.
Running in lockstep (or almost lockstep) is very hard on PC, since the CPU and GPU performances vary so much.
...
This means that the CPU has submitted the whole frame to the command buffer 25 milliseconds earlier than the last GPU command finishes. There's no way that untouched data in 8 MB L3 cache shared by CPU and GPU lasts for 25 milliseconds.
This is actually partly a response to some of the earlier discussion on transactional memory, that I was too slow to think about.
The first point on lockstep is interesting, in the sense that true lockstep execution is generally avoided outside of high-RAS setups where cores are purposefully slaved together at the level of silicon or hypervisor/firmware, and it's a non-trivial thing to accomplish.
I think those developing for discrete PCs were insulated by the horrendous latencies, but I'm not certain even the current console APUs have peeled away quite enough of that latency to see the gnarly reality underneath. Perhaps "lockstep" here is more tolerant up to some number of stall cycles.
To segue into the transactional memory bit: the discussion of short-lived 8MB cache.
At least for architectures like GCN, I'm not certain transactional memory is a win with GPUs as we know them. Looking Intel's somewhat mottled introduction of limited transactional support, we have transactions that currently rely on keeping their write set in core-local storage with a likely short wall clock period for when a transaction becomes globally visible.
GCN is not a very good fit for that. Its concept of coherence relies on missing the CU-local L1 and writing to a very dumb L2. Intel's transactions will fail if there's an eviction outside of the limited storage domain (not always if it's a line in the read set, not sure what that's about), and GPUs miss a lot.
IBM's Blue Gene implementation has an L2 that is capable of monitoring and invalidating a smaller number of L1 caches, which is a much more robust memory system. Its memory subsystem is probably faster at operations like draining queues and otherwise preparing for transactions compared to what it takes for GCN GPUs to drain their queues.
Because GPUs do so well on miss-prone and/or bandwidth hungry workloads, the likely amount of data and the length of time it takes to make anything globally visible makes an Intel or IBM transaction look a little shaky to me. There's such a wide window of time for transaction-aborting events on so much data, and there would be overheads associated with getting a pristine transaction set when memory is so inconsistent.
The SIMT paradigm is another headache, or at least could be. I'm really not sure what AMD's position is on SIMT or SPMD. If AMD continued on insisting each work item were a thread, the chance of 1 of 64 "threads" annihilating a transaction when everything else operates at the level of a wavefront sounds unappealing.
If the hierarchy doesn't change, it might work better if versioning were built into specific areas of memory on-die or physical addresses at the memory controller level, although this still seems like it's going to incur additional latencies.
You know, I had a thought. (wide) SIMD is something of a worst case for OOO.
The SIMD instructions themselves should be well-behaved for rename or the tracking of hazards and operand bypass.* Vector pipelines and FPUs also typically operate in a less precise manner than the more unforgiving standard the integer pipeline is held to.
*Unless you start playing with the number of operands, or start to get cute with the scheduling.
That the integer pipeline in various workloads does get a decent workout when the SIMD is running well can encourage a broader design than a core that can target one more than the other.
The memory subsystem is one point where the sides of the core don't always share the same tradeoffs, what with one being happy with smaller data elements and intolerant of latency, while the other latency tolerant, wide, and possibly involving some shifting or conversion as part of the process.
It's not actually required that they share all of it. Itanium, for example, had had FP loads and stores go directly the L2, but it's a slippery slope for those who want to maintain some kind of aesthetic purity. I mean, once you start differentiating one of the dominant architectural facets of a von Neumann machine, what else might you be tempted to split off...?
So about that register file: CPUs often use register renaming to effectively increase their register file size. The problem is that register renaming isn't as versatile as simply having more registers.
The renamer doesn't have to stop at the same points as a compiler. It can take whatever registers the CPU's dynamic execution goes down without care if said instructions are in a function the compiler could not optimize for.
Optimizations that bloat the instruction stream can lead to more instruction cache misses. It's something that early dynamic optimization schemes like Dynamo could take into account since they could target an OoO core. A loop the optimizer can leave as-is can mean multiple cache lines not at the risk of thrashing, or a powered-down front end in the case it fits in a loop buffer.
Being able to handle loops that can be renamed relatively painlessly is one area where I think Denver could have some architecturally-specific optimizations for, or could significantly to improve if it did go OoO someday.
The size of the register file and how it's designed is also an important consideration if a core is targeting serial performance, which at least I hope some will still try for in the future.
I'm going to go one step further and say that excessively IPC-focused CPUs are a temporal fluke that only matter so much due to the lack of innovation in the programming language world.
This is begging the question. If it's excessive, why should a thing persist?
It's also a bit too late to call decades of increasing single-threaded performance a fluke, up until this most recent chapter, the whole of modern computing was underpinned by it.
Reality keeps asking our idealized architectures to run code that wasn't written by the binary angels, so I admire hardware that isn't perfectly circumscribed by its glass jaws.
I don't see a zero-sum situation, although may arbitrarily declare things unacceptable if the whimsy strikes me.
One counterpoint to having everything on one die is a physical one. AMD's CPU cores have suffered mightily transitioning from a physical process that was tailored for them, and on the flip side the GPU portions of their APUs have had a progressively easier time as the CPUs have lost ground.
There are still gains to be had from recognizing different physical requirements, but the relative cost of going off-die these days is quite high.
For AMD, and probably any consumer device manufacturer other than Intel, one-die makes even more sense because it's unlikely they have can have a bleeding edge CPU or the process+engineering to go with it.
Intel is at an odd point, currently.
Intel's new process has tightened the pitch of its interconnect layers, which has closed a density gap with foundry processes it's had since 65nm or so.
Broadwell-Y shows a changed physical target for its design, which may have figured into the claim that Intel had shifted to a 2:1 perf/power tradeoff requirement, which leaves me antsy for a non-mobile variant or the advent of Skywell.
OOO hardware does not by any means schedule optimally! It effectively uses a crude greedy algorithm, which is likely worse than what a compiler could do offline.
The exact workings of today's high-end CPUs can be more complicated than that. It's not something that frequently gets disclosed, but if there were things that that deserve the label of "secret sauce", the internal settings, design tweaks, and heuristics that have been iterated and implemented in the real world for decades would be good candidates.
I doubt it - where did you hear this?
GCN is pretty heavy on the 4-cycle VALU loop, and most operations in the ALU category tend to be fixed within an implementation (but can vary between designs, like DP).
Nvidia's are pretty regular as well, although last I checked they had more variety.
GCN appears to divide its CU issue capability down into a series of imperfectly abstracted domains, where certain sets of operations with reasonably similar behaviors get managed semi-independently.
Certain things do break the veil, and necessitate explicit wait states.
CPU ops are more free to have varying latencies, but those that may vary a lot sometimes are only partly pipelined or get handled by microcode.
Or you could do a JiT stage and schedule specifically for the processor in question. Instruction scheduling is cheap - so much so that it can be done dynamically at runtime millions of times for each instruction in a loop.
At least current designs are going to have a problem. At some point the system is going to write the new optimized instructions out to memory and at some point a core may need to fetch them.
The optimizer needs to balance the cost of its own optimization and the cost in time and energy incurred by having to invalidate cached instructions and fetching them.
The thing about doing things in silicon is that they probably will mess up from time to time, but their mistakes in many cases can be small or may converge very rapidly with the lowest external cost.
