I guess some of it depends on whether the scheduler is unified or not. When executing 1024-bit instructions certain buffers would become full so things like register rename can be clock gated for a while. Note that Sandy Bridge's uop cache can cause fetch and decode stages to be clock gated. If I recall correctly it takes a couple cycles to start them up again, but it's well worth the power savings.
This would be most immediately implemented by breaking the instruction into multiple uops. The allocation logic would lump together adjacent register locations, and the renamer would only care about the initial location when doing the renaming.
The uops would be dispatched in sequence, with a completion flag that would not be flipped until all have completed.
It's not just the L3 latency. You need to send a large batch of data over, issue commands, wait for it to finish processing, and pull the results back in. Also, if multiple cores are doing this dance then you'll quickly get bandwidth limited. A lot of power and cycles go to waste doing nothing useful.
I doubt it's growing. Multi-core, wide vectors and FMA each enable the CPU to catch up with the GPU. For many years the GPU was able to increase performance/Watt by dedicating every more die space to shading units, but now they're out of space.
They're only slowly starting to get used to multi-core programming. You really think they'll ever become "experienced" at programming a wide variety of heterogeneous chips? And I'm not saying the average developer wouldn't be capable of writing such software; it's just not worth all the effort. Something that looks good on paper will be an utter failure if developers are not eager to invest their time and money into writing software for it.
Not the entire core. Intel has started to use fine-grained aggressive clock gating to reduce the power consumption of logic blocks when they're idle. Agner Fog discovered for instance that Sandy Bridge clock gates floating-point components, and might even power gate the upper 128-bit of the AVX units.
I'm not talking about increasing serial performance. I'm talking about keeping it while also turning the CPU into a homogeneous high throughput architecture.
Would there really be a need to explicitly break it into multiple uops to achieve this?
yeah but in GPUs you have a TON of really dumb computing units that are relatively inefficient at executing spaghetti while normal CPUs have tons of die space dedicated for OoO execution and massaging that spaghetti into something that can be executed effectively. Die area dedicated to cache sizes is in a somewhat similar situation.
Yes, wide SIMD units with FMA and other fancy things definitely help but as long as cores still drag their "legacy" single-thread acceleration stuff with them GPUs can always fit more FLOPS inside similar TDP/transistor count.
A "large batch of data" resides in the caches (or the memory) anyway. You won't transfer just some hundred bytes from registers for a few additions, of course.
It would be the most expedient way to implement it, by leveraging the logic that is already in place.
The register allocation logic already has the knowledge of the register ID and the logic needed to increment it for each successive uop.
The logic downstream currently doesn't generate new register IDs, because that is determined when the uop was created.
It seems possible that a scheme could be created where the scheduler or execution unit has additional stateful behavior, and it knows to loop on a uop and increment the register ID sent to the register file or bypass network for each cycle. It's a more complex solution and would require more work and time to implement.
This may add some latency to the process. When the register ID is static, the hardware can send the value out in parallel to various places in the same clock, like the register file access logic and bypass checks. In a stateful case, the ID value needs to run through a small extra round trip with an adder and cycle counter before it can be routed to those places.
There are probably other ways of doing it, but that is what comes to mind.
Clock gating would be a win if you can find 2 (and more like 4 cycles) idle. I don't think that will happen realistically.
In a well designed heterogeneous system, this overhead will be the same as the overhead in a homogeneous system.
It will be a total waste of effort. Developers will soon be pushing anything might need high throughput to on die GPUs. Intel will go down this road nonetheless and drag AMD with it.
The amount of dumb FLOPS isn't all that high. The Intel HD Graphics 3000 can only do 130 GFLOPS, while the Sandy Bridge cores can do 220 GFLOPS. Add FMA support and you're looking at 440 GFLOPS (at only a minor increase in area). Sure, power consumption for the CPU cores is higher due to clock frequency, but is anyone really going to opt for pushing his calculations to a 3.5 times weaker IGP, which probably has trouble getting anywhere near its theoretical performance? GPGPU on the same die is a dead end.
I think you're underestimating the amount of die space dedicated to various forms of storage on a GPU. Keeping thousands of threads in flight requires lots of register and stack space, to save on RAM bandwidth ever larger data caches are needed, and running long kernels concurrently requires larger instruction caches. Heck, Sandy Bridge's IGP even has access to the entire L3 cache, and it's still bandwidth limited. Some suggest adding a slab of eDRAM. Great, but then stop pointing at the amount of die space a CPU invests in storage as a bad thing.
Besides, 8T-SRAM is quite power efficient, so as long as it increases performance/Watt over other uses, it's a good investment of the transistor budget.
Sure, but you've got the solution right there. Manage the ratio of power consumed by control and computing logic. Ever since the Prescott architecture, Intel has been hard at work implementing ways to improve performance/Watt. Haswell will have a whopping 8-fold increase in FLOPS per core over Prescott, and there's talk of significantly reducing TDP, especially for mobile parts.
I believe eventually brute-force out-of-order execution will be replaced with alternatives that offer a better compromise between power consumption and serial performance. Intel's "Continual Flow Pipelines" research is one possible route, but I'm sure there are others worth evaluating too.
Suffice to say that there are plenty of parameters to tweak to continue scaling performance/Watt for a homogeneous architecture.
Now tell me about GFLOPS per transistor for the two architectures
No it doesn't. You still have to assemble the data into a batch suited for processing by a specialized heterogeneous component. A homogeneous architecture on the other hand can process smaller amounts of data much more efficiently, as the data likely never has to leave L1/L2 cache till it's done processing.
Exactly! And there are tons of cases like that.
So instead of moving workloads "to the particular area of the chip capable of executing that workload the most efficiently", why not make sure the place to be is the same core, using a suitable set of instructions?
Yes, the power consumption of out-of-order execution poses a challenge, but can be managed from one generation to the next. In contrast, memory bandwidth and latency don't scale well. So things have been evolving in favor of homogeneous processing for quite a while. Heck, the earliest graphics cards processed only one or a few polygons at a time. Nowadays we have to go to great lengths to ensure that the massively parallel GPU can limp behind a couple frames! There's no way to scale that any further, especially since GPGPU applications want to read back results sooner, not later. So the GPU has to get closer to the CPU, both physically and architecturally...
The future is homogeneous, and Intel is taking a head start.
You don't have to compute the full arithmetic difference of each index value. You just have to compare the upper bits for equality, and check whether the sum of the lower 7 bits of the base and index cause an overflow into the next cache line. Since the results aren't needed till a later clock cycle, it seems trivial to provide small and power efficient dedicated hardware for it.
I'm slightly more concerned about the hardware needed to extract multiple elements simultaneously. Considering that cache line sizes of 64 bytes have been common for a decade it might not be that big a deal to have multiple of these shift units though. Larrabee is even supposed to be capable of gathering up to 16 elements per cycle, and has smaller cores, so I'm optimistic. Latency might be affected, but there's actually an elegant solution for that; see below.
Two read ports, one write port, all 256-bit. It's pointless to feature two 256-bit FMA units if the caches can't provide sufficient bandwidth. x86's limited number of registers, and the fact that non-destructive AVX instructions help increase throughput, make this an absolute necessity.
Indeed this affects the gather implementation. I imagine either each load unit has a lightweight gather implementation, they cooperate in some way, or only the second one has an advanced gather implementation. The latter would be an interesting option since it allows for the second unit to have a higher latency. If most other load operations use the first low latency port, it would hardly affect legacy workloads.
Bulldozer's FlexFP unit can execute up to four instructions each cycle. Sandy Bridge on the other hand borrows some of the integer data paths for the 256-bit operations. So it seems cheaper to me to equip Haswell with two 256-bit FMA units, than to extend the FlexFP unit to sustain two 256-bit operations.
According to some rumors, they might extend macro-op fusion to pairs of mov and ALU instructions, turning them into a single non-destructive operation (if applicable of course). This would merely affect the decoders. The ironic part is that current compilers actually avoid emitting such code, so legacy code may not observe much of a benefit. Recompiled code (which is still perfectly compatible with older x86 chips) could run a bit faster though, and just like test and branch macro-op fusion this technique would slightly improve performance/Watt.
That makes more sense, I wasn't thinking it through. Equivalence checks for the high-order bits could be done in parallel with the small adds. Does this cover the case where the add from an index overflow to a value that matches the cache line for another index, or will an additional check after the adds be needed?
This could be done using a permute instruction. If the operations are fully pipelined, a string of gathers can reach a throughput of 16 elements per cycle.
It's potentially more complex outside of that case. Do you see if there is a part of the description for the permute instruction that allows it to selectively permute values to a destination without writing over any other elements, or would the chip need to run a permute and then a blend to get the effect?
The gather could be microcoded, so the load units wouldn't need to change at all.
It is cheaper in terms of shared data paths in a design that did not promote the INT SIMD paths to 256 bits. It hasn't been discussed how this will be arranged when they are promoted in Haswell.
Are we assuming Intel is not providing additional 256-bit data paths for the FMAs? Intel would run out of data paths to borrow in this case.
Intel is adding a permute capability that probably contributed to a very significant portion of the bulk in BD, aside from the other changes.
When executing many 1024-bit instructions (that take four cycles), the reservation stations for these ports will quickly fill up. The reorder buffer also becomes full, so after a short while no new instructions for other (scalar) ports can be dispatched and the reservation stations for these other ports run dry. At that point, the dispatch and register rename logic as well as the reservation station and ALUs of the scalar port, can all take a multi-cycle nap. Only once a few reorder buffer entries become available again, it's worth waking them up again.
So clock gating with a delay of 4 cycles sounds absolutely fine to me!
CPU cores: 217.6 GFLOPS / 4 x 3.15 mm x 5.46 mm = 3.16 GFLOPS/mm²Now tell me about GFLOPS per transistor for the two architectures
IGP: 129.6 GFLOPS / 4.71 mm x 8.70 mm = 3.16 GFLOPS/mm²
That's for a Core i7-2600 as-is, at baseline frequencies. Haswell adds FMA to the mix...
Any idea how big part of the chip is taken by fixed-function stuff that isn't counted in the FLOPS like texture filtering that has to be emulated in software on CPU?
Why? As long as 1/4 of an SIMD reservation station consists of 1024-bit instructions, there's plenty of work and no need to top it up. That's a huge number of cycles during which much of the other power hungry logic can be clock gated.
You advocate processing in AVX like units extended to 1024 bits at least logically. That are 32 floats, exactly the logical vector size nvidia uses in its GPUs. AMD uses a vector length of 2048 bits logically. Both are using units processing 16 floats/clock (512bits) physically. I don't see much of a difference or a need at all to "assemble the data into a batch suited for processing by a specialized heterogeneous component", it will work just the same.
If you would first need to "assemble a batch" to be suited for processing by wide vector units, it is probably better to resorting to plain old and maybe even scalar SSE
Because you can't eat the cake and have it too? Hardware build for a special purpose will always be more efficient for this purpose. There is some barrier for extending a CPU (and the software support) in a way, that throughput oriented tasks get offloaded to a wide vector engine, but ultimately it is more efficient for sure.
