Sandy Bridge already power gates the AVX units. Agner Fog discovered that the latency of the instructions depends on whether any AVX instructions have been executed recently.If you wanted AVX2 to be a viable option, personally I wouldn't widen it to 1024-bit-over-4-cycles, but rather include a ridiculously beefy 256-bit AVX2 with FMA pipeline (at least as fast as Haswell) as an option to the 22nm Silvermont Atom core. And I wouldn't just clock gate it; I'd power gate it like ARM optionally does for their NEON SIMD (obviously this adds some scheduling complications if you care about maximizing performance but I expect it to be manageable).
Sandy Bridge already power gates the AVX units. Agner Fog discovered that the latency of the instructions depends on whether any AVX instructions have been executed recently.
Clock gating the front-end is orthogonal to that. And the other benefit of executing 1024-bit instructions over 4 cycles is the latency hiding. That's something that would definitely benefit Atom too.
Rattner said:But in the future, MIC will be based on Atom cores, and then the integer gap between Xeon and MIC will start to close
Unfortunately unless they resurrect their mass market GPU strategy for these, the average person won't be able to get there hands on them.
AVX2 is obviously NOT power gated today. The warmup delay would be much longer (which poses all sorts of issues).
re Rattner saying future MIC will use Atom cores: here's hoping he means the 22nm OoOE Silvermont Atom cores and not the current ones. If so, that's certainly a good solution if they're willing to have both AVX2 and a MIC-only ISA, but it doesn't really solve anything in terms of mass market availability for developers.
AnandTech: "The upper 128-bits of the execution hardware and paths are power gated. Standard 128-bit SSE operations will not incur an additional power penalty as a result of Intel’s 256-bit expansion."I haven't seen anything stating they power gate the vector unit, much less the AVX units that are part of it.
What makes you think that?An implementation of power gates on that would be a physically difficult proposition.
If they're not power gated, why would a few more adders and multipliers require such a careful gradual ramp up? There are plenty of other non-AVX things you can do to create a local spike in power consumption, yet as far as I know these don't require microcoded sequences to ramp up the power delivery.The speculation I've seen is that the warmup period for AVX is a microcoded sequence of NOPs and FP instructions that gradually ramps up the FPU to full power so that the full-width vector units don't put too much of a load on the chip's power delivery network too fast for it to compensate.
The description of the power gates used indicates they are notably larger and have very long latency.What makes you think that?
It doubles the actively switching load of the vector unit. Clock gated units can draw much less power when they are quiescent. The transistors would still have static leakage to contend with, but this is far below peak draw.If they're not power gated, why would a few more adders and multipliers require such a careful gradual ramp up?
Wakeup from the deeper sleep states has latency associated with it, just on a wider scale than waking up the AVX unit. There are actions implied to bringing the pipeline back online that don't need to be exposed to the programmer.There are plenty of other non-AVX things you can do to create a local spike in power consumption, yet as far as I know these don't require microcoded sequences to ramp up the power delivery.
Haswell may not be a significant shift as far as client platforms are concerned.Anyhow, regardless of what they do right now it's of little relevance to where CPUs, MICs and GPUs are heading next.
Well with all due respect, David also said adding gather support to AVX would not be feasible, a few days before the AVX2 announcement...David Kanter said some time ago on RWT that AVX is definitely NOT power gated.
re Rattner saying future MIC will use Atom cores: here's hoping he means the 22nm OoOE Silvermont Atom cores and not the current ones. If so, that's certainly a good solution if they're willing to have both AVX2 and a MIC-only ISA, but it doesn't really solve anything in terms of mass market availability for developers.
Only the multiplier overlaps with the integer data path. There are five new 128-bit data paths which are only used by 256-bit AVX instructions, and therefore could potentially be power gated.Since the other half of the 256-bit data paths linked to it is the the data path for the integer units, which are still usable on demand, it didn't seem like a good idea.
Is that significant at all? ALUs only account for a fraction of a CPU's total power consumption, and we're looking at only half of a portion of them, per core. It just seems to me that there must be things other than 256-bit AVX execution that are more critical to power consumption, yet to my knowledge don't require any ramp up sequences.It doubles the actively switching load of the vector unit. Clock gated units can draw much less power when they are quiescent. The transistors would still have static leakage to contend with, but this is far below peak draw.
Well with all due respect, David also said adding gather support to AVX would not be feasible, a few days before the AVX2 announcement...
This forum is all about speculation, and you want to withhold judgement?Let's withhold judgement until we see the performance of gather.
Again, neither his sources nor his knowledge helped him foresee gather support for AVX2...Besides, I think David's sources and knowledge trumps Anand's anyway.
The data paths are the "stacks" the AVX blocks straddle in the diagram, not the individual units that are sometimes subsumed by the new blocks. If a blue unit is covered by a yellow block, I interpret that to mean that execution hardware is also shared.Only the multiplier overlaps with the integer data path. There are five new 128-bit data paths which are only used by 256-bit AVX instructions, and therefore could potentially be power gated.
My terminology was imprecise and misleading. The watts dissipated isn't the overriding factor. When I brought up power delivery, it had to do with layers responsible for providing a proper level voltage to the FPU, but I muddled my description pretty severely. When a significant amount of logic becomes active in a short time while the voltage levels are adjusted to a reduced electrical load, it can compromise the correct functioning of the hardware.Is that significant at all? ALUs only account for a fraction of a CPU's total power consumption, and we're looking at only half of a portion of them, per core. It just seems to me that there must be things other than 256-bit AVX execution that are more critical to power consumption, yet to my knowledge don't require any ramp up sequences.
That's about the entire FPU. Sure it makes no sense to power gate it, and it does make sense to clock gate it when running integer-only threads and perform a ramp-up sequence when activating it.The speculation on how this can apply to FP units was on realworldtech:
http://www.realworldtech.com/forums/index.cfm?action=detail&id=122180&threadid=122176&roomid=2
It was an entire FPU back when designs were simpler and FPUs were smaller in transistor count. The extra data paths and hardware for the upper half of AVX 256 have more logic than a two-pipe FPU from a 15 year old design.That's about the entire FPU. Sure it makes no sense to power gate it, and it does make sense to clock gate it when running integer-only threads and perform a ramp-up sequence when activating it.
It makes sense if the latency numbers make sense, and if it doesn't impact the active integer data paths the upper AVX units share.But that's not what I'm talking about. I'm talking about the extra ALUs that turn the 128-bit SSE units into 256-bit AVX units. AVX is still rarely used, so it makes sense to power gate this upper 128-bit part.
There were other measurements that put the warmup period around 70 cycles, which would translate to hundreds of instructions when there are several FP ports.Agner Fog: "I found that this doubled throughput is obtained only after a warm-up period of several hundred floating point operations. In the "cold" state, the throughput is only half this value, and the latencies are one or two clocks longer."
A 128-bit unit can execute 256-bit instructions if they are cracked by the issue hardware. This has been done for multiple designs.In other words, the 128-bit SSE execution units can execute 256-bit AVX instructions, split over two cycles, while the upper halves take their own sweet time to power up. This also suggests Intel won't have much difficulty executing 1024-bit instructions on 256-bit units.
Do you happen to know whether the Pentium 4 cracked them at decode or at issue?A 128-bit unit can execute 256-bit instructions if they are cracked by the issue hardware. This has been done for multiple designs.
Good points. In that case the 'sequencing' logic I talked about earlier already exists in Sandy Bridge, and AVX-1024 could build on that.The P4 was described as executing a 128-bit operation as two uops, which points to the decoder doing it.
That seems too funky for SB because of the uop cache and the decode restrictions for the simple decoders.
I did not see an indication from Agner's manual that AVX 256 experienced a drop in decode throughput, which a split would seem to do.
Additionally, the move from the slow to full mode would require that the decoders know to switch from a double op to a single uop, and the uop cache entry would need to be invalidated and replaced.
