I do not see why you think CPUs have a lot of room for growth. Agreed, GPU's are severly reticle size and power limited, but I dont see why you think CPUs have a lot of headroom for growth. CPUs must serve the ultra low cost market too, which is being eaten up by IGPs for GPU.
They have headroom in computational density. They can slowly evolve all the way to an architecture much like Larrabee, while keeping single-threaded performance steady. Current CPU architectures still cary a lot of weight from the MHz-race days, but they can gradually get rid of that (keeping the good bits of course).
This evolution has already started. They're no longer spending transistors on IPC unless it offers better than linear scaling. For instance Nehalem has less transistors than Yorkfield, primarily by reducing cache size per core, and spent a little of it on Hyper-Threading which yields 30% higher effective IPC. We'll see many more tradeoffs like this in the future.
Yeah, we have octa cores at 45 nm, but at what price? My claim, was qualified with ".. at the same price point". I don't think we had quad cores at ~$150 in 2006. The mainstreaming of quad cores has taken longer than the shrink time of 2 years.
According to pricewatch a mid-range 3.0 GHz Core 2 Duo still costs 160 $, today. So how you come to expect mainstream quad-cores to cost 150 by 2006 is a bit beyond me. You can however today buy a 2.5 GHz Core 2 Quad for 155 $, which in 2005 would have only gotten you a single-core.
How do you figure AVX and FMA can be done without a non-trivial increase in trannies?
Can I have some explanation? Unless you are speaking of a 128 bit wide AVX unit, just like we had a 64 bit wide SSE unit prior to conroe. In which case, there is absolutely no perf increase at all. And IIRC, AVX wont have FMA.
Looking at a
rough diagram of one of Nehalem's cores, and the
entire die, I estimate that at most 5% of the area is taken up by the SSE units. So it would only take another 5% max to make them 256-bit wide. And far less than that to add FMA. Worst case, we're looking at a 3.6 fold increase in computational density for MAD operations.
And they will remain lousy simply because they need to care for single threaded IPC, which is still what matters (may be not the most). GPU's dont even look at that market and hence can run waaay faster at workloads designed for them.
They no longer have to increase single-threaded performance at the same pace, or at all. For a single thread Core i7 is practically identical to Core 2, except that it has an 8 MB L3 cache versus a 6 MB L2 cache which sometimes yields a small benefit, sometimes not. People do accept that overall performance can no longer be improved significantly without some developer effort. It's just an ongoing transition. Soon the applications for which it matters will all be multi-threaded (else the competition will show how it's done).
GPU's do have a trick up their sleeve though (not sure if they'll follow the trail because of various reasons), increase clock speed. rv770 runs at 750Mhz, GT200 runs at ~1.5G. They can still scale along that axis.
Uhm, no. They'll hit that concrete wall again. Also, high clock speed doesn't come without an additional cost to increase the number of pipeline stages and keep them fed. Just compare die sizes and ALU counts for RV770 and GT200. Long story short: GPUs are bound by exactly the same physical laws as CPUs. Clock frequencies still go up, but only at the same pace as process improvements allow.
And last but not the least, don't underestimate the burden of backward compatibility. For GPU's, all code gen is dynamic/ at runtime. So they can aggressively drop old and useless crap.
True, but you shouldn't overestimate it either. Intel's reckless number of SSE extensions hasn't exactly costed them their position as dominant CPU manufacturer.
It's also almost ironic how some of the older instructions can be implemented using instructions that supersede them. Take for instance pshufb. It can be used to replace any of the other shuffle or unpack instructions. So once you have the logic to support pshufb, implementing the others becomes a simple translation from the instruction format to a vector representing the byte reordering. So the RISC core itself doesn't suffer from the CISC ISA.
Also, even though GPUs can get rid of instructions that have become useless, overall the number of instructions is still increasing. And I bet a lot of instructions already have to stay because driver developers won't accept having to rewrite everything every generation, and to offer backward compatibility with things like CUDA and other things that are close to the metal. So I really wouldn't say they can "aggressively"drop things. Lastly, because there's a need to evolve toward more & smaller independent cores the GPU will also suffer form an increase in control logic.
