Bulldozers are faster in reverse

I'm thinking more like 8 cores, 16 threads, 16 SIMD clusters with 2 FMA units each. That would still be barely bigger than 100 mm² at 14 nm, but could pump out 2 TFLOPS.

Leaving some of it dark isn't an issue at all. It's a necessity due to the power wall and the bandwidth wall. The important thing is to have the right execution logic for any type of workload, right where the data is.

AMD tried that with Bulldozer, and look where it got them. Even for Intel, the highest Linpack scores are obtained when turning off Hyper-Threading. Granted, that's a synthetic benchmark and Hyper-Threading typically does help, but it shows that it comes with overhead that needs to be 'overcome' before it starts to contribute anything. That's why I'm proposing to execute AVX-1024 instructions on 512-bit units, instead. It helps hide latency, while doubling the scheduling window, and lowering the front-end activity.

Unified cores then, right?

 

Why settle for diminishing returns already? That can always work as a last resort. Intel has been able to keep the L3 cache for four cores at 8 MB for three process nodes now, using the increasing transistor budget for better purposes. I'd rather know that each core is concentrating on one task, achieving maximum performance/Watt, than ensure full utilization of a few but leaving performance on the table.

There are three basic kinds of workloads to deal with: high ILP, high TLP, and high DLP. For high ILP, you want four scalar execution ports like Haswell. For high TLP, you want to share those between two threads. And for high DLP, you just want wide SIMD units, and lots of them, at a lower frequency and hiding latency while maximizing data locality with long-running instructions.

The architecture I'm proposing here is designed for each of these, and any mix of them. Basically, CPUs like Haswell are already very good at ILP and TLP, but they need GPU-like SIMD units, without lowering the data locality by running lots of threads, and other overhead associated with that.

I have no first-hand experience with Bulldozer, so I'm not doubting your findings at all. But improving the caches and lowering the latencies would still not make it a good step towards unified computing. To match my proposal, it would need 32 scalar cores. There's no use for that in the consumer market, and it would waste a lot of space (and I'm not talking about low utilization, I'm talking about a complete waste - even using it for more cache would have been better).

Of course the root of the problem is that AMD doesn't want unified computing at all. It wants small scalar cores that share an FPU for legacy purposes, and a big GPU to handle all throughput computing needs. That may sound good on paper, but it's fraught with heterogeneous computing issues. They're hoping for developers to miraculously deal with that, while Intel is pampering developers with a better ROI proposal.

Like I said, I mentioned Linpack only to illustrate that Hyper-Threading has an overhead. I'm not arguing that it increases utilization in mixed-load environments, but I do argue that it doesn't offer the best performance/Watt, precisely due to the inherent overhead.

Then why do I propose keeping Hyper-Threading for the scalar portion of the core? Because there's a tipping point where low utilization becomes a waste and using four scalar execution ports by two threads during high TLP workloads is more power efficient than one thread using on average only a couple of them. They do matter for increasing IPC in single-threaded workloads though due to Amdahl's law.

So it's all about finding the right balance for each type of workload. I don't think SMT is optimal for the SIMD units. They suffer the most from cache pressure when the thread count is increased. AVX-1024 offers the necessary latency hiding qualities to increase utilization (the good kind that improves power efficiency) while lowering front-end and scheduling overhead.

I'm open to alternatives, but I really don't think Bulldozer sets a good example.

I think it will happen regardless of Intel's desire for it. That, plus their process advantage, will just make it happen sooner than the sheer necessity dictates. I have continuously underestimated how fast they would converge things, and I think that's saying something. If Skylake features 512-bit SIMD units, then we're looking at a 32-fold increase in computing power between a dual-core Westmere and an 8-core Skylake, in five years' time. That would probably put getting rid of the integrated GPU on the roadmap next.

 

I think it can be done in the execution units instead of requiring a separate math box. Sines and cosines can be closely approximated with mainly just a handful of FMA operations. SSE/AVX also already has support for pipelined reciprocal and reciprocal square root approximation (although it could use some improvement). Next, gather instructions can be used for table look-ups for piece-wise approximations. The starting point for the exponential and logarithm function is to extract/insert the IEEE-754 exponent field, for which Xeon Phi has some interesting instructions. Extending the 'Bit Manipulation Instructions' to AVX would also help with that.

 

I'm not sure if I'm interpreting you correctly this time, but there are no different cores involved. Instead of like today's CPU cores with Hyper-Threading where both the scalar and vector execution units are shared, my proposal is to have dedicated vector execution units for each thread. That's really the gist of it. The front-end and back-end basically stay the same.

Just look at Bulldozer. It has two dedicated scalar clusters, one shared vector cluster, and one shared front-end and back-end. I just want the reverse for the scalar and vector clusters.

To keep that many vector units occupied with just a 4 instruction wide front-end, there would be AVX-1024 instructions that split into two 512-bit operations on issue, executed sequentially. Running the SIMD clusters at half frequency would save power and further lessen the burden on the front-end.