Some dedicated hardware is still there, for some years there's been that craze for h264 decoders in graphics card, then cell phones and we're almost done with it.. Because most every hardware integrates it by now. Not still sorted out as linux is seeing nascent support for them in open source drivers for AMD/ATI and nvidia cards/chipset/APUs.
And then we'll need h265 and/or VP9 decoding (unless everyone sticks to h264)
That's an example, not the whole grand future of things but look at what we find inside a phone SoC or even a PC CPU : CPUs, GPU, image processor (tons of phone/embedded chips but also Intel QuickSync), audio codec and other audio-related DSP, video decoders, video encoders, hardware blocks dedicated to software radio, crypto accelerators, TCP/IP off-loading (in e.g. Gb and 10Gb ethernet interfaces).
It's even increasing : Moore's law give more transistor benefits than power benefits so in critically power/battery limited chips you have a lot of "dead silicon" i.e. units that are turned on occasionally, and then power gated for several milliseconds or more (down to turning off CPU cores, or an entire CPU in the 4+1 or 4+4 arrangements)
The HSA foundation basically exists to promote stuff working better together.
It's not necessarily incompatible with stuff also getting more generic, i.e. graphics cards gained GPGPU abilities at the same time their video decoding abilities were increasing (and soon shaders were doing scaler/filter things)
Regarding external FPU example : we ended with a FPU inside the CPU, rather than the CPU becoming exceptionnally strong at emulating it with integer code. [/edit: well you're making that same point, Nick]
So now we have the GPU getting inside the CPU (which even brings the h264 decoder and stuff in) but not quite getting games run on software renderer yet.
Maybe they will fusion a bit more, so we end up with those 512bit wide CPUs doing graphics duties. But who knows, there may be texture filtering units built right in the CPU pipeline, S3TC and friends decompressors or whatever critical stuff is needed.