Deep buffers, prefetching, caching, speculative data loading and really high clock rates will help you overcome much of the latency problems. But to overcome them you'll need bandwidth, because you move data before you need it, not after, ideally it arrives right when you need it.