Pascal FP16/FP32/INT8 Performance

[CarstenS]

Very nice nick.

Seems like Nvidia is being really smart about how they segment their products:
– If you want high DP perf, must buy GP100
– GP100 is also the best card for training since it's the only desktop card that does 2x FP16. Assume FP16 is good for training but INT8 isn't.
– If you want really fast inference, must get P4/P40. Only "Tesla" variants have 4x INT8 perf. (not counting Titan X)
– If you buy a plain gaming card, eg GTX 1080, you don't get fast FP16, you don't get 4x IN8, and of course no usable DP64

Even on Geforce-Cards, 4×INT8 throughput seems to be enabled via Cuda. But if you're serious apart from university research projects on a budget, you probably won't get the level of support required for mission-critical stuff.