Did you notice that this performance chart is logarithmic. Polaris fps is around 40% better than Fury X at maximum tessellation factor. That's a pretty good improvement already.
Polaris is also likely bottlenecked by other areas. It's after all way behind Fury X in bandwidth and compute performance. Could be also rasterizer bound as highly tessellated geometry increases quad overdraw factor. We need to wait for Vega to see the real impact of these geometry pipeline improvements. Vega is the first high end product after Polaris geometry pipeline improvements and Vega also has much improved rasterizer.
Ignoring clocks again?
I am afraid Vega might have even more changes muddling the comparison.