Bouncing Zabaglione Bros.
Legend
This argument, while common, is a complete fallacy. You always want to do the computation on the most efficient (perf/$, perf/W, etc) place possible. That may well be a GPU for physics nowadays, perhaps even a low-to-mid range one.
In theory, that's true, but in practice, that's not what's happening. Game devs are not doing their physics on GPUs, they are doing it on CPUs, and even that they are not doing particularly well. It seems they are having trouble multithreading their code on CPUs as it is, let alone doing it on GPUs.