In the anals of "how much does real time scanning cost", a few years ago at work my team had special permissions to disable virus scanning during asset builds. Doing a data build with the scanner on would add over 20 minutes to the build time, usually a 2hour process.
Now I agree with most of the points here if your not doing massively IO bound opperations it's not a big deal, but it can for somewhat extreme workloads be very significant. You can configure them to ignore certain file types and in the case above this was supposed to have been done, but we still saw the 20 minute hit.
Like everything else in security, it's convenience vs how secure you want to be. Loosing my machine at home is just an inconvenience, so I just run scans by hand regularly rather than running a continuous scan. And as I said earlier, I've been infected by exactly 1 virus ever, and that was a direct result of my actions.