You're only aware of the benefits and performance hits of
current raytracing solutions, which are first-generation based on existing offline-rendering concepts shoe-horned into games not designed for them. We've already seen new developments such as screen-space secondary illumination.
And that's a
mod on top of an engine, rather than an engine designed from the ground up to support intrinsic ray-traced data. The people working in realtime graphics are utterly brilliant, and they will find utterly brilliant ways to leverage this new information. Compare and contrast first gen titles on any console with EOL titles, where new software concepts on the same hardware results in huge improvements, and apply that to your expectations of ray-tracing acceleration.