From what I see in reviews and DF's analyses, there's a lot of settings where we don't even notice the difference between medium and high, let alone high vs. ultra.
I was totally one of those guys who sought to get Ultra maxxed out everywhere almost all my life, but in recent years I came to understand that Ultra settings are mostly placebo, at the cost of immense drops in performance.
We also learn about that in the PC-equivalent settings that devs select for the console versions.
There are some devs whose job is to push the best possible visuals on consoles, and in pretty powerful beasts like the PS5 and the Series X we're learning that they're actually selecting "medium" settings on a lot of parameters because the IQ difference is negligible and they'd rather provide higher dynamic resolution and/or frame stability.