That was due to technologicaol limitations, like tablet computers taking a couple of decades until Apple could bring everything together and do it right. The tech is here now and just having a head-tracked, high refresh, stereoscopic window into a world, whether the protagonist or a virtual cameraman, is going to be intrinsically compelling. As long as there are VR specific experiences alongside conventional games 'shoe-horned' into VR, it should be good.
Personally, I think U4 in VR will be more appealing to most core gamers and PS4 owners than Street Luge, even if the latter is an ideal VR experience. I expect Street Luge to be a great party game experience, but have no legs and little replayability beyond the initial novelty. Of course, it may because we don't quite know how consumers will respond to VR, but U4's appeal is obvious to gamers and being more immersed can only make it more appealing.