This might create a whole new niche, though.
Joking aside, it's not as simple as that as the POV actor's body you're in and environment also have the wrong scale. References of scale (objects that your brain know to be of a certain size) are mismatched, including any visible portion of your POV actor's body. It just ends up looking all sorts of wrong. It's akin to watching a movie in the wrong aspect ratio - you can still follow what's going on, but you can never completely ignore it.
The horizon mismatch (where the recording camera's pitch angle is not parallel with the ground) is especially bad because it feels like the world you're in is pitched forward at an extreme angle and can actually give me mild vertigo. What's worse is after 5-10 minutes of being in that environment my brain has attempted to adapt the inner ear's sense of 'down' to what my eyes have been seeing as 'down', and removing the headset and seeing the real world actually feels uncomfortable for a while.
All of this gets compounded by the fact that the download sizes are massive enough that you really can't stream these videos, so at best you're left with paying a subscription fee to a site and spending a not-insignificant amount of time downloading videos only to later find that they're badly formatted or that the 'content' itself is not to your liking. VR (head tracked, stereo, spherical/hemispherical video) porn is definitely novel, but it's anything but convenient right now.
Production and logistical issues aside though, in the fleeting moments where everything works I've actually gotten flashes of something akin to VR "presence" in a way that I wasn't expecting. Things like a simulated kiss or similar gentle skin-to-face contact results in a very brief physical sensation. My guess is it's some sort of associative sensory memory being triggered (your memory associating a proximity and type of contact with a very very subtle physical sensation). Presumably the same idea as seeing a virtual spider crawling on your virtual hand, or being tickled by a virtual feather and having it provoke a shiver/tickle on the implied point of contact.
Other things that I've noticed that will be very interesting when the display tech gets much better, is with high quality video sources and close proximity between the camera and actor (such that the video source and display resolution have adequate resolving power for very fine material detail). The texture of skin, subtle brdf, specularity, scattering, etc, combined with the stereo recording gives a reality to the material that is unmatched in real-time VR rendering. With real-time rendering right now you've got some amount of specular reflection from environment maps that doesn't play well with stereo depth perception because they're effectively positioned at infinity. Couple this with the inability to have multiple planes of focus (eye lens accommodation) means that materials with a non-trivial directional/specular reflectance component produce bad stereo information. This is something that stereo video recording does not exhibit because the lighting is physically correct (obviously), and the camera lens has a natural limit of focus that ensures distant environment reflections are not at the same plane of focus as the surface. The result of this is that certain materials at certain distances can look
damn close to reality.