Playstation VR Tracking

[gmoran]

I've become interested in PSVR tracking again due to the following video, and I'm hoping there are some informed or knowledgeable folks out there who can provide further insight:

Although I understood—hazy lay-man style—that PSVR was using sensor fusion to achieve tracking, I didn't appreciate what this actually meant until watching this video. Specifically that PSVR uses a standard technique: a Kalman filter in which the IMUs provide primary low latency tracking data and the external tracking is used to prevent these from drifting. If you have an Aim controller its really easy to see this in action: start a game keeping the Aim controller still, when the game loads and the Aim is represented in game the orientation is completely incorrect, now just move it a bit and within a second or two the orientation is worked out correctly.

Now given how PSVR tracking actually works its no longer so understandable why there are significant tracking issues.

There are four understood tracking issues with PSVR, these are minor and easy to understand:

• Z-axis instability: the large source lights optical tracking system employed by PSVR is less accurate at estimating depth than point source arrays (e.g. Rift Constellation). This manifests as jitter of tracked objects.
• Narrow camera FOV: it is easier with PSVR for tracked objects, particularly controllers, to no longer be viewed by the camera. This manifests as a a tracked object position becoming unreliable outside of the camera FOV.
• Occlusion: if the tracked object is occluded to some degree then it will no longer be reliably tracked by the camera. Commonly experienced as occlusion wobble.
• Interference: as the PSVR tracking system is optical it can be interfered by reflections and direct lighting. Manifesting as unreliable tracking.

With some care and attention these can be minimised and/or mitigated—though some jitter will always be present. On this basis PSVR represents a system with good positional tracking, though lower quality than Constellation or Lighthouse.

However PSVR also manifests other positional issues and these are not—for me at least—understandable given what we think we know abour PSVR tracking:

• VR rotational drift: this is where the orientation of the player or tracked object rotates with respect to the forward position. The forward position is signalled to the application by holding down the Option button. Once this is known by the application its not easy to understand why the tracked object's orientation should drift with respect to it. If the system can determine correct orientation from a controller like Aim solely by sensor fusion and the Kalman filter then how can it sustain incorrect orientation at all? it seems more likely that the system lacks precision over time in reconciling actual orientation with the forward position?
• Positional parallax errors: in game you lean forward and near-by objects undergo lateral movement. I had always assumed this was due to z-axis imprecision. However I'm now observing that this seems unrelated to tracking fidelity and tracking environment. Also some games, e.g. Dick Wilde in game lobby, simply exhibit this as a matter of course. In these cases it seems like the application's 3D world is not being rendered with sufficient precision in response to player movement. In other games it appears that these problems manifest over time?

What strikes me about these issues is that they appear to be unique to PSVR out of the three major VR headsets?

Does anyone have any further information, observations, knowledge, opinions?

Thanks