Microsoft have purchased many augmented reality patents for 150 million dollars

[Shifty Geezer]

MS has the best consumer skeleton tracking I've ever seen and coupled with a vr head set will blow away the tracking that oculus or sony has.

Like I say, that's one very good input, but a limited one that doesn't automatically put MS in the strongest position. In a tennis game, for example, Kinect versus Move with a headset, Move would provide the better experience in a home environment. It's lower latency and higher accuracy and can tell wrist rotation. In a big hall where you can run around, Kinect could be better as you could actually run around and the game would draw your boundaries so you don't run into stuff.

Both system have advantages and disadvantages, but they both only pertain to the input mechanics. VR starts with a headset (VR at a desk with a joystick is still VR), ergo whoever has the best/most close to release prototype has to be the company ahead of the game. And most importantly, all MS's noise and patents and such these days are about AR, not VR, so a headset from them would be a surprise. To suggest they are in the lead because if they have a headset, they'll have headset with Kinect, is similar to saying Usain Bolt is in the lead in the Commonwealth Games 100m even if he hasn't entered because if he did, he'd be the fastest. MS have yet to enter the race as far as we're informed.

 

[Deleted member 11852]

Like I say, that's one very good input, but a limited one that doesn't automatically put MS in the strongest position. In a tennis game, for example, Kinect versus Move with a headset, Move would provide the better experience in a home environment. It's lower latency and higher accuracy and can tell wrist rotation.

How does Move track wrist rotation if it's just tracking a glowing ball? I.e, if my arm is outstretched as though holding a tennis racquet and I twist my wrist 70-80 degrees clockwise (to move the position of the racket head from in front of my face to being to the left of me) how does it know I've twist my wrist and not just moved my my whole arm?

Is it doing some body tracking as well, so has a point of reference?

 

[Shifty Geezer]

How does Move track wrist rotation if it's just tracking a glowing ball?

It's not just a glowing ball. It has both a 3 axis gyro and a 3 axis accelerometer.

Is it doing some body tracking as well, so has a point of reference?

Body tracking won't be able to track that unless you have incredible resolution and algorithms that can read finger positions and widths of the forearm and derive wrist and forearm rotations. It's a problem difficult to the degree of regarding it as impossible within the life of the consoles and probably a ways after that. A degree rotation in the hand would be imperceptible to the most awesome of depth cameras but makes a point-changing difference to the virtual ball's response to the virtual racquet. Of course, Move is far from perfect. There's no haptic feedback which would make any item-swinging confusing. Imagine VR Elder Scrolls where you swing a sword. In game, it'll collide with the enemy and stop, but in real life your arm will pass through the enemy. Ideal solutions will avoid contexts that break under the limitations of the input mechanisms. eg. Make the player a ghost, or provide them with a lightsabre that passes through all enemies and meets no physical resistance. This is where arcades could make a small comeback by providing an experience the home console can't match with relevant input stuff (I'm suddenly remembering a chair and accessories for haptic gaming back in the early 90s, not sure it ever launched).

In some cases, Kinect will be very good, like controlling an on screen hand that matches the player. Kinect can have you reach for the door knob and grab it and pull/turn. Move will require a trigger instead, or alternative input (how's about a squishy glowing ball that is squeezed as a button?) which are less realistic than hand tracking. For other games like FPS, the Poster Child for VR gaming, Move with its trigger will probably be the better experience for most gamers (it'll be subjective). Also we shouldn't forget that PS4 is equipped with stereo cameras which should afford a degree of body tracking in addition to Move.

 

[Deleted member 11852]

It's not just a glowing ball. It has both a 3 axis gyro and a 3 axis accelerometer.

D'oh.

Body tracking won't be able to track that unless you have incredible resolution and algorithms that can read finger positions and widths of the forearm and derive wrist and forearm rotations.

Really just the forearm, if my arm is not moving but the ball is relative to the end of the arm (where my hand is), it's my wrist turning. Or I'm a mutant

I don't know what Kinect is doing vis-à-vis skeletal tracking but based on some of the capture glitches that have been seen with Project Spark, it doesn't seem like a standard feature of capture is the software knowing what is a reasonable movement of the hand and wrists relative to the ulna/radius bones, those relative to the elbow joint, that relative to the humerus and so on and so on.

It's a problem difficult to the degree of regarding it as impossible within the life of the consoles and probably a ways after that. A degree rotation in the hand would be imperceptible to the most awesome of depth cameras but makes a point-changing difference to the virtual ball's response to the virtual racquet.

I'm quite confused about Kinect's capabilities. Microsoft have shown it capable of detecting the heartbeat and even the pressure of muscle use when a person is moving. In some regards is seems phenomenally clever in it's capture, in other things not so much. Perhaps some of this clever capture or analysis is done by the Xbox itself which may make really accurate capture a trade off to giving up some system resources.

 

[Shifty Geezer]

Really just the forearm, if my arm is not moving but the ball is relative to the end of the arm (where my hand is), it's my wrist turning. Or I'm a mutant

The wrist has 3 degrees of freedom as I count them. When holding a racquet, to determine a twist of the forearm you'd either have to observe the relative width of the wrist or look at finger positions. The finger positions would be far more reliable. Course, that breaks on a back-hand. Quite frankly it'd be an insane task to try and gauge subtle hand orientations with a camera!

I'm quite confused about Kinect's capabilities...

That's for another discussion.

 

[eastmen]

Shift , if your already using a prop to replicate the sword instead of just your hand , you can include vibration feed back as a way to signal you've hit something and should stop.

Anyway something like the Omni using the IR camera in the Kinect should allow you to track the full body much faster than the 1080p camera in the move.

 

[Davros]

Are you suggesting shifty owns something that is long and vibrates ?

 

[Shifty Geezer]

Anyway something like the Omni using the IR camera in the Kinect should allow you to track the full body much faster than the 1080p camera in the move.

Probably. Doesn't change the fact that full body tracking is only a part of VR though, and that the most important part, the part that starts VR as opposed to just being an alternative input for conventional games, and the reason this news was posted here, that being the VR headset, is still missing from MS's plans as we know them. Without that, everything else is pretty much immaterial. MS could reveal 1:1 realtime perfect motion capture via Kinect with detail down to fingers and wrist rotations and everything, which enable perfect 1:1 tennis etc. match up with on screen avatars, and they could include a haptic wand controller with pneumatic resistance for complete authenticity - without the headset, it'll just be ordinary games and not VR.

VR by definition shifts the experience from outside-looking-in to inside, and that requires either direct brain interfacing, or some form of stereoscopic optical (and ideally audio) input to the user's senses, which presently is confined to screen(s) in front of the eyes and speakers on the ears.

 

[Deleted member 13524]

The wrist is responsible for 2 degrees of freedom, radial/ulnar deviation and flexion/extension. Supination and pronation is done through the rotation between the two major bones from the forearm.


Regarding the subject at hand, VR and AR may be two very different things now but I think they will fuse eventually. AR glasses will eventually be also VR glasses and the only difference between AR and VR modes will be to activate an electronically polarized layer that blocks/unblocks the light from the exterior.

 

[MfA]

A LCD shutter would have max 50% transparency, an electrowetting shutter would be better.

 

[Deleted member 13524]

A LCD shutter would have max 50% transparency, an electrowetting shutter would be better.

I'm not literate on those techs, so I believe you
What I meant was that the only difference between AR and VR should come to a point where the system passes light through the glasses or not, and that should be controlled electronically.

 

[Shifty Geezer]

I'm not literate on those techs, so I believe you
What I meant was that the only difference between AR and VR should come to a point where the system passes light through the glasses or not, and that should be controlled electronically.

The requirements for see-through HUDs are different to those for 3D stereoscopic video.
The tech required for projecting graphics on glasses (eyes) can (and should) be very different to the tech required to put high quality, full colour opaque video in front of the eyes. Every VR headset AFAIK has a big ass visor in front of the use with one or two screens behind and some form of optics to correct distance viewing, which provides a 3D image where the user can focus to different depths. AR doesn't need or implement that, and sticking an opaque background behind a VR headset is not enough to enable it to provide full stereoscopy. In the distant future we'll have some tech that can serve both purposes, but for now and this gen of consoles, we're stuck with two discrete product types.

 

[MfA]

The acceptable compromises are different, but you can always turn AR glasses into VR glasses simply by shuttering the outside world (I don't consider HUD devices like Google Glass AR glasses). AR glasses for the moment just have really crappy FOV.

 

[Deleted member 13524]

Shifty, I obviously meant the same distant future that you mentioned.
Though I wonder how distant, actually. More than 5 years? Probably. More than 15? Maybe not..

Which makes me wonder: how long until desktop LCD monitors for productivity and gaming will all become obsolete?

 

[RudeCurve]

MS has the best consumer skeleton tracking I've ever seen and coupled with a vr head set will blow away the tracking that oculus or sony has.

Agreed...and the "goggles" doesn't even need to cost that much since it's just a HMD. Head tracking built into the HMD doesn't cost that much either if they want to add even more accuracy. There's already FPV goggles on the market that does this and only cost around $200. With clever design you could even add an external camera to the goggles so it could work for AR as well.

 

[Shifty Geezer]

There's already FPV goggles on the market that does this and only cost around $200.

It would not be hard for any company to bring a comparable headset to market, but let's not be blasé with our comparisons. When suggesting an existing product is suitable, you need to compare like for like, as products from other industries have other priorities. You'd need to find a product with comparable FOV, quality, battery life, and probably whatever other features (warped images with corrective optics to provide wider FOV and better quality in the centre of the screen; high quality, low latency MEMS).

There's a reason why the leader for the current wave of VR has been OVR and why they still haven't released a retail product yet, and why we're still waiting for Sony to release their product. A couple of low grade video screens on a plastic headset isn't what's launching VR, no matter how cheap and easy that solution might be.

eg. A lot of these FPV glasses I've just looked up at well over $200 feature dual 640x480 screens, so 480 vertical pixels, and 35 degree FOV. That's not VR quality. MS launching with something like this for $200 with chromatic aberration from cheap plastic lenses and narrow FOV will get laughed off the gaming forums.

 

[HTupolev]

eg. A lot of these FPV glasses I've just looked up at well over $200 feature dual 640x480 screens, so 480 vertical pixels, and 35 degree FOV. That's not VR quality. MS launching with something like this for $200 with chromatic aberration from cheap plastic lenses and narrow FOV will get laughed off the gaming forums.

I'd love to see the NeoGAF thread title making fun of an XB1 standard-def VR headset, though.

 

[Cyan]

The acceptable compromises are different, but you can always turn AR glasses into VR glasses simply by shuttering the outside world (I don't consider HUD devices like Google Glass AR glasses). AR glasses for the moment just have really crappy FOV.

Like some fellow forumers have said, maybe AR and VR will merge at some point. Think of Morpheus or Oculus with AR Contact lenses or Retinal Display or Spatial augmented reality....

zed, thanks for suggesting me to read the Wikipedia article.

 

[RudeCurve]

It would not be hard for any company to bring a comparable headset to market, but let's not be blasé with our comparisons. When suggesting an existing product is suitable, you need to compare like for like, as products from other industries have other priorities. You'd need to find a product with comparable FOV, quality, battery life, and probably whatever other features (warped images with corrective optics to provide wider FOV and better quality in the centre of the screen; high quality, low latency MEMS).

There's a reason why the leader for the current wave of VR has been OVR and why they still haven't released a retail product yet, and why we're still waiting for Sony to release their product. A couple of low grade video screens on a plastic headset isn't what's launching VR, no matter how cheap and easy that solution might be.

eg. A lot of these FPV glasses I've just looked up at well over $200 feature dual 640x480 screens, so 480 vertical pixels, and 35 degree FOV. That's not VR quality. MS launching with something like this for $200 with chromatic aberration from cheap plastic lenses and narrow FOV will get laughed off the gaming forums.

Resolution is trivial the only reason why FPV goggles haven't made the leap to 1080p is due to transmitting that HD video signal wirelessly long distances at acceptable latency levels.

The latest FPV goggles have SVGA resolution, glass optics and a bunch of stuff built in that adds to the total cost. Without these extra features the cost would be a lot cheaper for example a "dumb HMD" for VR. MS could very well be ahead of SONY given their superior Kinect2 hardware design. I don't think it is inconceivable that MS could have dual purpose AR/VR goggles in the prototype stages already. In fact by the time SONY actually releases a production VR headset at a reasonable price everyone would be at a similar stage. Display hardware is not the problem.

 

[Shifty Geezer]

MS could very well be ahead of SONY given their superior Kinect2 hardware design. I don't think it is inconceivable that MS could have dual purpose AR/VR goggles in the prototype stages already.

Indeed it's not, but until there's evidence of this, MS are definitely to be considered behind everyone else. We can't sanely say, "MS are ahead of everyone else because of what they might have secretly hidden away in their R&D department," not least because anyone else may have something secretly hidden away in their R&D departments!