MS Kinect Technology Legacy lives on ... Introducing Project Kinect for Azure

[Shifty Geezer]

If you need to use the cloud to power it, that's a choice to make. Same as using GPS maps to know where you are and expose yourself to being tracked. Or having an always-on Alexa etc. to provide on-demand services, constantly recording audio and sending it to some unknown place.

Everything you do with the internet goes to some unknown place, and who-knows-who does who-knows-what with it.

 

[cheapchips]

I'm assuming that the secret Kinect illuminati will use my data to sinisterly push ads for dance lessons into all my sidebars.

 

[function]

How does cloud power integrate with Kinect for betterness? Machine learning to improve detection and tracking?

Machine learning certainly, but more to generate data and tools to expand the scope of what could be done.

Like API's and frameworks (e.g. Dot Net,) abstraction is key to harnessing something like Kinect for the vast majority of developers. The best way to quickly do this is to collect huge amounts of data, and move bursty and compute heavy loads to where you can handle them. "The cloud" allows huge amounts of data to be collected centrally, and for enhancements based on studying it to be deployed very quickly to clients.

Developers could test the latest Kinect enabled player analysis tools and switch to using that version when they were ready. Would be ideal in a MMO game in continual development, like Sea of Theives, WoW, and other stuff that I think could be done but hasn't yet.

Over time as efficiency increases you could probably bake static versions of some of the tools into games to run locally, but the starting point is huge data collection, massive scale analysis and rapid iteration of powerful tools for developers.

Cloud-driven games was pie in the sky from day 1. Potentially way too much latency for anything approaching realtime graphics; if someone is on a cellular data connection they could have 500ms ping quite easily. I've had 1500ms at times when I've used my phone tethered. It could maybe work reliably for a non-twitch, point-and-click adventure game, but what such game would need cloud-driven AI?

There are many parts in games that aren't latency sensitive. Seconds or longer is absolutely fine. As for Kinect based games needing cloud assist (as much about the datasets as the processing power) there are obviously no examples as it's never been available and so can't ever have been done. But that's like someone in the 1800s saying "but who would need a heavy lift rocked: there are no roads in space for horse and cart to ambulate along!"

Think more Siri or Cortana (but for motion and biometrics) than joypad replacement.

Jesus. If only I wasn't a useless drunk. I'd show you all!!!

 

[eastmen]

I think it mostly comes down to game design. The Kinect games needed space for what they were (bouncing around hitting virtual objects). Those space requirements are the same no matter what input system you use. A lot of Kinect's failure to take off was software design, I think, and how it was implemented. The tech has always had potential, but like so many novel systems, game designers don't know how to use it best and no-one wants to take on the economic burden of finding out what's best.

VR has the same issues, but there are more people invested in experimenting to find what works and what doesn't.

What about an RPG like Skyrim or Fallout 4 where the cloud will power all the other AI in the game that your not interacting with at the moment . When you build your town in fallout 4 it continues to be a living city even when y our not playing and it can grow or contract with out you , it can even fight off attacks . This would make a world more alive .

What about games like crackdown with huge destruction . That be updated every second or so and remove those physics caculations off the cpu , same with things like capes or clothing.


As for the Kinect cloud stuff , I am sure that's more for HoloLens and tracking. HoloLens 2 is supposed to have a next generation HPU and an ARM chipset. So anything that they can offload to the cloud would improve performance and battery life . Kinect 3 for console would just get the benfits of the new hardware I'd imagine.

 

[Shifty Geezer]

What about an RPG like Skyrim or Fallout 4 where the cloud will power all the other AI in the game that your not interacting with at the moment . When you build your town in fallout 4 it continues to be a living city even when y our not playing and it can grow or contract with out you , it can even fight off attacks . This would make a world more alive .

What about games like crackdown with huge destruction . That be updated every second or so and remove those physics caculations off the cpu , same with things like capes or clothing.

This is a discussion about Kinect. I'm talking about Kinect, and how the space limitations of the original Kinect games for which it has a reputation are more a limitation of the software design choices than the hardware.

 

[function]

What about an RPG like Skyrim or Fallout 4 where the cloud will power all the other AI in the game that your not interacting with at the moment . When you build your town in fallout 4 it continues to be a living city even when y our not playing and it can grow or contract with out you , it can even fight off attacks . This would make a world more alive .

Not sure you could ever make the business case for this, but yep, it's an interesting idea isn't it?

I think a game's AIs being able to learn how to read a particular gamer's verbal and none verbal cue to make the game feel more natural and more immersive absolutely has value and it *will* happen.

But unfortunately, probably on mobile first, with some psycho-battering gambling based e-xtortion based mechanic. FFS.

 

[eastmen]

This is a discussion about Kinect. I'm talking about Kinect, and how the space limitations of the original Kinect games for which it has a reputation are more a limitation of the software design choices than the hardware.

Isn't this true for all new inputs ? How many games does Nintendo have that use hd rumble in a good way ? or the sensor on the remote. Or what about ps camera ?

Not sure you could ever make the business case for this, but yep, it's an interesting idea isn't it?

I think a game's AIs being able to learn how to read a particular gamer's verbal and none verbal cue to make the game feel more natural and more immersive absolutely has value and it *will* happen.

But unfortunately, probably on mobile first, with some psycho-battering gambling based e-xtortion based mechanic. FFS.

Depending on the game I would pay for cloud access. Imagine day z 3 with cloud access . I would easily pay a few bucks a month for that.

Face tracking for emersion plus head tracking for control would be reason enough for kinect

 

[Shifty Geezer]

Isn't this true for all new inputs ?

Yes. I already said, "The tech has always had potential, but like so many novel systems, game designers don't know how to use it best and no-one wants to take on the economic burden of finding out what's best."

 

[Deleted member 13524]

Same as using GPS maps to know where you are and expose yourself to being tracked. Or having an always-on Alexa etc. to provide on-demand services, constantly recording audio and sending it to some unknown place.

You can use GPS without any mobile data activated, and I dread at the thought of keeping something like Alexa in my home. There's enough tech in my house, but I enjoy the thought that when I'm talking in private, I am really talking in private.


Regardless, as you mentioned before this thread is about kinect and not about other techs that serve completely different purposes.

Perhaps the better question is why does the Kinect 3 need to send all the sensor acquisition data to the cloud? Didn't the motion and audio capture work fine with the XBone? Or did the 2013 Kinect use the cloud?

Isn't this Microsoft just trying to use and/or sell user metadata, like they seemed to do 5 years ago with the originally planned always-on Kinect?

 

[BRiT]

Kinect for Xbox 360 and Kinect for Xbox One worked completely isolated and did all the processing locally.

 

[Silent_Buddha]

Just like voice recognition, quality and accuracy for Kinect would improve significantly if you have access to a much larger database of information.

That's without even going into the potential of leveraging a neural network, although I'm not sure how beneficial that would be in real time over an internet connection.

However, that neural network combined with information from many connected devices would in turn improve the functionality, quality, and accuracy of Kinect operation in the future as the attached database is updated and improved over time by analysis of incoming data.

That in turn can only be of benefit for potential non-connected Kinect derived devices as the more limited database of a non-connected device would be more refined.

Basically it would be a data scientist's dream to have that much information to throw at an AI/neural net.

Regards,
SB

 

[Shifty Geezer]

You can use GPS without any mobile data activated, and I dread at the thought of keeping something like Alexa in my home. There's enough tech in my house, but I enjoy the thought that when I'm talking in private, I am really talking in private.

If you have any webcam and/or microphone enabled device, you are as much at risk as using Kinect 4 in theory.

Perhaps the better question is why does the Kinect 3 need to send all the sensor acquisition data to the cloud? Didn't the motion and audio capture work fine with the XBone?

Not particularly well. Clearly a local dataset comprised of thousands of hours testing with dozens of people is a miniscule fraction of what can be obtained from millions of users over many millions of hours of data.

Isn't this Microsoft just trying to use and/or sell user metadata, like they seemed to do 5 years ago with the originally planned always-on Kinect?

Rather than speculating on worst-case conspiracy theories, it's probably best to wait until there's a decent description of what the cloud integration is and how its used. May as well post in any 'new mobile phone camera sensor' article the same wild theories that it only exists to spy on people; Sony's ubiquitous imaging sensors have probably provided them with insane amounts of secretly extracted personal information...

 

[Deleted member 13524]

Rather than speculating on worst-case conspiracy theories (...)

Companies using and selling anonymous metadata is very, very far from "worst-case" and hardly a conspiracy theory in 2018.

Funny thing is, without even realizing, you suggested that Microsoft would do exactly that in the second paragraph of your post.

They're also not "secretly extracted" since most EULAS / terms of usage cover that kind of usage.

 

[BRiT]

Please, please, please keep Consumer Privacy Concerns to a different thread.

Kthxbi!

 

[Shifty Geezer]

Funny thing is, without even realizing, you suggested that Microsoft would do exactly that in the second paragraph of your post.

Yes, to improve Kinect's functionality. All these networked devices connect to the network to provide a service or improve functionality.

 

[iroboto]

IIRC there was a novel idea to allow for Kinect to create creature animations in project spark?

I think for prototyping or limited use cases for content creation is still pretty awesome as a concept.

 

[Shifty Geezer]

The irony is Dreams is the perfect candidate for motion input for virtual puppetry. I see Sony have announced a TOF sensor, so perhaps Kinect will have paved the way for Sony to follow suit with PS5's camera? With the things appearing in phones too, they must gain traction.

I wonder if a common learning platform would make sense? Seems it's something all companies/users would benefit from, pooling their data to create the most robust library possible, but they may want to try and leverage their own data as 'best in class' for a USP on an otherwise common technology.

Cloud data also makes sense to me for a headset with limited local storage. You could use appropriate datasets fetched from the Cloud for your specific environment. Inside a living room would want different data to wandering about town, for example.

 

[BRiT]

Latest revision of Kinect technology is a lot lighter (half the weight at 440 g) and contains 7-microphone array up from the previous 4, the RGB camera can do 3840x2160 resolution, and the depth sensor is the same as on HoloLens 2. Azure Kinect DK consumes up to 5.9W; specific power consumption is use-case dependent.

https://www.microsoft.com/en-us/p/azure-kinect-dk/8pp5vxmd9nhq

Azure Kinect DK is a developer kit and PC peripheral that contains a best-in-class 1MP depth camera, 360˚ microphone array, 12MP RGB camera, and orientation sensor for building advanced computer vision and speech models. It measures just under 5 inches long and 1.5 inches thin. Learn more at http://azure.com/Kinect/

 

[Malo]

It measures just under 5 inches long and 1.5 inches thin.

I'm sorry.