Voice recognition / Voice Control and Kinect

[Rotmm]

I suppose this is a question for bkillian really, but maybe others have ideas/opinions.

There's been a great deal of focus these last few months on the Microphone element of the Kinect unit. When the first wave of titles was released there were many commentators saw the lack of voice recognition as a negative.

Now that has been addressed to a certain extent (Dashboard, Apps, games like Dance Central 2, Forza 4 and the upcoming Mass Effect 2) the talk is about why is that functionality only available to Kinect owners, especially galling for Xbox360 owners (with regards to games) as every 360 comes with a headset in the box.

So if I'm playing Halo Anniversary (I don't have it, but a supposition) why can't I plug my headset in and say "Throw Grenade"? Or if I'm playing Mass Effect 3 on my PS3 why can't I use a bluetooth headset to say "Liara, take it all the way down"?

I must admit to being surprised by the comprehensive nature of the voice control in the 360 dash/apps as I know that the devs are still constrained by that 32mb limit. But knowing that the memory constraints aren't onerous, and I can only assume the processing requirements are low given how it seems not to impact on gameplay performance, it does beg the question of why no headset or other console. Especially as we've had cross-platform games in the past that offer voice control over headsets.

Purely good marketing by MS? Or is there a technical reason why Kinect is a better solution (excepting the very clever noise cancellation, but up against headsets that isn't an issue)

 

[kagemaru]

I normally don't go to this site, but it was linked to at GAF:

Mass Effect executive producer Casey Hudson says BioWare is bringing voice commands to Kinect and Xbox 360 because Microsoft's sound and motion-sensing controller is doing the hard work of voice recognition. Kinect's internal hardware is what makes it possible—or at least much easier to get voice commands working—not just its microphones.

http://kotaku.com/5810138/mass-effect-3s-new-voice-commands-are-xbox-360+only-and-heres-why

I would also imagine that MS provides libraries and tools for these types of additions, that way the developers don't have to develop them. Plus the Kinect does have a couple chips and 64MB of memory IIRC, so I'm sure there could be technical reasons (as described in the quote above).

I could be wrong though.

 

[patsu]

It's doable technically as long as you have a mic close to your mouth. SingStar users can speak into the mic to navigate and select songs for 2-3 years now. It works pretty well. There may be additional run-time license(s) involved though.

Kinect's tech is more for "across the room" voice recognition. The Move SDK also has a voice recognition library, according to SCE's tech slides. However, I suspect Microsoft or Kinect users may have paid for the voice recognition run-time licenses on behalf of the game developers ?

For other games, most of the time, pressing a button is faster since the controller is already in your hands. I suspect with touch input (3DS, Vita, WiiU pad, iOS/Android), user experience in games will become even more intuitive and flexible. So there hasn't been much need to yell "Throw grenade" in a FPS game (It would be too late !). Even something like Siri is only helpful for complex actions (e.g., replaces typing). Perhaps chatty MMO is the next step. ^_^

 

[kagemaru]

It's doable technically as long as you have a mic close to your mouth. SingStar users can speak into the mic to navigate and select songs for 2-3 years now. It works pretty well. There may be additional run-time license(s) involved though. The Move SDK also has a voice recongition library, according to SCE's tech slides.

For other games, most of the time, pressing a button is faster since the controller is already in your hands. Kinect's tech is more for "across the room" voice recognition.

I suspect with touch input (3DS, Vita, WiiU pad, iOS/Android), user experience in games will become even more intuitive and flexible. Something like Siri is only helpful for complex actions (e.g., replaces typing). Perhaps chatty MMO is the next step. ^_^

No one said it wasn't doable, games last gen had voice recognition.

He asked for some reasons and I posted a quote with a link.

 

[patsu]

??? I didn't read your post at all, sorry. ^_^

The OP asked if it's a technical reason or a marketing one. I simply typed my reply.

 

[kagemaru]

??? I didn't read your post at all, sorry. ^_^

The OP asked if it's a technical reason or a marketing one. I simply typed my reply.

Oh sorry, since you mentioned how sony "also" has voice recognition libraries, I thought your post was more of a reply to mine.

 

[BadTB25]

The Kinect voice commands can be quite fast as you don't have to wait for the page to load before issuing the next command. I find it as fast or faster than using the controller when navigating to an app.

I use it more than I use the controller mostly because it is so convenient while it allows me to leave my hands free.

 

[patsu]

For navigation and searching within a closed context, voice commands can be pretty helpful. It's also pretty handy for chaining actions across contexts/apps together.

For gaming actions, it will be case by case. the "throw grenade" example is probably not a good case because voice recognition is too slow, and you can't aim the drop by talking.

 

[Rotmm]

I normally don't go to this site, but it was linked to at GAF...etc

Thanks for that, interesting link.

However Casey Hudson is still being kinda vague about why "no mic" vs why "yes kinect" leading one to the suspicion that the Kinect unit itself is doing nothing special, it really is all down to the software MS have put in place and the reality is that if they included mics for voice recognition then they would work just as well.

Of course, with Kinect you would still have the luxury of no headset/mic when giving commands, which for Dashboard navigation and media stuff is great. But why not at least enable the use of mics in games such as Mass Effect 3, where wearing a headset to give orders would be no problem for most gamers?

 

[Rotmm]

...However, I suspect Microsoft or Kinect users may have paid for the voice recognition run-time licenses on behalf of the game developers ?

Yes I know later Singstar titles have had voice navigation, and as mentioned before titles like Endwar were built with voice commands built in. So it is technically doable.

I guess my question is more around, now we are seeing it in more Xbox titles, does Kinect actually make the process any easier / less resource hungry / more comprehensive, etc than it would be using a headset mic? Or is it just MS cajoling developers into integrating it then specifying only for Kinect even though the same librarys could just as easily be used with a headset microphone. ie: purely a marketing and not a technical limitation.

As for the part I quoted from you above, I never thought about the licenses part of the equation and how funding the use of said licenses on the Xbox platform could be a reason why the same functionality is not offered on other platforms (ie: PS3 and PC). Good call.

 

[Davros]

Yesterday I tried out the voice commands in Tom Clancy's Hawx, (on the pc, but it is available on the consoles) and to be honest its just easier to press the buttons on the joystick and so after about 20 minutes I stopped using them.

where I could see voice commands being useful however is in a proper flight simulator where it could actually be easier to learn the command names rather than the keypresses.

For those that are not familiar with flight simulators here are the controls for the mig29 in lock on modern air combat (there are 5 other planes in the game and while many commands are shared between planes many are unique)



ps: just tried quake 3 in win7 (win7's built in vioce recognition) (opened by saying "open quake 3") and i can shoot by saying "left click"

 

[Silent_Buddha]

Thanks for that, interesting link.

However Casey Hudson is still being kinda vague about why "no mic" vs why "yes kinect" leading one to the suspicion that the Kinect unit itself is doing nothing special, it really is all down to the software MS have put in place and the reality is that if they included mics for voice recognition then they would work just as well.

Of course, with Kinect you would still have the luxury of no headset/mic when giving commands, which for Dashboard navigation and media stuff is great. But why not at least enable the use of mics in games such as Mass Effect 3, where wearing a headset to give orders would be no problem for most gamers?

It could be as simple as the voice recognition software that Kinect uses is stored within the firmware of the unit itself. Without a Kinect hooked up to the X360, those advanced voice recognition routines aren't available on the console.

As well there's always a chance that the voice recognition routines are intimately interacting with or are a part of whatever is used to interface with the multiple microphones that the Kinect unit uses and hence may or may not even have a way to interact with an external microphone.

Regards,
SB

 

[Davros]

does the 360 multitask ?