News & Rumors: Xbox One (codename Durango)

[Cyan]

On the 360, any "bing" search does a cloud recognition (it sends the audio up to the cloud, which is enormously more capable at reco than a console). I suspect it's the same for the One. I am often amazed at how good the bing stuff is at parsing words.

I had never thought about that. If it uses the cloud it is quite surprising how fast the guy's voice is recognised and how quick it provides results.

From my little experience in games like NFS Rivals, unusual commands like Change View, Easy Drive, Interceptors, etc etc, work perfectly well. Surprising enough, that is.

I could test a game like FIFA 14 and share my results. No cloud involved, I think. If you say Substitutions a sub-menu with the name of your 11 starting players appears.

You have to select a player by pronouncing his actual name. i.d. Gareth Bale -or Bale- and then a list of substitutes appear, say.. Illarra, or something like that.

I wonder how you can pronounce certain names for Kinect to understand, like Xavi -Chavi or Shabi-. Some Manchester United players are; Dani Welbeck, David de Gea, Chris Smalling, Nani, Nemanja Vidic, Michael Carrick, Johnny Evans... :smile:

 

[eastmen]

Cyan its not any different than the original window's mobile phones , iPhone siri / google voice and the likes.

They all use much more powerful hardware in the cloud to get where they are going.

If you use google voice ( I don't use an iPhone) you can see the difference the connection speed makes by trying a search using 3g and then again lte.

 

[Jogi]

I have had pretty horrible experiences with old and new Kinect using voice commands My english is bit rusty sure but it just can't hear or understand me when I speak in normal talking voice. I have tried 2 Kinect 1 and there is no difference, and I had high hopes for Kinect 2 but it failed badly. I really have to raise my voice and even then just "xbox" is hard to get through. Yes multiple calibrations done, it does nothing to help issue, changed English US to UK, not much help either. It does not matter if I sit 4m from kinect 2 or 1m, normal speaking volume does not work. If I record video (using kinect2) to test how I sound, my sound is really metallic/robotic (4m from kinect) It gets bit better when I try from 1m but still awful sound quality.
Yes I have buzzing plasma tv and air condition unit in same room but sad thing is that even my plasma tv can hear and understand me much better even if it has just 2 cheap microphones integrated.

 

[bkilian]

I had never thought about that. If it uses the cloud it is quite surprising how fast the guy's voice is recognised and how quick it provides results.

From my little experience in games like NFS Rivals, unusual commands like Change View, Easy Drive, Interceptors, etc etc, work perfectly well. Surprising enough, that is.

I could test a game like FIFA 14 and share my results. No cloud involved, I think. If you say Substitutions a sub-menu with the name of your 11 starting players appears.

You have to select a player by pronouncing his actual name. i.d. Gareth Bale -or Bale- and then a list of substitutes appear, say.. Illarra, or something like that.

I wonder how you can pronounce certain names for Kinect to understand, like Xavi -Chavi or Shabi-. Some Manchester United players are; Dani Welbeck, David de Gea, Chris Smalling, Nani, Nemanja Vidic, Michael Carrick, Johnny Evans... :smile:

The reco for stuff that doesn't require cloud uses the built in speech database which essentially uses Bayesian classification to find the most likely text that your speech matches. It is limited in that you have to give it the list of words you expect, and it will give you the probabilities that what you said matches one of those words. You can specify custom pronunciations for words if you find that you're not getting the performance you expect from the defaults. On the 360 it couldn't handle more than 15 or so phrases at a time, but you could switch out the phrase dictionary at any time to enable multi-level menus. Phrases often get tuned to get the best differentiation among the selections. For instance, "Xbox off" was abandoned in favor of "Xbox turn off" because the first phrase wasn't hitting good recognition percentages.

 

[bkilian]

I have had pretty horrible experiences with old and new Kinect using voice commands My english is bit rusty sure but it just can't hear or understand me when I speak in normal talking voice. I have tried 2 Kinect 1 and there is no difference, and I had high hopes for Kinect 2 but it failed badly. I really have to raise my voice and even then just "xbox" is hard to get through. Yes multiple calibrations done, it does nothing to help issue, changed English US to UK, not much help either. It does not matter if I sit 4m from kinect 2 or 1m, normal speaking volume does not work. If I record video (using kinect2) to test how I sound, my sound is really metallic/robotic (4m from kinect) It gets bit better when I try from 1m but still awful sound quality.
Yes I have buzzing plasma tv and air condition unit in same room but sad thing is that even my plasma tv can hear and understand me much better even if it has just 2 cheap microphones integrated.

Your environment must be pretty bad. I never have to raise my voice at all. I'm guessing it is a combination of environment and accent recognition difficulties. Is your Kinect above your TV, or in front/ below it? If above then the high tension buzzing from the plasma will cause serious issues with speech, and would also be responsible for the metallic sound when you record video. I'm assuming you have trouble even if the console is not playing any audio, so it's probably not related to putting the Kinect in front of or on top of the center speaker, which is the most common issue I encountered when troubleshooting setups.

 

[NRP]

bkilian,
So are you saying I shouldn't mount my Kinect on top of my plasma TV? Damn, where else can I mount it? My center channel speaker is right below my TV.

 

[AlphaWolf]

Beside should be ok the fov is pretty wide.

 

[Scott_Arm]

bkilian,
So are you saying I shouldn't mount my Kinect on top of my plasma TV? Damn, where else can I mount it? My center channel speaker is right below my TV.

I can't imagine buzzing from your plasma will matter unless it's of appreciable volume. My LCD has a very quiet but audible fan. My Kinect is mounted on top of the tv. My speakers are probably 2 feet away from it on either side. Kinect can pick up my voice, at normal speaking volume, even when I'm playing music over my speakers while playing a game with the game sound coming from my tv.

 

[tuna]

I can't imagine buzzing from your plasma will matter unless it's of appreciable volume. My LCD has a very quiet but audible fan. My Kinect is mounted on top of the tv. My speakers are probably 2 feet away from it on either side. Kinect can pick up my voice, at normal speaking volume, even when I'm playing music over my speakers while playing a game with the game sound coming from my tv.

My plasma buzzes as hell when the screen is white.

 

[dobwal]

My plasma buzzes as hell when the screen is white.

Which shouldn't matter during the calibration of Kinect.

 

[bkilian]

Which shouldn't matter during the calibration of Kinect.

But will matter at other times. I have my Kinect above my plasma, but it's shielded from the buzzing by a shelf. I was just trying to think of situations that would give the issue jogi is describing. It's not a device issue, because he's had the same problems with 3 different devices, so it is an environment/user issue. He claims normal video recording renders his voice as "metallic", and that seems to indicate environment issues. The square wave generated by a plasma buzz (and some are much louder than others) would give that characteristic to the recorded voice, but other things could too.

Only worry about your placement if you find that performance appears very degraded. If it's working fine, it's placement is probably ok (even if it's on top of a speaker or in a cubby)

 

[ERP]

Your environment must be pretty bad. I never have to raise my voice at all. I'm guessing it is a combination of environment and accent recognition difficulties. Is your Kinect above your TV, or in front/ below it? If above then the high tension buzzing from the plasma will cause serious issues with speech, and would also be responsible for the metallic sound when you record video. I'm assuming you have trouble even if the console is not playing any audio, so it's probably not related to putting the Kinect in front of or on top of the center speaker, which is the most common issue I encountered when troubleshooting setups.

FWIW I have exactly the same issue, and my accent would be considered pretty mild, it's what's left of my English accent after 20 years in the States.
My Kinect is in front of the TV (which is a plasma), and I have to raise my voice for it to understand anything, some commands I simply cannot get to execute. For example Xbox Pause is virtually impossible for me to trigger, My GF can get that to trigger, but even she has to raise her voice.
Voice commands are a really very cool experience except when they don't work..

If Plasma TV's are an issue, exactly where is ideal to place the device?

 

[BeyondTed]

...

Do you know if the audio DSP in the Xbox One can be used to "improve" the speed, latency and accuracy of note detection for games like Rocksmith (using real guitars as opposed to the MIDI guitars used in Rockband 3)?

 

[BRiT]

If Plasma TV's are an issue, exactly where is ideal to place the device?

It tends to work fine for me on my Plasma TV, the Panasonic TC-P54V10, where it's placed on the front left side. I have the center-channel speaker of a 7.1 sound stage placed on the front right side on top of the media center (when looking at the TV).

 

[ERP]

It tends to work fine for me on my Plasma TV, the Panasonic TC-P54V10, where it's placed on the front left side. I have the center-channel speaker of a 7.1 sound stage placed on the front right side on top of the media center (when looking at the TV).

I have basically the same TV, I figured my issue was the center channel speaker which is under the TV, and was going to try mounting the Kinect on top of the TV, though I need to manufacture a mount for it.

Clearly part of my issue is the slight accent, since most commands seem to work with a raised voice, but I would have expected better sensitivity than I'm getting from the mic.

 

[bkilian]

FWIW I have exactly the same issue, and my accent would be considered pretty mild, it's what's left of my English accent after 20 years in the States.
My Kinect is in front of the TV (which is a plasma), and I have to raise my voice for it to understand anything, some commands I simply cannot get to execute. For example Xbox Pause is virtually impossible for me to trigger, My GF can get that to trigger, but even she has to raise her voice.
Voice commands are a really very cool experience except when they don't work..

If Plasma TV's are an issue, exactly where is ideal to place the device?

I don't know if plasma tvs, in general, are an issue. My South African after 14 years in the states accent also has the hardest time with "xbox pause". There are three reasons for this: 1) You very rarely say "Xbox Pause" when the XBox is just sitting quietly at the dashboard, it's a command that will always have to be paired with multichannel echo reduction - This makes it inherently slightly less effective than when talking to a quiet system, 2) Brits and South Africans say "pores", Americans say "paahz". To speech reco, those are two vastly different sounds, which brings us to 3) It's a very short phrase. Bayesian classification works better with more data, but "pause" is a single syllable.

As to your environment, a number of things contribute to how well your system performs.

• Size of the room - in large rooms less energy gets to the mics, and you may have to speak up
• Wall treatments, floor type - hard surfaces are bad.
• Sound system - Is your sound system doing any kind of non-linear processing (night mode, dolby pro logic - turning on either of those things degrades echo reduction by about a third)
• Kinect placement - Kinect should ideally be at least 3 feet from any speaker (this one is often the hardest one to do) and never in an enclosed space, as that messes up the calibration (which is really just measuring a room impulse so it can try to reverse the rooms reverb effect on the sound output)
• Temperature - Temperature fluctuations in your room can slowly invalidate the calibration, I have no idea why this is true, but I have thousands of samples showing the correlation
• Ambient sound - Any ambient sound will decrease reco performance. This includes the console fans, TV buzzing, Aircon unit compressor and fans, highway noise from outside, children, pets, and vacuum cleaners. The pipeline can compensate for this slightly, but removing those frequencies also removes data that would have been used in the reco, and your percentage matches go down.

The technology is improving rapidly. 10 years ago, the prevailing thought was that multichannel echo cancellation was an absurdity, theoretically impossible to achieve (which it is). Today we know that we can get close enough that the experience is reasonable. As processing power goes up, that performance will improve still further. The problem for Kinect, like any groundbreaking technology, is that it's sitting at the forefront of the research, and that may be it's downfall, if they're not careful. There is always a period when a new disruptive technology gets introduced where it's hard to use and can fail.

 

[bkilian]

Do you know if the audio DSP in the Xbox One can be used to "improve" the speed, latency and accuracy of note detection for games like Rocksmith (using real guitars as opposed to the MIDI guitars used in Rockband 3)?

No idea. Sorry.

 

[Cyan]

(....)

Well, I have been playing FIFA 14 to test voice recognition with "unexpected commands", like Substitutions.

The game let you choose the language (I choose English) and the accent of the Kinect voice recognition -UK English Neutral Accent, Mancunian, Cockney, and a couple of others-, and I chose English Neutral.

Voice commands worked perfectly for me. Saying Formations or Substitutions or Tactics while playing brought up a very useful command list which doesn't distract you from the action.

What I wanted to test was if Kinect recognised the names of the players. So I played a match between Manchester United and Real Madrid and began to make substitutions.

When I said "Substitution", a list of my starting players appeared horizontally on the screen.

It was then when it got interesting.

First substitution. I said Cristiano Ronaldo from the starters list and even if it is a compound name it recognised my voice perfectly.

Then a list of reserves appeared and I said Modric. When the ball got out of bands there goes Cristiano Ronaldo and Modric began to play.

I said "Substitution" again afterwards. The menu appeared and I chose another complicated name. Illarramendi in this case.

From the reserves menu I chose Di Maria. Again, success, and it got me surprised once more especially because of Illarramendi.

I also hinted at substitutions without actually completing them -you are limited to 3 per match and team- and when the menu appeared the game recognised all the names of the players -Xabi Alonso, Isco, Pepe, Arbeloa (this one is tricky, yet it got it), Khedira-.

I switched sides and began controlling the Manchester United. My substitutions list was like this:

Smalling – Büttner (pronounced Butner, Batner didn't work for me)

Fellaini-Hernandez (so surprised it worked perfectly well for both)

Welbeck-Evra

Went back to Real Madrid again afterwards and my next substitution was:

Sergio Ramos-Morata (it was neat to see how Sergio Ramos doesn't seem to be a particular easy name to pick up well, yet it worked)

I only had trouble with two players, Januzaj and Carvajal, because I don't know how to pronounce their names. (Yanusah?, Carvahal, Carballal?). Not sure.

All in all, it worked much better than I expected. Be it for Substitutions, Formations -4-4-2, 4-3-3, 4-3-3 False 9, 4-2-3-1 etc etc- or Tactics -Defensive, Attack, etc-.

These things greatly add to the gameplay and flow of the games that actually implement them, like NFS Rivals and FIFA 14.

In fact it is like 10 times faster to make substitutions that way than pausing the game, going to Team MAnagement, Squad, etc etc.

 

[Cyan]

News:

Phil Harrison said that the power of the console hasn't been realised. He went on to say that the development tools are still maturing and as they improve and developers familiarise themselves with the platform people will see more advanced technologies and games running on the console.

http://www.gamespot.com/articles/microsoft-full-power-of-xbox-one-not-yet-realized/1100-6416569/


Microsoft have released a video featuring some very interesting tips and tricks to use the interface as efficiently as possible. There are little secrets mentioned too.

 

[baten]

I wonder if it would be possible to re-assign commands, like changing "xbox pause" with another word - say "xbox interrupt".
Customise the dictionary?