The Next-gen Situation discussion *spawn

[TheChefO]

Retrofitting for usb3 might not be worth the effort.

Indeed it might not be.

Depends on how valuable kinect2's adoption is to MS ...

If they plan the two tier approach of high and low end as I proposed, I'd think it would be worth while as the current implementation of kinect is rather limited, and may be doing more harm than good for brand reputation at this point ...

IMHO

 

[bkilian]

Indeed it might not be.

Depends on how valuable kinect2's adoption is to MS ...

If they plan the two tier approach of high and low end as I proposed, I'd think it would be worth while as the current implementation of kinect is rather limited, and may be doing more harm than good for brand reputation at this point ...

IMHO

Mainly in the eyes of the hardcore. We'd have to pry the controller out of those folks cold dead fingers. We could deliver a perfect implementation of the Minority Report Interface, and those folks would complain that it doesn't have buttons. In the rest of the population, Kinect, even in its current significantly flawed implementation, is magic. I see this every time I have folks over and they see Dance Central, or voice control (or some of the other features I can't talk about ). In the news, every second microsoft story is about Kinect and it's multitude of magical uses. I really don't think it's harming our reputation.

We're not done with the current system yet, there are still a lot of improvements we can make without changing out the hardware.

 

[TheChefO]

Mainly in the eyes of the hardcore. We'd have to pry the controller out of those folks cold dead fingers. We could deliver a perfect implementation of the Minority Report Interface, and those folks would complain that it doesn't have buttons. In the rest of the population, Kinect, even in its current significantly flawed implementation, is magic. I see this every time I have folks over and they see Dance Central, or voice control (or some of the other features I can't talk about ). In the news, every second microsoft story is about Kinect and it's multitude of magical uses. I really don't think it's harming our reputation.

We're not done with the current system yet, there are still a lot of improvements we can make without changing out the hardware.

At first glance I'm sure it is (as it was with me and anyone I know that has used it briefly).

But then the limitations of the accuracy and more importantly, the latency, start to become a nuisance and a turnoff.

It's great at the store demo or over at someone's house that hasn't ever seen/used it before, but after the short introduction, I fear the lasting impression at home is "do not want".


Don't get me wrong, I'm about as far from a hater as you'll find on MS and Kinect. I think with the proper implementation, Kinect2 and xbox NG could radically change the way we think about games and entertainment in general.

But in order to get there, people need to not have a bad taste in their mouth from previous purchases. Otherwise, the sell becomes that much more difficult.


Bottom line, there are certain hard limitations with Kinect as is which are building the public perception on what the device can and cannot do. And with that, a decision is made to purchase or not ... and after purchase, what is the intended purchase purpose.

At this point, expensive kids toy is the answer.

Kinect2 may change that ... assuming it has the hardware partner to match in xb720.

 

[Shifty Geezer]

We're not done with the current system yet, there are still a lot of improvements we can make without changing out the hardware.

You might not be able to talk about this, but is Kinect (or rather, some features) being constrained by the amount of processing power available to drive it? That was a limiter for EyeToy where RnD stuff never appeared on PS2, and sadly Sony lacked conviction in their followup. But if Kinect is being limited by hardware resources then TheChefO's idea of extending XB360 to a new experience won't work, at which point MS will need to introduce new hardware and face the prospect of either having two new hardware platforms - high-end and low-end - or going with one platform and trying to fit it into whatever price bracket factoring in whatever they might be able to do with price drops. ie. If they want mass adoption and need new hardware because XB360 isn't up to snuff to get all the Kinect features they want, they'll have to go with a middling spec console and a lower entry price if cost reductions aren't going to be plentiful.

 

[Arwin]

I thought on 360 Kinect was more limited by the USB port having limited bandwidth available. The same device on PC could handle more data and higher resolutions.

 

[Shifty Geezer]

It depends what you're trying to do. Tracking more limbs, tracking them more accurately, performing other recognition processes, could all benefit from processing power, whereas having higher resolution requires bandwidth. A Kinect2 on XB360 with higher bandwidth may be limited like PSEye on PS2 - what's the use in having higher resolution if the processing power isn't their to churn through that data? But then if MS's developments are mostly focussed on something like searching massive datasets, then perhaps XB360 would be enough to drive their future vision for Kinect, at which point a cheap 360 SKU at the entry level makes sense.

 

[TheChefO]

...if Kinect is being limited by hardware resources then TheChefO's idea of extending XB360 to a new experience won't work...

Keep in mind, my idea is for kinect2, not kinect1.

With that, software would obviously be tailored for the intended devices and offer (hopefully) a different experience on xb360 and xb720.

So even though kinect2 with (1080p?) increased resolution and frame rate capture would use more resources and bandwidth, that doesn't mean it wouldn't work. It just means less resources would be available on xb360 for the kinect2 xb360 versions of gameworlds as more resources on xb360 are dedicated to interaction.

Assuming of course that kinect2 doesn't have beefier processing on board

 

[Shifty Geezer]

With that, software would obviously be tailored for the intended devices and offer (hopefully) a different experience on xb360 and xb720.

So you think the next five years should see Ms at the $200 pricepoint with XB360 and Kinect, and Kinect2 and XB720 should be sat at the $400 for that time?

So even though kinect2 with (1080p?) increased resolution and frame rate capture would use more resources and bandwidth, that doesn't mean it wouldn't work. It just means less resources would be available on xb360 for the kinect2 xb360 versions of gameworlds as more resources on xb360 are dedicated to interaction.

You need enough left over from processing Kinect to drive the games. EyeToy could have done more on PS2, but not rendering a game with it. You just couldn't do Move games on PS2 in a cut-down flavour. Assuming the software technology behind Kinect is advancing, it could (and maybe even should) be impossible to execute on 7/8/9 year old tech.

Assuming of course that kinect2 doesn't have beefier processing on board

Which would elevate the cost, destroying the notion of an entry level machine.

 

[bkilian]

You might not be able to talk about this, but is Kinect (or rather, some features) being constrained by the amount of processing power available to drive it?

I thought on 360 Kinect was more limited by the USB port having limited bandwidth available. The same device on PC could handle more data and higher resolutions.

No, as far as I know, the streams on PC and XBox are identical. And the Kinect is dependent on bandwidth, CPU, and memory. The Skeletal processing is a lib developers link in, and it uses a certain amount of memory and some part of a hardware thread. Speech is the same. The amount of memory required limits the skeletal database size and the number of joints. With huge gobs of ram, we could probably track lots more joints, although generating the machine learning database ("Exemplar" for those in the know) would take months on the gigantic cluster we currently use for it. Speech is the same, we generate a database using machine learning, and if I recall correctly, generating a new model took upwards of a week and cost in the 6-7 figure range.

There's a bunch of processing on the streams even before they get given to the skeletal and speech pipelines. For audio, this is relatively heavyweight and it does all the magic like echo reduction, speech isolation, and beamforming, this is so we can get the cleanest speech track possible to send to the speech subsystem. For skeletal, It's a lot more lightweight, and is probably just a noise reduction process.

So to answer your question, I'd say memory would be the biggest concern for most of the Kinect functions, it's by far the most precious resource on the box.

 

[TheChefO]

So you think the next five years should see Ms at the $200 pricepoint with XB360 and Kinect, and Kinect2 and XB720 should be sat at the $400 for that time?

Depends on the approach by MS, but I figure xb360+kinect costs MS ~$150 currently.

Replacing the cams with HD ones I don't think will involve a significant investment ... ditto a retrofitted usb3/firewire enabled xb360. The cost reductions enabled by 28nm on the xb360 architecture should more than make up for the increased cost of kinect2.

So for example, assume 28nm xb360slim2 was ready to go by the end of the year along with Kinect2, I'd expect that bundle price to be ~$150.

Again, my thought process is don't use hardware as a profit generator. The profit center is in services and software.

You need enough left over from processing Kinect to drive the games.

Like i said, I expect there would be reduced processing available for the gameworld. That doesn't mean there's nothing left for games.

Which would elevate the cost, destroying the notion of an entry level machine.

Perhaps I'm underestimating the processing power required to interpret ~4,000,000 pixels, but If Samsung can put the tech to do a version in a TV, I'm sure MS can work out an affordable Arm+DSP solution.

 

[Shifty Geezer]

Like i said, I expect there would be reduced processing available for the gameworld. That doesn't mean there's nothing left for games.

But less than a certain amount is next to useless. I don't think there's much scope to integrate Kinect tightly and advance it any more tha it currently is, same as the limits placed on EyeToy. As bkilian says, it's mostly RAM based. Perhaps a RAM-upgraded SKU could work like the PSP?

Perhaps I'm underestimating the processing power required to interpret ~4,000,000 pixels, but If Samsung can put the tech to do a version in a TV, I'm sure MS can work out an affordable Arm+DSP solution.

There's no maximum limit to processing power that can be used. It depends on what techniques you are using, and you can always go more sophisticated to get better results. Can Samsung's TV do a clean background removal, or user removal, or face-tracking with AR augmentations? If it can, perhaps that's because a large part of the premium price is enough specialised performance to achieve that?

 

[TheChefO]

But less than a certain amount is next to useless. I don't think there's much scope to integrate Kinect tightly and advance it any more tha it currently is, same as the limits placed on EyeToy. As bkilian says, it's mostly RAM based. Perhaps a RAM-upgraded SKU could work like the PSP?

FWIR, kinect uses about 10% of the processing capability of xb360. Assuming a 4x increase in processing requirement (rez/framerate) for kinect2, we're looking at 40% for k2 leaving 60% for kinect2 game experiences (on xb360). Might not be a bad thing to look into cost of bumping the ram for better kinect2 usage (how much can 1gb gddr3 cost these days?).

So a slightly worse looking COD MW4, but at ~30FPS instead of 60FPS. (Or less overhead if beefier guts are built into kinect2...)

Not great, but that's why the xb720 is available.

There's no maximum limit to processing power that can be used. It depends on what techniques you are using, and you can always go more sophisticated to get better results. Can Samsung's TV do a clean background removal, or user removal, or face-tracking with AR augmentations? If it can, perhaps that's because a large part of the premium price is enough specialised performance to achieve that?

Dunno, just an observation on my end which lead to an assumption.

In all, it might not be a good idea to fix up kinect2 to work on xb360 for fear of closing the gap between xb360 and xb720.

Especially if the hardware is as weak as the rumors suggest ... some might not be able to tell them apart. So a simple "kinect2 is for xb720" might be the defining selling point!

Oh well, the first 5 million are going to buy it anyways ... right Microsoft? *ahem*

Seems some lessons are never learned...

 

[Shifty Geezer]

FWIR, kinect uses about 10% of the processing capability of xb360. Assuming a 4x increase in processing requirement (rez/framerate) for kinect2,

That means exactly the same techniques as now only at higher res. If you want to do more sophisticated stuff, you might need one or two orders of magnitude more processing power. eg. 2x the skeletons tracked with 2x the sample points is a 4x increase. Add background removal or 3D scene scanning or something, and who knows how much processing power is needed (or RAM, or bandwidth)? What MS need in the box processor-wise is dependent on what they are wanting to do with their Kinect platform and how.

 

[TheChefO]

What MS need in the box processor-wise is dependent on what they are wanting to do with their Kinect platform and how.

Agreed.

It might make better sense to leave k2 as an xb720 exclusive as I said.

But with that, if the processing demands become that serious, I sure hope they aren't sticking to the rumored specs ...

Otherwise we'll be looking at xb360 games with an advanced interface ... ie Wii (but without the new "wow" factor).

 

[pc999]

There's a bunch of processing on the streams even before they get given to the skeletal and speech pipelines. For audio, this is relatively heavyweight and it does all the magic like echo reduction, speech isolation, and beamforming, this is so we can get the cleanest speech track possible to send to the speech subsystem. For skeletal, It's a lot more lightweight, and is probably just a noise reduction process.

So to answer your question, I'd say memory would be the biggest concern for most of the Kinect functions, it's by far the most precious resource on the box.

People always underestimate the processing power that good audio needs (I work with audio), and how useful (and many times "cheap") a dsp can be. Personally I always favored DSP on dedicated machines, if you can layer them it can give some amazing results.

That said I am still somewhat amazed that Kinetic comes without a dsp to send processed and somewhat compressed data for XB.

Would it really cost so much more money?

 

[Arwin]

No, as far as I know, the streams on PC and XBox are identical. And the Kinect is dependent on bandwidth, CPU, and memory. The Skeletal processing is a lib developers link in, and it uses a certain amount of memory and some part of a hardware thread. Speech is the same. The amount of memory required limits the skeletal database size and the number of joints. With huge gobs of ram, we could probably track lots more joints, although generating the machine learning database ("Exemplar" for those in the know) would take months on the gigantic cluster we currently use for it. Speech is the same, we generate a database using machine learning, and if I recall correctly, generating a new model took upwards of a week and cost in the 6-7 figure range.

There's a bunch of processing on the streams even before they get given to the skeletal and speech pipelines. For audio, this is relatively heavyweight and it does all the magic like echo reduction, speech isolation, and beamforming, this is so we can get the cleanest speech track possible to send to the speech subsystem. For skeletal, It's a lot more lightweight, and is probably just a noise reduction process.

So to answer your question, I'd say memory would be the biggest concern for most of the Kinect functions, it's by far the most precious resource on the box.

I was actually referring to this:

http://www.eurogamer.net/articles/digitalfoundry-kinect-held-back-blog-entry

 

[bkilian]

People always underestimate the processing power that good audio needs (I work with audio), and how useful (and many times "cheap") a dsp can be. Personally I always favored DSP on dedicated machines, if you can layer them it can give some amazing results.

That said I am still somewhat amazed that Kinetic comes without a dsp to send processed and somewhat compressed data for XB.

Would it really cost so much more money?

Xbox 1 had a dedicated audio DSP. No one used it. That's the main reason they dropped it for the 360.
I don't know why the kinect didn't have an onboard DSP, we actually do audio processing on the kinect, but it gets used mainly for chat.

 

[bkilian]

I was actually referring to this:

http://www.eurogamer.net/articles/digitalfoundry-kinect-held-back-blog-entry

Yeah, they're referring to the color camera resolution. The depth camera is 640x480 streams on both platforms. The color camera is much higher, but only allows a 640x480 stream out. It can either be a scaled full picture, or a zoomed 1:1 picture.

 

[pc999]

Xbox 1 had a dedicated audio DSP. No one used it. That's the main reason they dropped it for the 360.
I don't know why the kinect didn't have an onboard DSP, we actually do audio processing on the kinect, but it gets used mainly for chat.

Really didnt knew that?

Do you know why? A dsp, unless it is a really limited one and/or have a good API, is usually a CPU saver and given the effort at specific coding from this gen it would seem they wouldnt mind the extra effort.

Personally I would love if all the consoles had DSPs to the more straight forward/comun tasks.

 

[ERP]

I think "nobody used it" is a bit of an overstatement.
The tools for DSP development are considerably less good than CPU development, if you can spend 5% of your CPU and save development time, you'll probably do it.

I've always referred to audio as the "4th pillar of game development, the small feable looking one in the corner".
What I mean by that is it gets very short thrift by a lot of developers, they generally do not write the sound system and for the most part it's left to the audio guys to do what they can with the supplied tools.
That is not to belittle the value of well designed audio, it can change a game, but very few developers apply the same focus to audio, as they do Technology/Art and Design.