Sony's possible answer to the NR controller

[mckmas8808]

These are some of the thing that I can really see Sony going now that the Nintendo controller has been revealed. While reading about the demos keep in mind that this stuff was shown in Feburary of this year and that Sony did show something like the following at E3. So don't be surprised when Sony shows stuff in next year the closely resembles things that Nintendo will be doing on the Revolution. The following demos were performed by Dr. Richard Marks, one of the creators of the Eye-Toy and he talks about new directions in interface design.

Head Tracking: Head-tracking works with the current Eye-Toy, and it's been successfully applied to Eye-Toy: AntiGrav, where you'd move your head and arms to steer a snowboarder. But there are other applications, as well. One demo was of a 3D instant replay: Dr. Marks brought up a freeze-frame of a soccer match. Then, as he moved his head back and forth, the scene rotated accordingly, showing the action from multiple angles like in The Matrix and swinging the camera around with his head in real-time.

This technology could be very cool when applied to first-person shooters. Imagine that you creep up to the corner of an alley. Then, when you want to peek around the corner, instead of strafing your character over you could just move your head and literally 'peek.' Dr. Marks showed a demo of a street scene where he moved his head to look down a street and then moved it back to duck around cover as the bullets flew.

Unfortunately, head-tracking technology uses up about 20% of the CPU with the current generation of game platforms. No first-person game developer would want to do that. But in he next generation of hardware, the processors are many times more powerful and the drain on system resources is much more trivial. Expect to see some cool things!

The 'Minority Report' Demo: Okay, most of us saw the movie. Tom Cruise put on special gloves and then played with a computer interface by literally dragging and dropping folders and manipulating data with his hands. Guess what? The technology is just about there, and even though it's still rooted to a TV screen, at least you don't need the goofy gloves.


The secret is in "retro-reflective" material. This material takes low light (say, the dim LED on the front of the Eye-Toy camera) and reflects 100% of it back at the lens. The upshot is that the camera can track specific objects with nearly pixel-perfect accuracy. Dr. Marks picked up a small rubber doohickey that he held in his hand coated with a retro-reflective surface. Then, he opened up an on-screen photo gallery. Using the device, he could reach around the screen to select individual photos. He could "grab" them by squeezing the device (the demo made a satisfying 'click!') and drag them around the screen. He could even grab them and then rotate them. The rotation was a little dodgy (sometimes it took two tries), but it worked, and it worked using technology that's pretty much available to consumers today.

Future Cameras and "Per-Pixel Distance Measurement." I didn't really 'get' this technology until Dr. Marks showed the demo. Here's the deal: current camera interfaces (like the Eye-Toy) can track movement. You wave your arms in front of the camera, and it can track the waving motion. But if someone stands behind you and jumps around, it'll track that motion, as well. And if you stand in all black against a black background, forget it: no matter how much you move, the camera can't see you.

Sure, that's cool. But once the camera knows how close or far away you are, it opens up other possibilities, specifically in the way that digital objects interact with the real world depicted on the screen. In another demo, Dr. Marks held up a wand that attracted butterflies that swirled around it on the screen. As he moved the wand, the butterflies flew to chase it. But then, when he passed the wand behind his back, the butterflies on the screen flew behind him. It really looked like they were flying around him. The illusion was much more complete than you can get with today's technology.

Hold up, he hasn't even started. Cameras with this kind of resolution can do real-time motion capture. So, you can dance in front of the camera, and all of your movements can be tracked and then applied to a digital model rendered on the screen. In his next demo, Dr. Marks moved around and on the screen a skeletal version of himself moved to match. He'd wave his arms and the skeleton would do the same. Physics was built into the simulation, so when he punched his arms forward, the skeleton punched, and it could hit objects around the virtual room. Because the camera was tracking distances, it could actually track where he was in a 3D space -- standing in certain spots triggered certain actions, for instance. The Eye-Toy's motion tracking looks pretty primitive in comparison. Imagine the gaming possibilities of this kind of interface! You'd literally be, full body, involved in the on-screen action, stepping into another character. Lastly, he showed a Matrix-style bullet-dodging game, in which the player physically leaned in various directions to avoid bullets coming at his on-screen character.

New Interfaces for Handhelds.
Adding a camera to a handheld can open up tons of possibilities. Dr. Marks, who used the PSP in all his examples, is particularly excited about the idea of using a handheld as a "lens" to view the world. In other words, stick a camera on top of the PSP facing out, then look "through" it (at your screen) at the real world but with digital stuff overlaid on top of it.

His example completely blew me away. Imagine being able to set a retro-reflective marker onto your coffee table, something that the game can track. Now, you point the camera at it and look through your game console screen and 3D characters could be imaged right on top of the real-time picture of your coffee table. Characters in a fighting game, for example, could be squaring off on your furniture. You could stand up, move around, and look at the battle from different angles: the characters would all be rendered appropriately. In a multiplayer game, you could have several players standing around the coffee table all looking at different views of the same 3D combat.

Some neat stuff huh? A little PSP stuff too just for chucks. Looks like Sony will have some Nintendo like stuff heading into the next-gen future including the PSP. My biggest point in this thread topic is to discuss from the above demos what else we can think of to further video game development. The first thing that I thought of was Fight Night 3. Can you imagine fighting the computer controlled Bernard Hopkins or with your real hands? OMG next-gen will be great.

Oh and the second reason that I posted this is the show that Nintendo is not the only innovators of video game technology. I'm tired of seeing Nintendo iz teh inn0vators, $ony and M$ are not. They all are doing big things for videogames.

 

[gkar1]

Oh and the second reason that I posted this is the show that Nintendo is not the only innovators of video game technology. I'm tired of seeing Nintendo iz teh inn0vators, $ony and M$ are not. They all are doing big things for videogames.

Hasnt it ocurred to you that the reason why everyone keeps saying Nintendo is innovative is because its true? Next time post this on your blog

 

[BTOA]

Isn't this old news?

 

[mckmas8808]

Hasnt it ocurred to you that the reason why everyone keeps saying Nintendo is innovative is because its true? Next time post this on your blog

Did you see the part where I stated that some people say Sony and MS are not? They are that was my point in saying that.

Isn't this old news?

Yes I stated that in my post. Yet it is nice to see what someone from SCEA is doing. What's wrong with talking about what Sony is doing? I love what Nintendo is doing with the new controller (I wish the shell was the real controller and the wand was something extra) but I wanted to keep people updated on what Sony is doing. No problem in that is it?

 

[mckmas8808]

Screenshot that is demostrating the 3D mo-cap abilites of the eye-toy.

Notice when the man does something the skeleton does the samething. I also like the Reset and Next areas on the digital floor. And those colored blocks must be what demostrator was punching.

 

[Powderkeg]

I always thought Sony's answer to the Nintendo controller would come in the form of dominating sales.

 

[mckmas8808]

I always thought Sony's answer to the Nintendo controller would come in the form of dominating sales.

Well ok. Being serious though shouldn't everybody here be really excited to hear that the eyetoy on PS3 could give near Minority Report feel?

Dr. Marks picked up a small rubber doohickey that he held in his hand coated with a retro-reflective surface. Then, he opened up an on-screen photo gallery. Using the device, he could reach around the screen to select individual photos. He could "grab" them by squeezing the device (the demo made a satisfying 'click!') and drag them around the screen. He could even grab them and then rotate them. The rotation was a little dodgy (sometimes it took two tries), but it worked, and it worked using technology that's pretty much available to consumers today.

Isn't that great?

 

[Steve Dave Part Deux]

Seeing what the Revolution can actually do is the only thing really keeping me interested in next gen at all. Although, I imagine I'll end up owning all three systems at some point. I'm still interested in what MS and Sony have to offer. The difference between the eye-toy and what revolution is doing, is in the focus of the company at large. Nintendo is going to throw everything they have into Revolution (it's they're best strategy). I doubt you're going to see Sony's top software designers start developing software exclusively for the eye-toy just yet. It's obviously not their bread and butter and they're dominating the market already. Why would they fundamentally change anything unless they had to? Your point is taken though. The research is very interesting.

 

[Sis]

[I posted this in another thread, in response to your post about the Eye Toy. this is a much better place to discuss it ]

The Eye Toy is an optical device. It requires that the camera is able to accurately record information and the processor to accurately process that information (that is, figure out what the player was doing). In controlled environments I'm sure that the Eye Toy may work very well as an input device, but not so much in most living room environments. Similar to voice command systems, there will always be enough ambiguity of input that complete accuracy called for by a controller will not be achievable.

People aren't falling for Nintendo "hype". These guys are innovators and have been for a long time, and doing it both in hardware and software. Giving credit where credit is due is not "falling right in for it".

.Sis

 

[mckmas8808]

[I posted this in another thread, in response to your post about the Eye Toy. this is a much better place to discuss it ]

People aren't falling for Nintendo "hype". These guys are innovators and have been for a long time, and doing it both in hardware and software. Giving credit where credit is due is not "falling right in for it".

.Sis

When I say falling for it I mean the part when people think Nintendo is the only ones doing something innovative. What I mean is they are not the only was changing how videogames are played or viewed.

Keep in mind that the eyetoy according to Richard is only using less than 1% of the processing power of the CELL while it used 20% of the PS2's power.

 

[SGX-1]

Difference here is Nintendo will have many games that make use of their controller since it's the main interface. Eyetoy will be some accessory which gets little support. Doing research in the lab is fine and dandy but there are a lot of companies doing the same thing not just SONY or Nintendo when it comes to research. Heck I'm pretty sure Nintendo has been doing lots of reasearch on VR goggles using panoramic flexible displays and are trying to get the costs down so they can inplement it into a future console. This VR goggle might even work with Evolution the portable. In fact this VR goggle might be the main display for a future portable successor to the VB...

What I see is some people falling for the SONY hype after seeing some rubber ducky demo.

 

[DemoCoder]

What some of Marks demos are showing with eyetoy/psp+cam is AR or Augmented Reality, which is much more innovative and bleeding edge than a 3D gyro mouse. The algorithms involved and technology required are much more advanced than what Nintendo requires to make their device work.

 

[SGX-1]

What some of Marks demos are showing with eyetoy/psp+cam is AR or Augmented Reality, which is much more innovative and bleeding edge than a 3D gyro mouse. The algorithms involved and technology required are much more advanced than what Nintendo requires to make their device work.

What really matters is the experience, not how complex the technology is afterall we're talking about a user interface here. AR is a good marketing buzzword though...

I wouldn't be surprised if Nintendo's research on panoramic VR goggles using flexible displays has vertical and horizontal head tracking and will work hand in hand with Revolution's DPD.

 

[oli2]

The algorithms involved and technology required are much more advanced than what Nintendo requires to make their device work.

If i read correctly what you said in the other thread, Nintendo 's device is not supposed to "work", 'cause we will have awfull gameplay with it.

What you add here is : "Nintendo's device is not as advanced as what Sony will bring to us".

Ahhhh, okay ....

 

[mckmas8808]

What really matters is the experience, not how complex the technology is afterall we're talking about a user interface here. AR is a good marketing buzzword though...

Buzzword eh? Well what would you call it?

What I see is some people falling for the SONY hype after seeing some rubber ducky demo.

So when Sony shows something innovative its teh SONY hype, but if Nintendo shows something innvative its the second coming of Jesus Christ. Hmmm... I really can't figure that one out.

 

[Vince]

What you add here is : "Nintendo's device is not as advanced as what Sony will bring to us".

And are you seriously trying to have us believe the above statement isn't true?

 

[Bad_Boy]

Screenshot that is demostrating the 3D mo-cap abilites of the eye-toy.

Notice when the man does something the skeleton does the samething. I also like the Reset and Next areas on the digital floor. And those colored blocks must be what demostrator was punching.

is there a video of this? looks interesting.

 

[SGX-1]

And are you seriously trying to have us believe the above statement isn't true?

You're taking his comment out of context. He's saying more advanced doesn't equal better gameplay.

Buzzword eh? Well what would you call it?

You're barking up the wrong tree. What it's called is just marketing. What it does for games is what matters. A dance mat, bongos, or a lightgun are accessories. Eyetoy is an accessory. Nintendo's DPD is not an accessory.

So when Sony shows something innovative its teh SONY hype, but if Nintendo shows something innvative its the second coming of Jesus Christ. Hmmm... I really can't figure that one out.

What they're showing are niche uses in the lab. What Nintendo showed will be packaged with every Revolution system and used by many different types of games. That's the difference between hype and non-hype.

 

[xmu]

Screenshot that is demostrating the 3D mo-cap abilites of the eye-toy.

Notice when the man does something the skeleton does the samething. I also like the Reset and Next areas on the digital floor. And those colored blocks must be what demostrator was punching.

Actually this isn't Eye-Toy. This image is a part of the SCEE presentation at GDCE, and this was from the "glimpse of stuff to come in the future". The camera they used in this one was said to cost something like £20k. And by future they meant like 5-10 years. The demonstration itself was very cool, especially the Matrix style part where the guy had to dodge bullets.

Btw I don't see what there's to LIKE about the reset and next buttons, their function was just that.. reset the current demonstration or move to the next one.

Oh and I should mention that the demonstration was naturally in video form only.

 

[oli2]

And are you seriously trying to have us believe the above statement isn't true?

Of course not.
What interested me was that some people are ready to give arguments after arguments to deny all the possiblilities openened by the controller.
Either this doesn't work, or Sony or Microsoft have better to give us. Don't you think logical order in the arguments would have been the other ?

aka : "i think what Sony will provide us will provide better immersion, and i have doubts about the quality of the gameplay with NR's controller".