3D Scanning: A case for standard Kinect 2.0 / PS4 Camera this generation ?

onQ

Veteran



This tech should be a standard for all Xbox One & PS4 games, Not only for face scanning they should also use it to allow scanning of toys or your room into games.
 
This tech should be a standard for all Xbox One & PS4 games
Huh? It clearly isn't suitable for all games. How do you mandate 3D scanning in something like Rayman Legends or Child of Light?
Not only for face scanning they should also use it to allow scanning of toys or your room into games.
How do you create and construct the geometry? Laa-Yosh has described the issues with scanning on numerous occasions - it doesn't create workable, game friendly geometry. The head capture examples work by being variations on a standard structure, but there's no way to scan a teddy bear or toy snake or Buzz Lightyear figurine and have it converted into game geometry. The best you could hope for is a voxelised object, or simple objects being turned into surfaces (boxes). A game engine created specifically around object scanning may be possible, but there's absolutely no way it can be dropped into every game.
 
A lot of games use customiseable characters though, and a lot pf them might benefit from this. I'd certainly find it amusing if I could walk around in The Witcher with my own likeness.
 
Sure, for characters it's a great fit. I believe the first version of that was actually EyeToy with 2D stick-ons. If they're extra smart about it, they could probably use the TV as a light source to get a reasonable albedo. Straight images won't work with baked in lighting.
 
Nintendo had a mode in ... Perfect Dark .... I think it was, where you could import face photos from the Gameboy camera and transfer them into the multiplayers game.

But then it got pulled because of something about turning children into murdererereders or something.

So there you go. Something that wasn't Sega first, for once!
 
Huh? It clearly isn't suitable for all games. How do you mandate 3D scanning in something like Rayman Legends or Child of Light?How do you create and construct the geometry? Laa-Yosh has described the issues with scanning on numerous occasions - it doesn't create workable, game friendly geometry. The head capture examples work by being variations on a standard structure, but there's no way to scan a teddy bear or toy snake or Buzz Lightyear figurine and have it converted into game geometry. The best you could hope for is a voxelised object, or simple objects being turned into surfaces (boxes). A game engine created specifically around object scanning may be possible, but there's absolutely no way it can be dropped into every game.

I mean have it as a standard part of the SDK for all games to use. Something like a Teddy Bear & Buzz Lightyear should be doable by letting us attach pre defined templates to the scans. So something like a teddy bear & Buzz could use a skeleton template while something like a snake could have a string template.


Sure, for characters it's a great fit. I believe the first version of that was actually EyeToy with 2D stick-ons. If they're extra smart about it, they could probably use the TV as a light source to get a reasonable albedo. Straight images won't work with baked in lighting.

NBA 2K15 lights the TV screen up white to help with the lighting. But I think they can take that even further for toy scanning by having you place the toy down in front of the TV while it flash different colors to create more accurate scan.


They did something like this before with the tank demo & Little Big Planet but it was more of a texture overlay now they are using the processing of the PS4/Xbox One to make the 3D scans into polygons then texture them.

 
Nintendo had a mode in ... Perfect Dark .... I think it was, where you could import face photos from the Gameboy camera and transfer them into the multiplayers game.

But then it got pulled because of something about turning children into murdererereders or something.

So there you go. Something that wasn't Sega first, for once!

Not the same there is a 3D mesh being created from the scan.
 
Not the same there is a 3D mesh being created from the scan.
Your examples don't create a mesh from a scan. The mesh exists already, with correct topology, seams, skinning, etc., and it's just adjusted to the 3D depth data.
 
Nintendo had a mode in ... Perfect Dark .... I think it was, where you could import face photos from the Gameboy camera and transfer them into the multiplayers game.

But then it got pulled because of something about turning children into murdererereders or something.

So there you go. Something that wasn't Sega first, for once!

Actually I think you could import your face in the arcade version of Funky Head Boxers, so Sega was first (as always :) ).
 
Your examples don't create a mesh from a scan. The mesh exists already, with correct topology, seams, skinning, etc., and it's just adjusted to the 3D depth data.

Champion creation use your 3D face to make the polygon mesh they just stylize it.


Edit: To be clear I'm saying that your 3D point cloud is being used in the creation of the shaping of the polygons & not that the polygons are being created by your face.
 
Last edited by a moderator:
Right. They shape an existing mesh to fit the 3D data. That only works when the scanned data is comparable to the existing mesh. Arbitrary scanning and object creation is tricky. It's been demonstrated with MS's realtime Kinect room scanning demo, but the structure of the 3D mesh was never explained AFAIK. Could it scan a 3D teddy? Yes. Could it create a triangle mesh from that scan that'd work with skinning, so you could play the scanned character in game? No. Creating an object that works skinned to bones with correct deformations is a not inconsiderable task. You could scan objects that have a high similarity with a template, which is effectively what MS does with Kinect recognising people (a standard skeleton is mapped to a range of bodies that closely match that skeleton structure), but even that's not perfect, and the variety in objects will greatly exceed variance in people. Google image search "cuddly toy" and the first few example bare little structural similarity.

Creating a static object would be a lot easier as you wouldn't have to worry about topology too much, as long as triangle count wasn't ridiculous. But that has limited uses.
 
Right. They shape an existing mesh to fit the 3D data. That only works when the scanned data is comparable to the existing mesh. Arbitrary scanning and object creation is tricky. It's been demonstrated with MS's realtime Kinect room scanning demo, but the structure of the 3D mesh was never explained AFAIK. Could it scan a 3D teddy? Yes. Could it create a triangle mesh from that scan that'd work with skinning, so you could play the scanned character in game? No. Creating an object that works skinned to bones with correct deformations is a not inconsiderable task. You could scan objects that have a high similarity with a template, which is effectively what MS does with Kinect recognising people (a standard skeleton is mapped to a range of bodies that closely match that skeleton structure), but even that's not perfect, and the variety in objects will greatly exceed variance in people. Google image search "cuddly toy" and the first few example bare little structural similarity.

Creating a static object would be a lot easier as you wouldn't have to worry about topology too much, as long as triangle count wasn't ridiculous. But that has limited uses.

Sometimes things just need to work good enough it doesn't have to be perfect in every aspect.
 
I think Shifty's main point is that there is currently no reason to suggest all games should use it, just that there are some that could benefit from it.
 
I think Shifty's main point is that there is currently no reason to suggest all games should use it, just that there are some that could benefit from it.

Not all games but it would be good to have it as a standard part of the SDK.
 
Actually I think you could import your face in the arcade version of Funky Head Boxers, so Sega was first (as always :) ).

God damn it, you're right tuna! Why did I have to doubt you, Sega.

You were first. You were always first. :(
 
Back
Top