MS Research: IllumiRoom *spin*

So you'll need to factor in higher framerate 3D projectors. For gamers and immersion, 3D is going a be a huge plus.

Really? That doesn't seem to be the case, but probably a different discussion.

That makes no sense to me. You're talking about cave as a commercial venture, for a professional userbase. Why will they want to use four-wall projection in a cluttered room instead of having a room for the job? That can be done without any need for Illumiroom. Illumiroom is attempting to filter out the imposition of furniture, or use it to artistic effect, and in that environment four wall projection makes no sense.

Which isn't what cave projections are about. They splat the same resolution everywhere and waste all that rendering behind you and to the side of you. An HMD only concerns itself with where you are looking. I even expect eye-tracking and foveal rendering to be easy to incorporate into an HMD, which'll be much harder to manage in a multi-projector setup.

So in summary, in the consumer space HMDs are clearly the way forward for full 360 VR immersion. In the commercial space, there'll be different pros and cons for different systems, but Illumiroom isn't going to help really. It could be used for some niche purposes, but it provides nothing for 360 degree imaging where you'd clearly set aside a blank room for that. Illumiroom is something else that may have it's time in the sun but is destined to be phased out, unless people decide the fashion with video interacting with furniture is the future.

Being a console forum I was talking about consumer implementations. I am not saying IllumiRoom is a replica of a cave, but follows it's principles. It DOES allow for cheaper caves however as it brings consumer level tech into this area.

From a commercial perspective there are plenty of applications for something that works in a cluttered room. There are many more simulator users than there are users with dedicated clean rooms.

We sell a flight simulator that splits into three sections so it can be wheeled into storage which with many of our clients is just the side of the room. The space is needed as many sim labs are shared spaces.
 
Maybe the Display Planes hardware in Durango could help minimize the (performance) impact and ease the implementation? (i.e. a very low res "illumiroom" layer separate from the main game layer, rather than the console needing to render at a native res that it isn't equipped to handle...)

Looking at the video, for the Focus+Context Full mode:
http://youtu.be/nGGMv9RnJIA?t=1m15s

I don't think you'd get away with a resolution much below 720p, don't forget it's blown up to cover the entire wall (around 8x more area than the main display) so it can't be too low res.

For example if that 720p framebuffer covered a rectangle 300 cm (120") wide by 175 cm (70") high - which is similar in size to what is depicted - the DPI would only be about 10 pixels per inch, which is probably as low as you'd want.

Even if you used upscaling to scale the projector output, you'd still want the source framebuffer to be at least 480p (7 ppi at the dimensions above).
That still requires rendering 20% extra pixels over your 1080p main framebuffer, not a trivial thing to do (especially when you only have 16 ROPS).
 
Looking at the video, for the Focus+Context Full mode:
http://youtu.be/nGGMv9RnJIA?t=1m15s

I don't think you'd get away with a resolution much below 720p, don't forget it's blown up to cover the entire wall (around 8x more area than the main display) so it can't be too low res.

For example if that 720p framebuffer covered a rectangle 300 cm (120") wide by 175 cm (70") high - which is similar in size to what is depicted - the DPI would only be about 10 pixels per inch, which is probably as low as you'd want.

Even if you used upscaling to scale the projector output, you'd still want the source framebuffer to be at least 480p (7 ppi at the dimensions above).
That still requires rendering 20% extra pixels over your 1080p main framebuffer, not a trivial thing to do (especially when you only have 16 ROPS).
It could easily be lower than 720p. I've projected 480p at sizes that large and it was fine, you can go even lower with this since it is really just there for your peripheral vision.
 
The example in the vid seems to project at a reasonable resolution, say 480p.

So maybe 480p is all you need, but even that is 20% more pixels to render if you have a 1080p game.
 
The example in the vid seems to project at a reasonable resolution, say 480p.

So maybe 480p is all you need, but even that is 20% more pixels to render if you have a 1080p game.

Sure but they don't have to be 20% more high quality pixels, I wouldn't bother with anything other than basic shaders given what you're trying to accomplish, so probably less than 20% overhead in real terms.
Which doesn't seem unreasonable to me.

Of course it still requires a rear mounted camera to auto calibrate and a wide FOV projector with a decent lumen output, which doesn't scream cheap to me.
 
You are more likely to see HMD glasses facillitate an illumiroom like feature. HMD like a google glass derivative has implication across a broader set of utilities.

HMD has to the opportunity to exist as a peripheral that is serviced and services multiple devices where the device itself can drive the computional power needed by the functionality of the glasses. We might see a future where HMD can be driven by your PC, tab, console or smartphone depending on the circumstances.

HMD will encourage a greater level of investment from a wide variety of sources because potential for a very large userbase driven by a broad feature set.
 
Last edited by a moderator:
I can see Illumiroom being really compelling for learning software. A six year old learning about dinosaurs and just being mesmerized. Way more practicle than HMD.
 
I can see Illumiroom being really compelling for learning software. A six year old learning about dinosaurs and just being mesmerized. Way more practicle than HMD.

I can see far more potential than that. The potential to bring the in game world alive and have it appear to be all around you. There doesn't need to be much detail outside of your TV screen in order for this to happen.

Stare at the middle of your TV screen and without taking your eyes off the center, attempt to read something outside of the TV. It's generally not possible. Doing that with my LG TV, I can't even read the LG logo on the bottom of the TV, it's just 2 indistinct white blobs on a black background. Things outside the screen devolved into generalized fuzzy/blurry shapes. Color definition also takes a hit. And that's with 20/20 vision.

There has been extensive research done that basically shows that 20/20 and 20/10 vision isn't possible outside of a very narrow visual FOV, hence why your eyes are constantly in motion when reading or trying to discern detail in objects.

Hence, as long as all of the detailed action is happening on the TV screen, all that is required outside of the TV screen are general "shapes" to represent an extension of the world being portrayed. In fact, having more detailed images outside of the screen could be detrimental as it offers detail that could draw they eye to it, when that is the last thing you want to happen.

Now, as mentioned it isn't perfect. In a racing game you might want to actually look to the sides to try to see if there's a car to your side before turning in that direction.

HMD's with head tracking will be superior for something like that. It allows you to move your head and receive a different perspective based on where your head is turned. HMD's without head tracking won't be as you'd still have the same static POV and the same inability to determine detail in your peripheral vision.

But that's the thing. Something like Illumiroom is trying to expand the world presented on a Television screen at a relatively cheap price both computationally (rendering, etc.) and with regards to hardware.

Just like 3D-TV's failed at least somewhat due to the requirement of having to wear glasses, HMD's also have limited appeal for general use due to the requirement of having a piece of hardware mounted to your head.

Illumiroom is aiming for a more mass market appeal than is possible with a HMD. The challenge for Microsoft is whether they can implement it at a mass market price.

Regards,
SB
 
I can see Illumiroom being really compelling for learning software. A six year old learning about dinosaurs and just being mesmerized.
How? They'll see the dinosaur on the TV and the forest on the wall behind it? Or the dinosaur will fill the wall - which is no different to using a projector.

Hence, as long as all of the detailed action is happening on the TV screen, all that is required outside of the TV screen are general "shapes" to represent an extension of the world being portrayed. In fact, having more detailed images outside of the screen could be detrimental as it offers detail that could draw they eye to it, when that is the last thing you want to happen.
But that means the experience is little more than the TV. The detail and interaction is in the TV. If the outside is just a bit of stuff, like gunshot or foliage, it'll look nicer but you'll still be tied to the TV. What you won't be able to do is see something moving in your periphery and look at it - you'll just see the low-res projection. You won't be able to see a dinosaur's foot and look all the way up to see the dinosaur towering above you; certainly not with the clarity of an HMD. You'll instead see the dinosaur perspective corrected onto your ceiling with a big crease across the middle where the wall joins. Or you'll top out at the top of your wall and not be able to look all the way up.

Now, as mentioned it isn't perfect. In a racing game you might want to actually look to the sides to try to see if there's a car to your side before turning in that direction.
That's going to be true of all environment immersion unless the periphery doesn't contain anything of importance, at which point it's just a nice aesthetic.

Just like 3D-TV's failed at least somewhat due to the requirement of having to wear glasses, HMD's also have limited appeal for general use due to the requirement of having a piece of hardware mounted to your head.
I agree with that, but I don't see Illumiroom as any alternative. It's a different experience, both niche (until HMDs become super-lightweight as RnD concepts suggest). And it's a worse experience IMO than just playing a game projected on a clear wall. If I had a projector (and I do!) that's what I'd be doing with it, not limiting my game to a small spot in the middle and projecting ambience around it.

Think of it this way - would you prefer to go to the cinema or set up a home cinema, and see a honking great screen, or would you rather see a small, brighter screen in the centre with the rest ambient projection? I'd definitely choose the former, and that's accomplished with just the projector, so is slightly cheaper than Illumiroom. What Illumiroom may do is make a cheap projector that's unsuitable for replacing the TV into a useful addition adding fill and offering something between a TV and a projector at a lower price. If it garners any userbase, we may see some cool uses like prompts on the floor or huds on the outside of the screen and things, and ambient effects for the fun of it.
 
But that means the experience is little more than the TV. The detail and interaction is in the TV. If the outside is just a bit of stuff, like gunshot or foliage, it'll look nicer but you'll still be tied to the TV. What you won't be able to do is see something moving in your periphery and look at it - you'll just see the low-res projection. You won't be able to see a dinosaur's foot and look all the way up to see the dinosaur towering above you; certainly not with the clarity of an HMD. You'll instead see the dinosaur perspective corrected onto your ceiling with a big crease across the middle where the wall joins. Or you'll top out at the top of your wall and not be able to look all the way up.

That's going to be true of all environment immersion unless the periphery doesn't contain anything of importance, at which point it's just a nice aesthetic.

Yes, you have to think about it not as to whether it affects the actual gameplay or not, but more as in another level of immersion.

So, instead of playing the game and always being aware that you are in a room playing a game on a TV, the illusion may be enough to get you to think you're actually in a cave, building, racetrack, flying in the skies, whatever.

As said, it isn't perfect, you can't turn your head to look as the illusion then falls apart somewhat, but you can still change your view perspective as you normally would in games via the controller.

Additionally it can potentially be far more immersive than even something like the Occulus rift which has a wider FOV but still doesn't fill your entire FOV. Illumiroom has the potential to fill your entire FOV with stuff for your peripheral vision.

As said, it's more about making things more immersive to mainstream audiences. The big question, and thus the big challenge to Microsoft, is whether it can be implemented at mainstream prices. IMO, it can't cost more than the original Kinect (150 USD), and even that would possibly be viewed by many as too much. Although if a person experiences it in person they may change their mind. However, demoing it in a retail store would be difficult. :)

For myself, I love surround gaming due to the immersion that is added just by having the game world exist in your peripheral vision if you don't look directly at it. You can still notice it and react by moving your POV via traditional methods. But for me, I can't quite justify the price of another 2x 30" monitors or a new set of 3x24" monitors dedicated just for that. My work machine doubles as my gaming machine so 3x24" monitors wouldn't suffice for that purpose. I'd love if AMD or Nvidia eventually supports PLP then I could pair 2x20" monitors in portrait which I could justify as they'd take the place of my one 24" monitor in portrait that I currently have as a secondary display.

Anyway, long story short. I could see something giving me a similar experience to widescreen gaming (world exists in peripheral vision) at a much lower cost. Another benefit is that it would extend the world in my peripheral vision not only horizontally but vertically as well.

Also if used in a similar way to that one device that someone linked for movie watching (which only changes the color of lighting behind the TV screen) then it also helps with more immersive movie watching while also addressing the dangers to a person's eyes from watching a Movie or TV show in a dark room without any lighting other than the TV screen. Of course then you have problems with what do you do with black bars above and below moves that are wider than 16:9. :D I guess just the top and bottom remain unlit and the light extends sideways?

Regards,
SB
 
How? They'll see the dinosaur on the TV and the forest on the wall behind it? Or the dinosaur will fill the wall - which is no different to using a projector.

The parent next to the child can encourage exploration by asking whats that off to the right and Kinect is tracking the child.
 
Yes, you have to think about it not as to whether it affects the actual gameplay or not, but more as in another level of immersion.

So, instead of playing the game and always being aware that you are in a room playing a game on a TV, the illusion may be enough to get you to think you're actually in a cave, building, racetrack, flying in the skies, whatever.

I think it would be great to do this but I wonder how could it work practically, at least to make a great looking effect. I have seen great examples of wall projections that are "picture perfect" but they were tuned perfectly to fit a wall or room to put the projection feat to great effect. Right now I'm just doubting a consumer product can become a "one size fits all" solution to this and really make it look good.

Also how is all of this additional rendering going to happen outside of the TV screen? The examples I've seen where it looks really great is when the projector graphics match the TV graphics in quality essentially. How does this fit into the hardware rendering I don't quite understand yet.
 
The parent next to the child can encourage exploration by asking whats that off to the right and Kinect is tracking the child.
Kinect in this case isn't tracking the child (if youre child was in Kinect's view, they'd be casting a big shadow on the wall!). You'd need a second Kinect on the console. If you simply replace your TV with a projector on a blank wall and use Kinect, you'll have that full big-screen experience.

What could be awesome is something like EyePets that interacts with your furniture, like looking for fairies hiding around the room, although of course you couldn't get close as that'd occlude the projection, and it'd be limited by whatever furniture you have. But interactions with the room would be the most magical contribution this tech can make IMO. I still see that as remarkably niche though.
 
It's hardly an equivalent technology as it's not doing anything based on room analysis. It's just a secondary environment film, and then a waste of some decent projector space by tying the viewer to a small screen and filling the rest up with low-res filler. I point to my previous cinema analogy and suggest that anyone with a (decent) projector in their room would rather just have a big screen.
 
This has really surprised me, I didn't expect some of the stuff you can see in the video.

This astonishing new video about Ilumiroom shows the possibilities of the technology. :oops:

 
This has really surprised me, I didn't expect some of the stuff you can see in the video.

This astonishing new video about Ilumiroom shows the possibilities of the technology. :oops:


That actually looks a lot more awesome than what I'd seen a long time ago. A few problems. You need Kinect facing your tv. You need a projector. And you're rendering a view that's much larger than your tv. I don't see how any of that could work with Xbox One. To use Kinect you'd need a Kinect facing you and a Kinect facing your tv, as well as a secondary HDMI output to hook up to the projector.
 
That actually looks a lot more awesome than what I'd seen a long time ago. A few problems. You need Kinect facing your tv. You need a projector. And you're rendering a view that's much larger than your tv. I don't see how any of that could work with Xbox One. To use Kinect you'd need a Kinect facing you and a Kinect facing your tv, as well as a secondary HDMI output to hook up to the projector.

If they bring it to market I'm sure they would create an integrated Kinect/projector device that's separate from the Kinect you already have.

Tommy McClain
 
Back
Top