Microsoft HoloLens [Virtual Reality, Augmented Reality, Holograms]

[dcbronco]

There are a ton of uses for this. Do it yourself would be huge. It could direct on how to repair complicated things like car engines or simple things like Ikea furniture by highlighting parts. Same for education with virtual models.

It would be huge in medical. 3D images based on MRIs could be used to plan surgery or consults with doctors in completely different countries. To assist paramedics with emergency field procedures with doctor instruction. Doctor shows paramedic first virtually then walks them through it.

Pedestrian GPS. Minority Report style advertising. Mystery style games where people or events take place virtually during your day. It would be kinda freaky but you're sitting at a table having lunch and someone(virtual) sits down and gives you information. It would be like that Michael Douglas movie The Game. Zombies. Haunted houses on Halloween. Survival horror games at home where sometimes things don't stay in the TV. That spider on your leg isn't real. The creature running at you runs right out of the screen. Museums where the exhibits come to life. A whole new level of 3D movie. Military. Pair with drones to mark targets for soldiers Ghost Recon style. I'm sure they can add Wi-Fi if it doesn't already have it. That's off the top of my head. I'm sure I'll come up with some more. Interior design.

 

[Silent_Buddha]

It could be a great competitive product, but right now it looks like a PR game.

There's a dozen serious companies working on AR, none of them want to unveil their product yet. I'm trying to see what's new here other than an arguably false impression that they are first with something described as magic, no less, because they unveiled an R&D prototype using vague BS wording. Until they describe this using plain scientific terms, I can only guess.

Tech for VR is relatively easy today, it's mostly combining an HMD with a Move-like 1:1 mapping. AR is a slightly bigger deal, it needs a Google-Glass-like semi-transparent display, an RGB or Kinect-like depth camera on the HMD, and processing using whatever compute resources is available,CPU, DSP, GPU, or even an ASIC. All of these components for VR and AR are becoming available, not because they were invented recently, but because the required sensors, accelerometers, gyros, displays, ToF camera, and compute power, are becoming less expensive, and now available outside of military/medical applications price points. The new thing happening in the AR field today is diffraction lenses becoming less expensive, which is what I'm guessing they are talking about. Maybe they have a novel production method for a computer-generated diffraction lens, or a generic diffraction pattern that can fit the purpose without needing a complex generated pattern (like maybe linear gratings? easy to mass produce?). Canon went the complex route for their compact telephoto lenses, and it's currently too expensive to produce. Bluray/DVD drives have been using holographic lenses for years but it's like a two milimeters lens, nothing like what is needed here.

The wild card I think is patent warfare.

It's a proto-type definitely. But one that is going to see real world use by this summer. NASA will be using it extensively if what Microsoft and some people from NASA said are true.

As to who was first. The idea for this was pitched to Steve Ballmer 7 years ago. Serious work started 5 years ago. It is entirely likely that no-one else in the industry is as close as Microsoft is to an actual product. It's also possible that no-one else even has a proto-type with the ability to showcase something remotely like what MS has shown.

The only other major player we've heard of, only kickstarted back in 2013. Less than 2 years ago. It's quite likely they will be lagging behind MS quite significantly.

As to what is required for this? You could say the same thing about Kinect and Kinect 2. Yet both were revolutionary to the industry and significantly ahead of all the competition. If not in hardware then in the actual application of everything it does. Skeletal tracking, etc.

So, this may be similar here. Other than the HPU, presumably all the competitors have access to the same hardware. But it's on the software and implementation side that MS likely have a significant lead on anyone else attempting to do something quite so comprehensive.

One advantage for competitors, however. Windows 10 will support holographic worlds and interactions out of the box. Meaning you could in theory use any hardware as long as it could run Windows 10 and interface with whatever APIs are needed.

So, yes, MS are likely pushing the implementation of this far more than any other player in the industry at the moment.

Regards,
SB

 

[Silent_Buddha]

This isn't going to be any more successful than the Kinect. Lets say it adds just $100 to the console. It'll hobble sales just as the $500 SKU of the X1 with Kinect was hobbled.

It looks impressive in the videos but after the novelty wears off, it's unlikely to become a part of people's daily routine. They showed a 3D or holo version of a weather forecast chart. Are you going to dig up that headset and do whatever gestures are needed to bring up a weather forecast that is rendered holographically?

No most people will just pull out their phones and swipe once or twice or just ask Siri or Google Now or Cortana.


The practical applications, like how-to videos or tutorials showing overlays over your plumbing or whatever will only work if third-parties develop compatible content. That could be about as likely as the Kinect games with innovative gameplay mechanics.

This isn't targeted at gaming.

This is targeted at replacing personal computers in the home and workplace.

Price won't be an issue for the workplace, and for home computing, the price ceiling for most families is significantly higher than it is for console gaming.

Regards,
SB

 

[wco81]

ASPs of PCs are like $500-600.

Anyways I don't see people putting on some head gear to read emails or do browsing.

 

[Deleted member 11852]

Minority Report style interfaces would be nice with this except no need for display screens.

Instead of one screen that everybody in the room can see, everybody in the room needs a visor. Frank forget to bring his his and Bob has about 5 minutes of charge left and everybody still has wet hair.

It's got some killer applications for sure but it's an inclusive medium - like VR.

 

[tritosine5G]

As to who was first.

http://graphics.tudelft.nl/~wouter/curriculum.html
(?)

From 1997 till October 2001 I worked on the Ubicom project, a project working on the design and implementation of a wireless augmented reality system. My research focused on low-power, low-latency rendering using a distributed rendering system. Latencies down to 8.5ms have been realized with our system.

 

[Arwin]

I just think this thing is cool. Other than that, I expect it will be a ways off before we'll see it also be useful to even 1% of the population (meaning one in 100 has one of them at home), but it will find some cool uses soon enough. I don't think it will find uses as soon as Oculus Rift and other full VR systems do though. I was in a board meeting yesterday for a daycare center and of the four ladies present, two had tried VR the last year. It's a big thing in the area of fear treatment, among other things, and they were particularly stunned by an application where cameras had just recorded a 360 degree field of view in 3D, of standing in the middle of a crowded square, and someone even coming up to ask the time. No computer generated graphics, but real footage instead. If that works well, then imagine what the next Avatar (just an example, pron could be cool too ) could be like, even apart from when graphics cards do have the power to do all that in realtime.

I'm ready for this future, and I'm liking it. For Microsofts project, if Windows X supports it well and development is easy, that's a huge thing in and of itself and I'm a big fan.

 

[Shifty Geezer]

Not necessarily.

Using the Mars Rover example. They are able to have the Mars Landscape occlude everything in the room, except for a computer terminal they had setup in the room. In other words, not something you could do with a VR HMD.

You can with cameras on the VR headset to composite the real world.

 

[mosen]

Meta just kickstarted last year 2013. So if MS bought their patents, Meta would have to pay royalties to MS to develop and sell the Meta SDK, which if I understand correctly they are still selling today. That doesn't make a lot of sense to me. The royalties from Meta would never amount to anything for MS.

edit: Oh right. Well clearly I got my dates wrong.

The purchase of patents basically acquired this company
Osterhout Design Group.
http://www.osterhoutgroup.com/products

Their website basically was cleaned out since the acquisition.

This product looks familiar
http://gizmodo.com/these-augmented-reality-glasses-are-james-bond-worthy-1677094990

They bought their patents, not their company. Their website is up and running well, they have a product named "R-6 Glasses Kit" for sell (5000$) and a new product (recently introduced in CES 2015) for 2015 and their target is to price it lower than 1000$ (see here).

According to this report Microsoft problem was lack of patents, not the technology or implementation:

We assume that means Project Fortaleza is put on hold, instead of gone for good. The reason, according to the source, has entirely nothing to do with technology implementation. Microsoft "has no problem at all in the development of the device," however, a fair portion of the patented technologies involved do not belong to Microsoft itself. It's the necessary licensing reportedly that's keeping Fortaleza from consumers. Apparently, Microsoft has licensed some of the for the second generation of the Kinect sensor, and is not allowed to use them on anything else.

http://www.windowscentral.com/rumor-microsoft-puts-project-fortaleza-augmented-reality-glasses-hold

 

[pjbliverpool]

I don't think the VR vs AR argument is something we'll need to worry about for too long. Clearly both are just two sides of the same basic technology and I see it as inevitable (likely wthin 10 years) that there will ultimately be headsets that can provide both full VR or AR as the user desires. On top of that is the general trend of miniaturization which will ultimately see such a device (again within 10 years) being little bigger than a regular pair of sun glasses. Add to that very quickly developing technologies of natural voice recognition and hand gesture control, and combines with eye tracking I am reasonably confident that within 10 year, glasses of this sort will have replaced smart phones completely.

Imagine, if they are not much bigger that a normal pair of raybands, cover your full field of vision, Wifi/mobile network enabled, can switch between AR/VR/Normal vision seamlessly and are fully controllable through voice, eye tracking and gesture then they would make todays best smart phones looke utterly archaic by comparison and almost completely redundant - and all that tech is easily within our grasp within a decade. Retina scan security (your glasses only work on your head) would also be a pretty awesome and obvious progression.

 

[London Geezer]

So the way I understand, the opacity of the 'holographic' images is such that you can't see anything behind it?

What's stopping them from using this as an Oculus Rift kind of device, then?

If someone can built an entire scene which completely blocks your room, then it's VR. But I'm not sure the visor has 100% opacity, some light must come through - say, a bright light in the room- so the images would be 'mixed' with the background.

 

[wco81]

So 10 years from now people will be wearing HMDs of some kind, which cover up their eyes from being visible to others, instead of carrying phones?

I don't think so.

Even the little bluetooth headsets were obtrusive and had limited penetration. People wearing them were mocked. If anything, the use of bluetooth headsets seem to be in decline.

Look at how people wearing Google Goggles, which has very limited distribution, have been hassled, with some places banning them.

 

[Shifty Geezer]

I don't think the VR vs AR argument is something we'll need to worry about for too long.

I agree. It's easy enough to put a blanking plate behind AR glasses to make them VR, or add cameras to VR to enable AR. However, MS's implementation doesn't provide the peripheral FOV as I understand it. Everyone's mentioned it's a small portal. So for the VR experience, you can't just occlude the background with Hololens (in its current version) to get peripheral vision. Whereas you can with a VR headset. So for total immersion, VR is presently the better option.

Actually there's a very important plot point here. MS has actually got working eyetracking in a headset. This means we have a prototype for foveated rendering which is exactly what VR needs to ease the considerable rendering burden. If MS included that in an Oculus style headset, they could get high quality VR on XB1 despite its low performance. The highest quality rendering would be a little window, maybe 400x400 pixels. The rest, even a 4k screen, can be rendered in low res, lower quality, and upscaled to fill. Whoever owns the patent on this idea/tech should dominate VR.

http://www.roadtovr.com/fove-eye-tracking-vr-headset-hands-on-ces-2015/

Perhaps no-one owns any blocking patents and it'll be a free-for-all? This is very hopeful for VR on consoles though. (And AR, for those still hankering after playing games in their bedroom instead of on other planets and jungles and giant candy worlds. )

 

[fehu]

The big bang theory is still running? Because I want an Hololens episode

 

[pjbliverpool]

So 10 years from now people will be wearing HMDs of some kind, which cover up their eyes from being visible to others, instead of carrying phones?

I'm not sure the visor/lens would have to be opaque but even if it is, is that particularly different to wearing sunglasses? Perhaps you're right though and for indoor use something like this would be combined with a smart phone or watch which provides a cut down experience. At least in the short term anyway while people acclimatize to the idea of wearing smart glasses and it becomes the norm.

Even the little bluetooth headsets were obtrusive and had limited penetration. People wearing them were mocked. If anything, the use of bluetooth headsets seem to be in decline.

Look at how people wearing Google Goggles, which has very limited distribution, have been hassled, with some places banning them.

I'v not really seen that with blutooth headsets. I thought they were largely accepted now. At least here in the UK. As for google glass, I grant that but tat's clearly a very early prototype of what's to come and doesn; preclude the commercialisation of true smart glasses once the incentives to use them are stronger and technology allows the aesthetics to fall closer to nomal fashions. See smart watches for a clear example of that.

Ijust think there are far too many extremely enticing commercial applications for this technology for it not to cause a technical revolution in the same way smart phones have. You an basically do everything with this that you can with a smart phone in a far more effective and functional way - once people start build apps for it of course.

Just take a few obvious exampls of what you currently do with your phone:

#Calls - these would have all the benefits of a voice activiated bluetooth headset without the need to be tethered to a phone and with the added benefit of including a visual element that floats to the side of your vision (as opposed to having to stare down at your phone for current mobile skype calls)
#Photos/videos - look and blink, need I say more?
#Music - no more earphones required, their already right here built into the glasses. And the addd bonus of a music video floating to he side of your vision if yo like as well.
#Satnav/maps/GPS - I don't think I can overstate the usefulness of having a GTA style floating map in the corner of your vision while trying to navigate or even havng the route itself highlighted directly on the road/path in front of you
#emails/txts - pop up in front of you as they come in (or by voice command) and then respond directly through voice as well
#Mobile entertainment - The abiluty to project a cinema sized 3d movie screen directly in front of you whenever and wherever you are to say nothin of switcing over to full VR gaming certainly has it's advantages!

All of the above has enormous and very wide ranging commerical appeal and that's just what I pulled off the top of my had in 5 minutes. I really don't see how this tech can't pull off a technical revolution once it becomes sophisticated and mobile enough. The 10 year timeframe of that is just my guess.

 

[pjbliverpool]

The big bang theory is still running? Because I want an Hololens episode

Give it a couple of weeks and I'm sure there'll be a South Park episode....

 

[Rodéric]

I like that, I was already interested in CastAR, that's definetly something I'd consider buying !

 

[Jwm]

So the way I understand, the opacity of the 'holographic' images is such that you can't see anything behind it?

What's stopping them from using this as an Oculus Rift kind of device, then?

If someone can built an entire scene which completely blocks your room, then it's VR. But I'm not sure the visor has 100% opacity, some light must come through - say, a bright light in the room- so the images would be 'mixed' with the background.

Seen mixed messaging from the press on this. Some say they can see objects through the hologram, others tend to not notice it. On the photos around it maybe seems it can be a design choice in setting the opacity levels?

This shot was a capture from the unit the lady was wearing, in which we were told it was still a feed from her device.

If you watch the trailer though, which I am sure is all concept or it could have been that camera rig, everything seemed to be slightly transparent.

Edit: If you look at the bottom row of icons, you can see the wood trim running across, but really they do look pretty solid.

 

[Shifty Geezer]

Even when opaque, her finger can be seen behind the hologram. I think it's basically the same looking at a window reflection from the inside. During the day, you don't see your reflection because the brightness is very low against the daytime illumination; at night, you do see your reflection. The darkened visor reduces brightness of the surrounds allowing the brighter projection to take prominence. I'm pretty sure a bright light behind the visor will bust through the projection.

 

[Jwm]

Even when opaque, her finger can be seen behind the hologram. I think it's basically the same looking at a window reflection from the inside. During the day, you don't see your reflection because the brightness is very low against the daytime illumination; at night, you do see your reflection. The darkened visor reduces brightness of the surrounds allowing the brighter projection to take prominence. I'm pretty sure a bright light behind the visor will bust through the projection.

That was my thought when I saw how they tinted the visor. More details at //Build according to the FAQ -