Photosynth / Seadragon (TED demo)

[[maven]]

Here's something that totally passed me by until now: A 7 minute video of a demonstration of Seadragon (essentially a clever multi-resolution imaging framework) and Photosynth (a computer vision algorithm that matches up photos of real-life objects, including orientation, using the Seadragon technology).

This is incredible technology, much more "relevant" than things like Surface IMO...


I wasn't sure what forum to put this in, so mods, feel free to recategorise the thread.

 

[DeanoC]

Its amazing... and its here right now

http://labs.live.com/photosynth

 

[AlNom]

Sounds pretty cool, but....

Like leaves on the wind,
This page has finished loading,
Yet nothing happens.

.........*scratches head*...................... Who wrote this?!


(I should probably try this on a computer that meets the requirements. )

 

[Davros]

Sounds pretty cool, but....

.........*scratches head*...................... Who wrote this?!

Bob Dylan wasnt it

 

[alexsok]

I just tried the demo and it's incredibly confusing and useless IMHO. Lots of black spaces, a bewildering lack of direction, etc.
Why isn't it seamless and not true 3D? I thought they actually came up with technology to take all of these pictures and create a true 3D environment which can be navigated. This is pretty lame, truth to be told...

 

[memberSince97]

Its amazing... and its here right now

http://labs.live.com/photosynth

That is nice stuff.

 

[Blazkowicz]

it's mayoe a nice toy but I'm not impressed, judging from the video it looks much like Nasa Worldwind or Google Earth software.

 

[Squeak]

The ZUI interface would be much better employed as a general way to navigate all information, not just images and text documents, together with pervasive general search. And is BTW, in case anyone should be in doubt, a rather old idea (70s AFAIK).
The seamless smooth zooming is just fancy MIP mapping (using waveforms instead of just downsampled pictures).

The 2d picture to full 3d model thingy can't possibly be as automatic as they allude too. That would mean that they have figured out a way to make computers understand "random" images, like humans do, which would be so big news that it would be more important than just about anything in computing.
There has to be someone, sitting down with the picture and laboriously tagging reference points (the more the better) so the computer has something to go by.
This "labour" means that, no matter how easy they might have made the program to use, it will never become a standard pervasive feature on Flickr or Google Image Search.

 

[[maven]]

Squeak, just because an idea has been around a long time, that doesn't mean that a decent implementation of the same is anywhere near as trivial as you allude to. The problem is not the display, but in the data and resource management.

As for the Photosynth stuff, I doubt it was tagged manually; that's what makes this such an interesting technology. What makes you think it wasn't automatic?

 

[DemoCoder]

There is no labor needed. The technology to do what PhotoSynth does has been available in the medical imaging community for a long time. Look at "Image Registration" algorithms, especially feature based image registration.


I'd guess that the 3D recreation can work like so using brute force:

1) download every photo of 'notre dame' you can fine. It's ok if a few bogus tagged ones are found

2) use image registration to compute, for each image, the images with the best registration characteristics (i.e can be fitted via feature extraction with fewest errors or warps)

3) continue putting 'puzzle pieces' together until no images left unregistered, or only images with very bad or impossible registrations

4) now, using EXIF data from the digital photos (focal length, size) as well as the warp transformations needed to in the registrations, one can compute the relative position of the camera for each photo.


Notice that they did not demo a real-time version of photosynth construction, my guess it, it is an offline process that takes a large computing cluster.


Finally, Seadragon is nothing more than tiled mip-mapped mosaics with smart prefetching. It looks cool and is very smooth, but it's all implementation.

PhotoSynth definately has some innovative in it from an algorithmic point of view.