[Lazy8s]

Joining that catapult game that was in development for Gizmondo which used augmented reality, a new, little demo showing a detailed character jogging smoothly in place has been made:

http://studierstube.icg.tu-graz.ac.a...r/gizmondo.php

[Guden Oden]

That character sort of looks like Zorro.

Weird that someone would actually use a gizmondo for something impressive. What framerate is it running at?

[Shifty Geezer]

25fps. Looks much smoother than the EyeToy monster demo from E3, and I guess shows you don't need stupid amount of processing power to do AR.

Edit : It's actually an AR engine, with Gizmondo just as an example of a handheld console running it. ARToolkit, their library (available free it seems) also runs on phones and PDAs.

[hidefguy]

Nice find, Lazy8s - just loaded it onto my gizmondo and it works really well as an AR demo.

[Skrying]

Originally Posted by hidefguy

Nice find, Lazy8s - just loaded it onto my gizmondo and it works really well as an AR demo.

Whoa, people actually have these? Lol.

[Kristof]

Originally Posted by Guden Oden

That character sort of looks like Zorro.

Isn't that a Quake MD2 model ?

K-

[hidefguy]

Originally Posted by Skrying

Whoa, people actually have these? Lol.

Something of a rarity, but I've had one on my shelf for quite a while, hoping to put it to some good use. Presumably there must be warehouses full of these devices somewhere, probably bought up from the liquidation sale. I tried quite hard both in person, at the Gizmondo E3 appearance the year before last, and by mail, to convince the gizmondo outfit of the benefits of opening up development on these devices, to no avail. I guess that they were hoping for PSP-scale sales and the same business model. It's really too bad, because Daniel Wagner's application shows just how powerful this device is.

[drgoldie]

hi this is Daniel, the author of the Giz demo...

i agree, the Gizmondo is a very powerful device.
i've rarely seen such a good camera and the 3D chip is blazin fast (although i don't like that it just has ~800KB texture memory).

we will maybe start some project using these devices...

Daniel

[Shifty Geezer]

Have you any insight into PSP and it's camera? Would you look to try to get your tech onto there too if not already making motions in that direction? And have you any plans for real applications for this tech or is it more a proof of concept, just to show it can be done?

[drgoldie]

i have no insight into any PSP things.
i would like to use it but as far as i know this is completely out of question - at least with the official setups. Sony is very, very strict about keeping the PSP a closed system. looking at what is going on in the PSP homebrew scene i even understand that a little bit. yet, i would have loved to see Sony actively support a homebrew scene instead of fighting it.

i didn't know there was a camera for the PSP. i'll have a look into how well it can be used using homebrew. maybe this is an interesting platform after all. i fear though that Sony would not like us to use it. does anybody have experiences with legal issues in that area?

about using the Gizmondo: this demo is ofcourse just to show that it can be done. we are always searching for new platforms to use for our research purposes and i believe the Gizmondo is definitely interesting. so yes, we do have plans, but this is all in a very early stage (might not go anywhere at all...)

bye,
Daniel

[Shifty Geezer]

The PSP camera isn't out yet; it's set to be released in September. I'm having difficulty thinking of applications for augmented reality as demo'd, cool as AR is!

Another point I notice is your system is far smoother than the PS3 implementation in Eye of Judgement. Do you have any theories as to why that is? Have you just developed a better, faster system? That game would definitely benefit from the smoother updates.

[drgoldie]

i do not know that game, can you post a link to a video showing this effect?

about applications for AR: there are many, starting in medicine and manufactoring over to games (which is what i am concentrating more on) and to advertisement. one of the main problems AR faces today is the lack of inexpensive, robust hardware. that's the reason why we are working on devices such as the Gizmondo or Smartphones. these are inexpensive and ideally people already own them...

bye,
Daniel

[drgoldie]

ok, i looked into the videos at IGN and i'm quite impressed. this is one of the first times that a major game developer makes a nice, high quality (AAA) game that uses AR technology.

yet, i'm not sure how this game would be set up. would players have the TV box next to the table (positioned like a third player) and look at the TV while they place the cards on the table?

about the tracking: it looks pretty smooth to me and it obviously does not require those black borders that are required for ARToolKit. but then ARToolKit is quite old by now and a PS3 has a magnitude (or two...) more power than a Gizmondo has...

Daniel

[Lazy8s]

Strategy-action gameplay and wireless-networking multiplayer were a part of Catapult for Gizmondo.

http://fireant.tv/directory/episode/1108716



AR Tennis is a research project using a customized ARToolKit for exploiting the advanced features of smartphones for multiplayer gameplay, translating the movement tracking of the phones into control over the tennis rackets and feeding back tactile response for ball contact to the player with the phones' vibration function.

http://www.hitlabnz.org/route.php?r=prj-view&prj_id=35

[Shifty Geezer]

That game certainly looks different. I don't think it's particularly suited to a mobile though. As a mobile is small and...mobile!, using a camera based based game on a static viewpoint doesn't make much sense versus same game on a static console setup. What would be better is something like 3D objects superimposed on the real-world view and you point and shoot with the mobile device. Perhaps in the city you have virtual ducks being overlaid, as as you move the deivce the camera tracks movement and you've got to shoot the ducks?

[Zaphod]

Originally Posted by Shifty Geezer

virtual ducks being overlaid, as as you move the deivce the camera tracks movement and you've got to shoot the ducks?

There are several games like that for Symbian smartphones, but thay haven't made much impact yet. Here's one. There are also other proof of concept apps around using the camera for interaction/AR. Here's another example with source available.

[Shifty Geezer]

Originally Posted by Zaphod

There are several games like that for Symbian smartphones, but thay haven't made much impact yet. Here's one.

I can understand why. Playing a mobile game at a restaurant table or busstop is ordinarily inconspicuous and I expect most would want to keep it that way, rather than waving their phones around!

There are also other proof of concept apps around using the camera for interaction/AR. Here's another example with source available.

Is the fidelity enough for the motion tracking ball game? I'd have thought a camera would give a pretty rough interpretation of subtle movements, and you'd need unrealistic degrees of movement versus the real physical game.

The non-gaming apps make more sense to me. An idea I had yonks back was virtual museum exhibits, where you stand inside a virtual structure and use your mobile device as a window to view it. Perhaps a large blue room that could fit lots of visitors in with various markers on the wall to track, and the software determines where you are looking and renders the ancient building in that view chromakeyed to the background. That'd be a really nice advance for museums and work very well with large screen portables. Something like PSP ought to be ideal at that.

I'll add that the ARinterior software seemed rather slow and primitive. Once they've got the camera lens matched, they should be able to draw 3D at full rate over the top. They woud benefit greatly from taking a mirror-ball photo to get a spherical illumination map. Then at least the furniture would match the lighting and fit in rather than looking like a composite with 1990's vector graphics!

[drgoldie]

there are several games (one of the first i know is the Siemens Mosquito Hunt) for "low-end" mobile phones that use very simple pixel flow detection for camera tracking. these games usually asume a fixed camera location and thereby translate pixel movement of the camera directly into rotational changes of the camera (simple concept and works great).

a main problem most of these tech demos have is that the performance is often not good enough for real games - especially for fast-paced games such as tennis or shooters.

that's why i like the Gizmondo quite a lot at the moment. this is the very first time i got or saw something like this to work at a really high-fidelity frame rate (constantly above 25 fps). i'm quite sure that the PSP could be a great device for AR games too...

Daniel

[Zaphod]

Originally Posted by Shifty Geezer

The non-gaming apps make more sense to me. An idea I had yonks back was virtual museum exhibits, where you stand inside a virtual structure and use your mobile device as a window to view it. Perhaps a large blue room that could fit lots of visitors in with various markers on the wall to track, and the software determines where you are looking and renders the ancient building in that view chromakeyed to the background. That'd be a really nice advance for museums and work very well with large screen portables. Something like PSP ought to be ideal at that.

Don't know about the fully virtual museum, but the virtual-assisted museum is certainly being experimented with. The advent of WIFI-enabled mobile devices with a camera and wireless audio will eventually see the replacement of information booths, cassette tours, and whatnot. Just take a picture of a symbol and have auxiliary audio, video, and AR content directly on the mobile. Market penetration for the devices isn't quite here yet, but it's not far off and such systems won't be all that expensive to implement.

[Graham]

Originally Posted by Lazy8s

AR Tennis is a research project using a customized ARToolKit for exploiting the advanced features of smartphones for multiplayer gameplay, translating the movement tracking of the phones into control over the tennis rackets and feeding back tactile response for ball contact to the player with the phones' vibration function.

http://www.hitlabnz.org/route.php?r=prj-view&prj_id=35

I had a play of this a while ago when it was still a reasearch project at the hitlab. It was interesting, but the hardware was definitely the weak point.
It was based on a modified version of AR Toolkit Plus from memory. ARTK+ is basically a modification of the standard ar toolkit, adding a few (desperately needed) features, and supporting fixed point processing.

From what I know, the AR-tennis project received an nvidia-sponsored grant to pursue commercial development and application.

Overall to be totally honest, the AR toolkit is pretty bad. The code is a shocker, and it's tracking performance can be down right terrible. It is difficult to setup as well. But at least they are finally investing some proper money and time into getting it into a more commercially sound state, however licensing the toolkit is still a struggle (to be kind).There are other alternatives out there. There is the MXR toolkit, which is *much* smarter than the ar toolkit, and has *far* better tracking performance, however it's code is probably even worse, is bloated, is no longer updated, is buggy, and has performance issues.
Even the BBC has a secret project to do their own tracking toolkit, but I don't know the details. The BBC actually used the AR toolkit in a medical themed television programme a while back.

I'll add that the ARinterior software seemed rather slow and primitive. Once they've got the camera lens matched, they should be able to draw 3D at full rate over the top. They woud benefit greatly from taking a mirror-ball photo to get a spherical illumination map. Then at least the furniture would match the lighting and fit in rather than looking like a composite with 1990's vector graphics!

There are real-time implementations of this (mirror ball for ambient lighting). They look very good too, however the camera calibration has to be absolutely perfect. The hitlab have yet to update their site with information on the project however. Here is a paper on the same subject.

The non-gaming apps make more sense to me. An idea I had yonks back was virtual museum exhibits, where you stand inside a virtual structure and use your mobile device as a window to view it. Perhaps a large blue room that could fit lots of visitors in with various markers on the wall to track, and the software determines where you are looking and renders the ancient building in that view chromakeyed to the background. That'd be a really nice advance for museums and work very well with large screen portables. Something like PSP ought to be ideal at that.

Done already a few times. With varying levels of success. example that is slightly different but still in a museum - here is a video.
I don't have any hand-held examples I can link to, but they are out there. Anything else gets into the 'too expensive' or 'too dangerous' or 'too unhygenic' department. So all the museum based ones are usually fixed.

The exhibits I've programmed have ended up being mostly single user activities, the exception is the AR-starwars project I did last year, which is on tour with the 'star wars: where imagination meets science' exhibit. (was in boston till a month or two back). - picture - thats not me btw. This project worked VERY well for user interaction.

[Zaphod]

Originally Posted by drgoldie

that's why i like the Gizmondo quite a lot at the moment. this is the very first time i got or saw something like this to work at a really high-fidelity frame rate (constantly above 25 fps).

Trouble with the Gizmondo is that it's going the way of the Dodo. Here's a video of similar stuff that you've shown running on last gen Series 60 (papers available as well). Certainly less impressive graphics wise than your demo (but if Nokia get their way) probably a more viable platform going forward.

Edit: Didn't notice that Lazy8s had already linked that. Silly me...

[Zaphod]

Originally Posted by Graham

It was based on a modified version of AR Toolkit Plus from memory. ARTK+ is basically a modification of the standard ar toolkit, adding a few (desperately needed) features, and supporting fixed point processing.

FWIW: Here's an earlier S60 port of the ARToolkit from the same guys. I think that is the first smartphone AR-implementaion I read about.

[hidefguy]

Originally Posted by Zaphod

Trouble with the Gizmondo is that it's going the way of the Dodo. ... Certainly less impressive graphics wise than your demo (but if Nokia get their way) probably a more viable platform going forward.

Yes, in fact it's already "gone", apart from Ebay and other private purchases, since Gizmondo Europe went bankrupt in January. I agree totally that currently, the best hardware bases are in Symbian - e.g. the N93, which we've got on order specifically for this purpose, but Windows mobile-based devices will be appearing within the next six months with more capable GPU's than the GF4500 in the Gizmondo.

[drgoldie]

Overall to be totally honest, the AR toolkit is pretty bad. The code is a shocker, and it's tracking performance can be down right terrible. It is difficult to setup as well. But at least they are finally investing some proper money and time into getting it into a more commercially sound state, however licensing the toolkit is still a struggle (to be kind).There are other alternatives out there. There is the MXR toolkit, which is *much* smarter than the ar toolkit, and has *far* better tracking performance, however it's code is probably even worse, is bloated, is no longer updated, is buggy, and has performance issues.

The ARToolKit source code is not beautiful. ARToolKitPlus is a little bit more cleaned up, but also not perfect. I don't know about the tracker of the MXR toolkit that much, but ARTag is to my knowledge the best marker tracking library at the moment (closed source and commercial though and no PDA/phone versions...).

FWIW: Here's an earlier S60 port of the ARToolkit from the same guys. I think that is the first smartphone AR-implementaion I read about.

The Hitlab's Symbian version of ARToolKit is basically just a very old version of ARToolKitPlus with Symbian specific modifications.

I agree totally that currently, the best hardware bases are in Symbian - e.g. the N93, which we've got on order specifically for this purpose, but Windows mobile-based devices will be appearing within the next six months with more capable GPU's than the GF4500 in the Gizmondo.

Can you back that up? Which Symbian hardware is better than the best current Windows Mobile hardware? I always had the impression that WinCE hardware was more powerful (which is also because WinCE wastes more resources than symbian...)

Which devices with new nVidia chipsets are gonna be released? I know only about the HTC Trilogy which will have a GoForce 5500. Any other?

bye,
Daniel

[Lazy8s]

As a starting reference, the Mobile Data Visualization site that's linked in hidefguy's signature provides an overview of many of the different 3D portable devices for the market.

http://mobile.sdsc.edu/devices.html