HTML5 on consoles

[jeff_rigby]

D-Bus Will the next desktop be the Browser?

While many people envision that the next desktop will be the browser, many more do use Web applications almost exclusively, already today. The traditional separation between Web and Desktop app development is blurring. Browsers have become powerful platforms for running complex applications, and this situation is speeding up with the broad and increasing adoption of HTML5 standards by major browsers.

On the other hand, we have D-Bus, a freedesktop.org standard that is at the core of almost every GNU/Linux system out there. It is the de-facto IPC mechanism on which your applications talk and share. D-Bus allows us to write a program in any language, and export its usefulness over a standard channel. Also allows us to write differentiated UIs (e.g, Qt vs. GTK+ vs. NCurses) to interface a common functionality. Yes, one bus to bind them all!

Joining these two pieces together is just the next logical step. A step towards bringing together the best of two contexts: the ubiquity of the Web and the inter-process collaborative nature of the Desktop.

We need to write applications that you can host and use securely and reliably not only in your computer, but anywhere in the Planet where you happen to have a browser plugged into the Net; whether it is your laptop, mobile phone, tablet or your neighbors’ PS3. We also need to encourage application developers to export the logic of their programs over D-Bus, to allow other platforms (like the Web!) to reuse it. Telepathy is a good example of such program.

Almost identical applications in terms of functionality are written for FOSS environments like GNOME, KDE, MeeGo, Android, etc; yet many times only the user experience and the technologies used to build it are different. I think there is room for a wider code-reusing culture if we come back to the original Unix philosophy:

Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

This article grabbed my attention not because it mentions the PS3, D-Bus and Telepathy which are Collabora projects and I believe Collabora is working with Sony in some way, but because it mentions; "the next desktop will be the browser".

I believe Cairo is being used for the PS3 and NGP for their desktop as well as to support webkit. Cairo can be used to support a desktop that is a browser or has a browser like capability.

Patsu and I speculated on HTML5 UI subsections in the XMB and I and others on HTML5 widgets. A better choice is SVG using Cairo and possibly bypassing the HTML5 javascript engine. Clutter support MAY be in the webkit port and on the Desktop but there is no way to confirm this.

The logic of this clicks in my mind. The uses for this in an on-line world beg for some speculated uses in a thread on this forum.

Would such a scheme result in a smaller XMB that is at the same time more powerful? Is a subset of this idea already being used for the PS3 XMB with the full implementation in Firmware 4.0? Anyone else think this is coming?

Spawned to: Will the PS3 have an XMB desktop that has Browser abilities

 

[jeff_rigby]

IBM article on Cairo cross platform Graphics

Summary: Built from the ground up to create identical output on both printer and screen—all in a cross-platform way—cairo is becoming a huge player in the Linux® graphics space. Harness the same 2D power used by GNOME, GTK+, Pango, and many others. Also in CE and Embedded linux platforms.

A significant design decision in cairo is to support nearly identical output to the greatest extent possible. This consistent output lends itself exceptionally well for GUI toolkit programming, or cross-platform application development. The ability to print a screen at high resolution, and draw on the screen contents with the same drawing library has obvious advantages.

An added benefit to the vector nature of cairo drawing is that vector images tend to be smaller in size. This is because a relatively large amount of information can be encoded in a relatively small equation. The beauty of vector drawing is that the drawing tends to be relatively straightforward. The onus of actually converting the points, lines, and their associated equations into something you can see rests on the drawing library.

As mentioned previously, several graphics toolkits provide bindings to make cairo development even easier. Gtk+ versions newer than 2.8 contain full support for cairo, and cairo has been selected as the strategic drawing system to support future GTK releases. Additionally, toolkits like GNUstep and FLTK are beginning to support cairo for their graphics rendering needs.

Cairo makes perfect sense to select as your drawing API if you plan on doing anything cross-platform that requires low-level control of drawing operations and compositing. And if you want to have the cross-platform capabilities but do not want to draw at a low level, there are some other convenience drawing libraries that sit on top of cairo. Cairo-Clutter

Applications of cairo in the wild
A large number of influential open source projects have jumped on the cairo bandwagon, and cairo has positioned itself to be a huge player in the Linux graphics space. Some of the more influential projects that are already embracing cairo are:

Gtk+, everyone's favorite cross platform graphics toolkit
Pango, a Free Software library for laying out and rendering text, with emphasis on internationalization
Gnome, a Free Software desktop environment
Mozilla, cross-platform Web browser infrastructure on which Firefox is based
OpenOffice.org, a free software office suite comparable to Microsoft Office

The backend for Cairo can be built on OpenGL or custom supported by Sony with their PSGL. Backends supported include Linux, BSDs, Microsoft® Windows®, and OSX (a BeOS and OS2 backend are also being developed). Linux drawing can be accomplished via the X Window system, Quartz, image buffer formats, or OpenGL contexts.

Cairo can be ported to other platforms and then provides support for cross platform application GUIs and possibly some games. Perhaps the biggest use for Cairo is in Webkit rendering on multiple platforms.

For Firefox which uses/depends on Cairo for it's GUI (graphics) is able to port it'self to different platforms because Cairo backend libraries exist for those platforms. For AV support Firefox uses Gstreamer which is also cross platform.

Again Sony is going to, or is using Cairo and Gstreamer in their multi-platform ecosystem, it's in the Sony Snap Developers Site and the PS3 webkit port is going to be based on Cairo.

Should spawn to the PS Suite thread: PS Suite message 131

 

[whome0]

I don't know if you have seen this Netflix+Nokia presentation but here is a snippet. I don't have an online link of pdf file.

Qt in home media –Netflix case
"As we developed the Netflix Player by Roku, we were keenly focused on creating the best possible user experience for Netflix's eight million-plus customers. Qt provided us with the tools to implement our elegant and powerful user interface quickly, providing an important time-to-market advantage”
Tim Twedahl, vice president of consumer products, Roku, Inc

Prepares a client player for integration by TV, set-top box manufacturers. Flash user, but needed richer, server-driven UI for its TV-based client.

Problems solved with Qt
• HTML5 alone, even Adobe Flash inadequate for performant, flexible, rich UX
• Invented Qt/Webkit hybrid development pattern (web+native)
• Enables rich UI/performance of native with web flexibility
• Solves hardware resource problem –easier to ship
• Services easier to deploy on new devices –thanks to Qt

Keywords: Nokia Qt, QtQuick/QML, QtWebkit

Its not always a pure html5 seen as a complete solution, Qt sdk provides an excellent cross-platform toolkit. It was about to become a major player in mobile phones until Nokia ditched it and joined Windows Phone train. Time will tell what happens to Qt mobile, but its becoming a very strong player in PC and set-top box devices.

edit: VV points to jeff_rigby post below, its Qt library. QT stands for Apple QuickTime.

 

[jeff_rigby]

I don't know if you have seen this Netflix+Nokia presentation but here is a snippet. I don't have an online link of pdf file.

Its not always a pure html5 seen as a complete solution, Qt sdk provides an excellent cross-platform toolkit. It was about to become a major player in mobile phones until Nokia ditched it and joined Windows Phone train. Time will tell what happens to Qt mobile, but its becoming a very strong player in PC and set-top box devices.

Yes the platform support (C library) is a big factor in cross platform applications and with multiple C libraries and embedded platforms sniping out some of the library to reduce the size of the kernel, having a C library and webkit toolkit you can count on being, cross platform the same, has it's benefits. And we don't fully know what Netfix used QT Webkit for. DRM?, their own Gstreamer Adaptive streaming open source player modified to support ultraviolet DRM? Still lots of questions. QT does interface with Gstreamer. Maemo uses Gstreamer.

Nokia (purchased QT, sponsors development and open sourced it) has a version of a webkit (QT) based on Cairo and is one of three Cairo main branches of webkit that could be used for the PS3. QT, Gnome GTK+3 and FL something are based on Cairo with a branch of GTK+3 being written to support games sponsored by Second Life(configuration options to reduce resources needed by webkit).

I only mention this because it appears in the embedded arena, they are one of the leaders, or were a leader with QT and Maemo (Intel and Nokia co-sponsored). Now with the decision to go with Microsoft and WP7 things might change. (For this reason alone, Sony might cross QT off the list of potential webkit ports.)

It's hard to determine which route Sony will go with the PS3 beyond the given information that it will be Cairo based. Google searches are confusing because these branches are leapfroging each other and search results often do not give information on the date. GTK+3 Gnome appears to use the fewest resources again supported by Second Life sponsoring development for a branch to be used by it's Game.

Then there is the C library support in the PS3 and the various webkit branches support various C libraries. I think....and this needs to be confirmed as it's a wild guess, the PS3 uses GlibC and prefers an Objective C compiler (applications). GTK+3 Gnome supports objective C, the Power PC processor and Gstreamer and appears to use the fewest resources.

 

[patsu]

jeff, after the NetFlix tester, I met one of the NetFlix managers in charge of the PS3 client about a month ago. They were just using a regular WebKit port (i.e., HTML4). The current UI runs on JavaScript.

 

[patsu]

jeff_rigby found a blog by one of the Video Unlimited developers:
http://www.satine.org/archives/2011/09/27/playstation-web-app/

This is the new PlayStation Video Unlimited service. This PlayStation app runs at a full 60 frames per second (when you see it on a PS3), has tons of 3D graphics effects, full-speed 1080p video playback, and a fluid, hardware accelerated, animated user experience.

...

The Video Unlimited service is a JavaScript application with a carefully designed runtime platform and very lightweight APIs to access hardware accelerated 3D graphics and shader effects, video playback engine, and other aspects of the PS3 hardware.

...

Trilithium’s strength comes from taking full advantage of the PS3 hardware and existing well-optimized frameworks to do everything from graphics to video playback, leaving the decisions about the high level application to a very flexible JavaScript core API.

We built Trilithium for several reasons:

Make good use of the complex 8-core + GPU PS3 hardware without killing ourselves.

Give this power to our UX developers and designers.

Let partners easily build their own PS3 apps with little knowledge of PS3 architecture.

Rapidly develop with a flexible environment.

It would appear HTML5 isn't the future then. Instead this is offering a proprietary framework built around Java code access. That's an efficient, lowish level application environment similar to mobile devices. I wonder how Trilithium relates to PSS? And future HTML5 development. We're still stuck with a crappy browser, and there isn't a secret HTML5 core in operation on PS3 that could be turned into a decent browser.

 

[jonabbey]

Yeah, I noticed the PSN store now has an 'Apps' section under Media. Wonder how committed Sony is to it. They could certainly give Apple and Google TV a run for their money if they cared, but if they cared, one would think they'd have done this years ago.

 

[patsu]

jeff_rigby found a blog by one of the Video Unlimited developers:
http://www.satine.org/archives/2011/09/27/playstation-web-app/

It would appear HTML5 isn't the future then. Instead this is offering a proprietary framework built around Java code access. That's an efficient, lowish level application environment similar to mobile devices. I wonder how Trilithium relates to PSS? And future HTML5 development. We're still stuck with a crappy browser, and there isn't a secret HTML5 core in operation on PS3 that could be turned into a decent browser.

Interesting ! B3D mods... someone posted the above note using my ID.
Thanks to jeff_rigby for pointing this out to me. ^_^

I'll leave it alone and update it with my own comments in the mean time.

Idiot Mod who edited, not quoted, says:

Trilithium is JavaScript based. Not Java based. And we don't know if a HTML5 WebCore is available on PS3. There is a HTML4 WebCore used by NetFlix and EA though.

It would not be surprising if these apps rely on some combination of HTML, XML and CSS for UI presentation. In fact, the "About Video Unlimited..." section includes open source licenses for libxml and WebKit.

 

[patsu]

Yeah, I noticed the PSN store now has an 'Apps' section under Media. Wonder how committed Sony is to it. They could certainly give Apple and Google TV a run for their money if they cared, but if they cared, one would think they'd have done this years ago.

It makes sense if they want to bring PS Suite applications over to PS3 and Vita.

I think Sony should continue to work with Google on their media management and search features.

As a consumer, I think Sony should offer some key services on iOS so that I can continue to use PS content on iOS devices.

 

[Shifty Geezer]

Interesting ! B3D mods... someone posted the above note using my ID.

Yes. You did! I copied you post from the Media Player thread.

Trilithium is JavaScript based. Not Java based.

Yeah, my bad. My point is though, Sony haven't followed through with HTML5, but are providing a script-based application layer of their own. If PS3 was going to go HTML5 as some suggested/hoped, it should have happened with the switch to UV and xxx Unlimited. I'm wondering if Sony considered that and felt it left their platform too open for non-Sony sanctioned content, so instead are limiting apps to a particular Sony flavoured SDK giving them control, but designed around JS so it's easy to develop for.

 

[deathindustrial]

Not sure what it means exactly as far as HTML5 is concerned but the credits section for the new video browser includes a bunch of FOSS libraries including WebKit.

 

[AzBat]

Not trying to overshadow the great PSN chat here, but I just found this rumor...

our source also revealed that, in addition to a new Windows 8-inspired user interface and a variety of Kinect control enhancements, the update will include web browsing through an Internet Explorer app. Internet Explorer on the Xbox 360, along with the new Bing Search, is apparently “very cool” according to our source.

http://www.warpzoned.com/?p=14390

Seems out there considering Microsoft hasn't said anything about it. However, with the new Bing integration & a Kinect interface it might actually be possible.

Tommy McClain

 

[bkilian]

Not trying to overshadow the great PSN chat here, but I just found this rumor...



http://www.warpzoned.com/?p=14390

Seems out there considering Microsoft hasn't said anything about it. However, with the new Bing integration & a Kinect interface it might actually be possible.

Tommy McClain

I wouldn't get my hopes up. Rumors are generally just that. I mean, they've been insisting we'd get a blu-ray addon for the last 3 years. From the looks of that article, their "source", if legitimate, is not on the xbox team.

 

[patsu]

Yes. You did! I copied you post from the Media Player thread.

Hah, you copied my post and added your own comments too !

Yeah, my bad. My point is though, Sony haven't followed through with HTML5, but are providing a script-based application layer of their own. If PS3 was going to go HTML5 as some suggested/hoped, it should have happened with the switch to UV and xxx Unlimited. I'm wondering if Sony considered that and felt it left their platform too open for non-Sony sanctioned content, so instead are limiting apps to a particular Sony flavoured SDK giving them control, but designed around JS so it's easy to develop for.

If the PS3 JavaScriptCore is using hardware accelerated graphics, then it should not be far from the HTML5 canvas capability. Media tag may be a separate thing altogether. e.g., NetFlix continues to use the <Object> tag for video playback. My guess is the PS3 web app framework should be HTML4+.

The new NetFront NX browser is based on WebKit and supports HTML5. If Sony continue the relationship with ACCESS for Vita, then the Vita browser should understand HTML5.

 

[Shifty Geezer]

Hah, you copied my post and added your own comments too !

Ack! I edited instead of quoted. I'll update the copy to reflect this.

 

[patsu]

No problem... I thought someone hacked my account. ^_^

Standalone HTML5 browser is useful. I reckon Google and many contents providers also want to use the same (or similar) web app for all devices.

However Sony's "Skinny app" framework is important to make PS3 apps feel "connected". The look & feel and UI elements can be enforced. The new Home v1.55 startup screen has similar color theme as Video Unlimited. I hope v1.55 adds the same navigation concept to Home. Can't enter Home right now since the servers are under maintenance.

When you look at the entire XMB environment, it has:

* Location-based navigation (The Globe in "Life with Playstation")

* Timeline navigation (PlayView calendar of event, Photo Gallery timeline)

* Media meta-data navigation (Video Unlimited, and in the future Music Unlimited and PS Store too !).

A uniform 2D user experience can help to consolidate all these navigation scheme together (into widgets, "desktop/launch pads" and applications). Instead of *standalone* Facebook, FourSquare, ... apps in PS Vita, I feel that PS3's approach is more useful.

e.g., The Photo Gallery app in PS3 is one of the first apps to present Facebook albums in timeline view:

You can manage and organize the photos with advanced analysis (Separate pictures with children from those with adults, group landscape pictures together). I remember the same framework works for Picasa also. I should be able to manage all my social albums from one location. Now if they can combine all the PS3 photo apps using the new HTML framework, and integrate 3D media support, I think it would be a lot more usable, seamless and forward looking.

In particular, Playstation Home should be able to assimilate and present all 3 navigation schemes seamlessly in an abstract 2D + 3D world.

This is one of the reasons why I think the Vita UI is a step back from PS3. Fortunately, Vita has LiveArea (Activity, and real geolocation feature). But those can be added to PS3 XMB apps and PS Home easily too. Besides, the XMB chatroom is under appreciated. It is a very powerful concept.

 

[patsu]

Bringing Cut the Rope to Life in an HTML5 Browser: Behind the Scenes
http://blogs.msdn.com/b/ukmsdn/arch...fe-in-an-html5-browser-behind-the-scenes.aspx

Cut the Rope is an immediate favourite for anyone who plays it. It’s as fun as it is adorable. So we had an idea: let’s make this great game available to an even bigger audience by offering it on the web using the power of HTML5.

To do this, Microsoft’s Internet Explorer team partnered with ZeptoLab (the creators of the game) and the specialists at Pixel Lab to bring Cut the Rope to life in a browser. The end result is an authentic translation
of the game for the web, showcasing some of the best that HTML5 has to offer: canvas-rendered graphics, browser-based audio and video, CSS3 styling and the personality of WOFF fonts.

You can play the HTML5 version of Cut the Rope at: www.cuttherope.ie.

...

 

[wco81]

That game is purely for touch UI since the main mechanic is to do a swipe.

You'd use a controller to run the browser in a console?

Game is 99 cents on iOS and free on Android isn't it?

 

[patsu]

Console concept will evolve to reach the audience. 3DS, Vita and WiiU have touch control. Wii, Move, and Kinect have pointer control. Plus the same thing can serve iOS/Android/PC/Mac crowd.

Ad model, cheap model, or free-to-play, or subscriptions for a large base may be ok.

 

[patsu]

... and here's a Google Map game using WebGL:

EDIT: Reminds me of The Last Guy:

With its rich OS and entertainment focus, Vita should try to turn the entire Internet into a huge gaming experience. I remember when The Last Guy was launched, they released the same game to run on any web page. Give the site any URL, you can evade monsters and rescue people on the web page itself (Using the HTML layout as the map):
http://multiplayerblog.mtv.com/2008/07/31/play-sonys-nutty-last-guy-game-on-this-website/