HTML5 on consoles

[Shifty Geezer]

Which both supports your statements and goes to show that development on web games are on-going. Adobe has shown a demo of a Flash 3D engine that seems to provide near Burnout Paradise graphics. The equipment it's running on was not mentioned...

But that's not HTML 5 per se, but a browser plugin. And we were doing that years ago wtih all sorts of browser gaming plugins. Someone writing a cross-platform engine that plugs into different browsers that allows hardware access is going to have to develop that per platform. You may as well use Unreal Engine 5 that's (hypothetically) developed with a version for every platform out there, then use Flash v12 in a PS4 browser running on a Sony developed Flash engine, as you'll get much better use of the system resources.

It's also worth considering that by the time browsers can play COD, proper PC and console games at that perioid will be two generations beyond. Which sounds about right, browser games being comparable to some 16 bit titles, although the lack of hardware acceleration has held back browser based (read Flash plugin based, as they're not written in javascript on an HTML canvas) games so we haven't had acess to Amiga level games with 60fps smooth scrolling and many sprites.

All in all, browsers becoming an actual high-performance gaming base is both a long way off and relatively off topic, because AFAIK you can't write a full game beyond the basics in javascript running in a browser window (edit: clarify that for proper fast action games, and not puzzlers etc). Server based apps where the browser just shows a video feed are more OnLive! type technology, for discussing in that thread.

 

[Demirug]

All in all, browsers becoming an actual high-performance gaming base is both a long way off and relatively off topic, because AFAIK you can't write a full game beyond the basics in javascript running in a browser window (edit: clarify that for proper fast action games, and not puzzlers etc). Server based apps where the browser just shows a video feed are more OnLive! type technology, for discussing in that thread.

Well Goggle was able to port Quake to run as a pure HTML/JavaScript+WebGL application some time ago. It runs with acceptable frame rates.

Yes, Quake is an old game but it shows that is possible. It although has shown the problems of the browser as platform that are not solved with HTML5.

 

[patsu]

What are those problems ? How did Google fix them ?

 

[Demirug]

They have not fixed them. The whole project was a technology study to see what’s possible.

One of the main problems is that you can’t get exclusive access to the input devices. That makes it impossible to get the standard FPS interface working in a perfect way.

The HTML5 audio system doesn’t give you position control. This way you can’t give the player feedback about the position of sound sources.

JavaScript itself has two flaws when it comes to larger apps.
1. As it is stored and transferred as sources instead of byte code you have to pay for a huge startup overhead. This could be improved at least to some point by storing a preparesed version in the cache for future uses. A better solution would be defining a bytecode/pcode for JavaScript. This would although reduce the download times.
2. JavaScript as a dynamic language is not very JIT friendly. This way a huge amount of performances is wasted. I expect some improvements here with the next ECMAScript Version that Introduce strong typing and OOP support.

Beside of this games will need real threading support. The webworkers solution is just too limited.

 

[jeff_rigby]

But that's not HTML 5 per se, but a browser plugin.

It's also worth considering that by the time browsers can play COD, proper PC and console games at that perioid will be two generations beyond. Which sounds about right, browser games being comparable to some 16 bit titles, although the lack of hardware acceleration has held back browser based (read Flash plugin based, as they're not written in javascript on an HTML canvas) games so we haven't had acess to Amiga level games with 60fps smooth scrolling and many sprites.

All in all, browsers becoming an actual high-performance gaming base is both a long way off and relatively off topic, because AFAIK you can't write a full game beyond the basics in javascript running in a browser window (edit: clarify that for proper fast action games, and not puzzlers etc). Server based apps where the browser just shows a video feed are more OnLive! type technology, for discussing in that thread.

What's attractive about cross-platform HTML5 webGL games is the large potential market. I expect the first games will be rehashes of older simpler games like what we are seeing in the Minis from Sony that run on both the PSP and PS3. Cell phones, tablets and iPod-Pad being a large new market that can't run AAA games but can run "Angry Birds" which I expect could be a HTML5 game. Then you have Chess, Checkers, Monopoly, Card games of all types, puzzle games you mentioned which don't have fast moving objects and can be tied to a server so there is little chance of piracy.

As handhelds and "personal" internet platforms advance in power and HTML5 evolves then faster more traditional AAA games would be possible. The timetable I expect will be limited by Cellphone and wireless networking development rather than HTML5.

We are already seeing performance increases in Cell phones being driven by competition. I'd guess about 40% increase in performance per year. Does that seem right? It's a somewhat geometric increase that will top out with battery use/life.

Off topic but Sony has a prototype fuel cell battery for laptops that uses alcohol to charge a normal battery. A couple of ounces of alcohol and you have power for a full day. There is also a military and Satellite battery that uses isotopes and converts low level radiation directly into power. This is not like older nuclear batteries that used heat and thermocouples.

 

[Shifty Geezer]

They have not fixed them. The whole project was a technology study to see what’s possible.

Well games have been written that run just on the GPU, but it's not really a tech that's going to oust the CPU for a while yet. Outside of fringe experiments that test theories, practicalities of games mean we aren't going to have procedurally generated games make artists redundant for a while, and we aren't going to be writing COD-type games in HTML for a while.

 

[Demirug]

Well games have been written that run just on the GPU, but it's not really a tech that's going to oust the CPU for a while yet. Outside of fringe experiments that test theories, practicalities of games mean we aren't going to have procedurally generated games make artists redundant for a while, and we aren't going to be writing COD-type games in HTML for a while.

Correct, but I don’t expect a COD style game with a general plugin like Flash either. We might see more such games with custom plugins. But the only reason for this is that average people have fewer problems to download and install a custom browser plugin than a full traditional game client.

In general the target audience for browser games currently expect only something like Farmville when it comes to the presentation. Adding some 3D would be already a big step for these people.

 

[Shifty Geezer]

Correct, but I don’t expect a COD style game with a general plugin like Flash either. We might see more such games with custom plugins.

Indeed, and IMO that's what Harrison was getting at with his vision. It'd only need Epic to create an Unreal Plugin supporting Android, Windows and iOS, and create Gears Web so it's downloaded and installed by millions of core gamers, and we'd be there.

How would HTML 5 improve this? I don't see what advantage it brings, and don't see how plugin-based gaming is a part of the HTML 5 on consoles development; web-based gaming can develop independently (although will no doubt grow around browser as they develop).

 

[Demirug]

There are some advantages by using pure HTML5 over custom plugins.
1. There is no need to port the plugin to different systems. As not every system supports custom plugins at all you need to write “player” apps for these systems.
2. As long as the target system provides free internet access there is no need for certification. It although simplifies patching on some platforms.
3. If you ask people to install a plugin to play a game in the browser over 50% of the potential players will not do it. Either they have bad feelings about it or they can’t do it because the have not enough privileges on the system.

But to going back to the general topic: The biggest problem of HTML5 for games on consoles would the performances. Current console CPU are very bad when it comes to execute dynamic script code. You need big chunks of code for games. Last time I checked we have went beyond 2.5 MB of pure JavaScript code.

 

[jeff_rigby]

There are some advantages by using pure HTML5 over custom plugins.

But to going back to the general topic: The biggest problem of HTML5 for games on consoles would the performances. Current console CPU are very bad when it comes to execute dynamic script code. You need big chunks of code for games. Last time I checked we have went beyond 2.5 MB of pure JavaScript code.

Just to be clear, you are projecting based on the current performance of Wii and PS3 HTML4 browsers that are not multi-threaded and do not JIT compile to native language which is a feature of current HTML5 Webkit javascript. Do you have a rule of thumb feeling for the % increase in performance going from an older HTML4 without multi-threading and JIT to newer Webkit HTML5? Single CPU platform would be better for giving us a min baseline.

CPU/GPU performance: I believe Sony will want the PS3 to efficiently play WebGL games. Much of a WebGL game are OpenGL calls which are already optimized for the cell. Sony may further optimize javascript CPU performance by including SPUs in some of the critical routines. This is speculation and we will have to wait for the release of the new Webkit browser unless you have inside information given Sony might be lining up companies to market WebGL games to the PS3. If this is the case, any firmer info on the webkit release date?

HD space: not a problem

Memory: This might be a problem for some games. Your game for instance, what are the hardware requirements?

Sorry if it seems like we are picking your brain but there has been too much unsupported speculation on the forums, my speculation being a large part of this. We do appreciate your input and consider it valuable.

 

[Shifty Geezer]

CPU/GPU performance: I believe Sony will want the PS3 to efficiently play WebGL games.

This would definitely not be true if it means people playing COD type games without having to pay Sony royalties!

 

[jeff_rigby]

This would definitely not be true if it means people playing COD type games without having to pay Sony royalties!

You may be right but Gabe (Portal 2) and Phil Harrison seem to believe in open systems. I quote again the relevant part of the article I posted from Phil Harrison.

The companies that succeed in that industry of the future, he said, will be the ones that can “satisfy the needs of the core gamer” on an open, browser-based platform like Facebook, rather than closed platforms like the iPhone and major consoles. “The console companies, I believe, will want to figure out how to become more open and to deliver more content without restriction to more people, to stimulate creativity,”

"Satisfy the needs of the core gamer" Social networking = Home, Twitter, Facebook, Picassa 3, JIVE, not the current Jive but what it can grow into using Java and Javascript which Sony can use to modify and customize the experience for the gamer.

If they do this properly with support for Cameras, and other CE equipment then consumers will never venture outside Sony provided cloud services. This requires great imagination of which we can only see occasionally in Sony products. There may be a cultural issue with the likes of Gabe and Phil (US and British culture) professing what Sony management (except for the Sony CEO now British) may not see and you are right or they do see it and my speculation is correct.

 

[Demirug]

Just to be clear, you are projecting based on the current performance of Wii and PS3 HTML4 browsers that are not multi-threaded and do not JIT compile to native language which is a feature of current HTML5 Webkit javascript. Do you have a rule of thumb feeling for the % increase in performance going from an older HTML4 without multi-threading and JIT to newer Webkit HTML5? Single CPU platform would be better for giving us a min baseline.

I am projecting based on my knowledge about webkit and PS3 programming (had done some in the past).I am very skeptical if Sony will port the JIT engine at all. In general they don’t like generating code at runtime.

CPU/GPU performance: I believe Sony will want the PS3 to efficiently play WebGL games. Much of a WebGL game are OpenGL calls which are already optimized for the cell. Sony may further optimize javascript CPU performance by including SPUs in some of the critical routines. This is speculation and we will have to wait for the release of the new Webkit browser unless you have inside information given Sony might be lining up companies to market WebGL games to the PS3. If this is the case, any firmer info on the webkit release date?

1. PS3 uses a custom GPU API not OpenGL
2. WebGL is a JavaScript wrapper for OpenGL ES 2.0. That’s not the same as OpenGL. Because of this browsers uses OpenGL ES to OpenGL wrappers (or DirectX) and IHV starts to add OpenGL ES drivers to their desktop drivers.
3. There is an OpenGL ES wrapper for the PS3. But it’s version 1.1 with custom extensions for shaders that are not compatible with the 2.0 model. Beside of this the wrapper is well known for being very slow. Nobody use it today.

I am not sure how the SPUs can help at executing JavaScript code. There might be some possibilities with the 2D canvas implementation but the GPU should be even better for this.

As said we have only limited information about future plans from Sony. WebGL is still a draft. If you take a look at the current browser games market most of them build either classic web pages or make use of Flash. You will not see many HTML RIAs out there. To make things even worse most of them haven’t used the 2D canvas yet as there was no support from Internet Explorer.
Developers of classic console games would not be interested at all. The already have their multiplatform engines and writing HTML RIAs is a pain as there are hardly any good tools for this job. It would be a step back for them.

Beside of this technical stuff I can bore you with commercial reasons why currently no company is interested in pushing HTML5 games to the consoles. But to make a long story short the console market is only in third place here. Depending on the kind of game desktop or mobile is a much stronger market. So let’s wait until we see WebGL games there and then talk again about porting them to a console browser.

 

[I.S.T.]

I've read unofficial reports that Silverlight version 5 will support HTML5 and your news that there will be hardware 3D support fits with Phil Harrison comments.

I've speculated that the "5" showing up in HTML5, Air 2.5, Flash Media server 3.5, Silverlight version 5, PS3 Firmware update 3.5 (which we now know has a HTML5 javascript engine in it) and PSP 6.35 which probably also now supports Flash Media server 3.5 streams (HTML5 and at least a partial Flash 10.1 Access for DRM). Can you confirm some of the speculation?

Are you really basing this off of the fact that they all have 5 in their names? Were you ever taught logic in whatever schools you went to?

 

[jeff_rigby]

I am projecting based on my knowledge about webkit and PS3 programming (had done some in the past).I am very skeptical if Sony will port the JIT engine at all. In general they don’t like generating code at runtime.



1. PS3 uses a custom GPU API not OpenGL
2. WebGL is a JavaScript wrapper for OpenGL ES 2.0. That’s not the same as OpenGL. Because of this browsers uses OpenGL ES to OpenGL wrappers (or DirectX) and IHV starts to add OpenGL ES drivers to their desktop drivers.
3. There is an OpenGL ES wrapper for the PS3. But it’s version 1.1 with custom extensions for shaders that are not compatible with the 2.0 model. Beside of this the wrapper is well known for being very slow. Nobody use it today.

I am not sure how the SPUs can help at executing JavaScript code. There might be some possibilities with the 2D canvas implementation but the GPU should be even better for this.

As said we have only limited information about future plans from Sony. WebGL is still a draft. If you take a look at the current browser games market most of them build either classic web pages or make use of Flash. You will not see many HTML RIAs out there. To make things even worse most of them haven’t used the 2D canvas yet as there was no support from Internet Explorer.
Developers of classic console games would not be interested at all. The already have their multiplatform engines and writing HTML RIAs is a pain as there are hardly any good tools for this job. It would be a step back for them.

Beside of this technical stuff I can bore you with commercial reasons why currently no company is interested in pushing HTML5 games to the consoles. But to make a long story short the console market is only in third place here. Depending on the kind of game desktop or mobile is a much stronger market. So let’s wait until we see WebGL games there and then talk again about porting them to a console browser.

Good stuff, thanks.

I am aware that PS3 games not designed to be portable across platforms use low level GPU calls published by Sony in APIs. OpenGL and DirectX are published routines to allow easy portability for games. Even with hardware changes in the same platform (a newer faster graphics card) the same DirectX and OpenGL calls will work. And this is the reason the X-box is called Xbox for the DirectX that Microsoft wrote to compete with OpenGL on Unix and other platforms for GAME portability.

We can go on to say that Consoles which do not have to worry about upgraded hardware or hardware changes can use the more efficient (in most cases) lower level GPU routines.

The PS3 webkit port is going to be a Cairo/Posix port. Cairo can use the OpenGL library for the backend. The ES is assumed as WebGL is based on the least common denominator and that is handsets. The PS3 was designed with the same subset of Desktop OpenGL graphics calls that handhelds use.

I was not aware that Sony had not optimized the OpenGL ES library of calls provided by Nvidia for their low level hardware routines in the RSX. I guess this was predictable as the shader routines are custom Sony code for the SPU/RSX combination.

To determine if Sony did port the Webkit JIT javascript engine unchanged, one would have to look at the GNU license required Sony site to see if the published changes to webkit included removing/rewriting a substantial amount of code. I don't think that is the case given a quick look at the changes documented. I can be wrong here and this needs confirmation.

You brought up a good point about the possible current state of PS3 OpenGL and shading being slow. This is another thing Sony will have to fix and might explain delays in the Webkit release.

 

[jeff_rigby]

Are you really basing this off of the fact that they all have 5 in their names? Were you ever taught logic in whatever schools you went to?

Quite extensively, were you taught the scientific method? You notice a correlation and speculate, you have a theory. You publish the theory and suffer criticism in a peer review.

Some criticism will be personal attacks and some will poke holes in your logic. In this case the only logic is the obvious "5" in all version numbers and that the industry considers HTML5 a pivotal - revolutionary upgrade.

When you combine HTML5 usually associated with WEBKIT and webkit by Google and named Webkit2 by Apple, which allows any object oriented language to call routines out of webkit, use the HTML5 javascript engine or even use routines out of Flash 10.1 or later, you have a massive impact on internet connected platforms.

There is also that Flash Media server 3.5 supports: HTML5, ultraviolet DRM, detects commercial blocking, supports commercials as well as social networking and SONY is supporting this with multiple IPTV ports and PlayTV. The PS3 Firmware update that supports Flash media server 3.5 streams is 3.5 and on the PSP the firmware update that supports this is 6.35. Another correlation with fewer data points.

You also have the Adobe open screen project and Ultraviolet DRM which over 60 industry leaders are supporting. Adobe provides Air 2.5 and Flash 10.1 free to all platforms. Both support HTML5 with a webkit javascript engine.

I may be wrong, I may be right. Without proposing and seeking confirmation I will never know. I thought this an interesting observation worthy of investigation.

 

[rpg.314]

When you combine HTML5 usually associated with WEBKIT ...

That is a wrong association in the first place.

...and webkit by Google and named Webkit2 by Apple...

Webkit and webkit2 are very different beasts, with material differences between them.

..., which allows any object oriented language to call routines out of webkit, ...

Object Orientation has got nothing to do with it. You can call webkit routines from haskell if you want to.

 

[Demirug]

I am aware that PS3 games not designed to be portable across platforms use low level GPU calls published by Sony in APIs. OpenGL and DirectX are published routines to allow easy portability for games. Even with hardware changes in the same platform (a newer faster graphics card) the same DirectX and OpenGL calls will work. And this is the reason the X-box is called Xbox for the DirectX that Microsoft wrote to compete with OpenGL on Unix and other platforms for GAME portability.

The DirectX Version on the XBox is different from the DirectX versions that are used on PCs. The share some common stuff but in the end you rewrite huge bunch of code. Anyway modern game engines uses API abstraction layer. Therefore the API used is less a concern.

The PS3 webkit port is going to be a Cairo/Posix port. Cairo can use the OpenGL library for the backend. The ES is assumed as WebGL is based on the least common denominator and that is handsets. The PS3 was designed with the same subset of Desktop OpenGL graphics calls that handhelds use.

The ES version of OpenGL is different. It's not just a subset. The PS3 make use of a desktop GPU derivate. Handhelds GPUs at this timeframe had much less features.

Beside of this I am not even sure if the general source of webkit even contains WebGL yet. It might e possible that this code is only in the development branches of Google and Apple.

I was not aware that Sony had not optimized the OpenGL ES library of calls provided by Nvidia for their low level hardware routines in the RSX. I guess this was predictable as the shader routines are custom Sony code for the SPU/RSX combination.

To make a long story short. What Sony has is completely useless to get WebGL running.

To determine if Sony did port the Webkit JIT javascript engine unchanged, one would have to look at the GNU license required Sony site to see if the published changes to webkit included removing/rewriting a substantial amount of code. I don't think that is the case given a quick look at the changes documented. I can be wrong here and this needs confirmation.

It’s not uncommon to throw away the whole Webkit JavaScript engine and use a different one. As JIT engines always requires some changes when porting to another system it’s highly believable that they don’t use it if they don’t publish changes. It’s still possible as you don’t need to publish your changes as long as the software is not public.

You brought up a good point about the possible current state of PS3 OpenGL and shading being slow. This is another thing Sony will have to fix and might explain delays in the Webkit release.

I don’t expect that the first release of a webkit browser on the PS will contain WebGL. The best I expect would be hardware accelerated 2D canvas. But even there I would bet my money on the worse software implementation that webkit use in general.

 

[jeff_rigby]

Webkit and webkit2 are very different beasts, with material differences between them.

What they have in common, developed by Google and adopted by Apple is multi-threading. Besides allowing, now requiring, multi-threading support in the OS which is also needed for multiple CPUs which increases page redraw speeds, the same process allowed for routines in a webkit to be called and used by other languages.

Object Orientation has got nothing to do with it. You can call webkit routines from haskell if you want to.

The call is object oriented and yes you can, by following object oriented language rules, use anything to call those routines.

You know this and should know the history. Have I upset you in some way?

 

[jeff_rigby]

The ES version of OpenGL is different. It's not just a subset. The PS3 make use of a desktop GPU derivate. Handhelds GPUs at this timeframe had much less features.

I'd forgotten that. Floating point and a couple of other features I think but you are right, the PS3 does support more Open GL ES (2.0 I think) than Handhelds. And OpenGL ES 2.0 is the proposed WebGL standard as well as the PS3's.

Beside of this I am not even sure if the general source of webkit even contains WebGL yet. It might e possible that this code is only in the development branches of Google and Apple.

Again, you may be correct here. I think I saw an accelerated option turned on in the config but that could be for 2D. But OpenGL 2D support in Webkit has been around since 2008. http://www.atoker.com/blog/2008/01/28/accelerating-webkit-with-openvg/

To make a long story short. What Sony has is completely useless to get WebGL running.

It’s not uncommon to throw away the whole Webkit JavaScript engine and use a different one. As JIT engines always requires some changes when porting to another system it’s highly believable that they don’t use it if they don’t publish changes. It’s still possible as you don’t need to publish your changes as long as the software is not public.

Sony did port the webkit javascript HTML5 engine as that is all they ported.

Again, it's possible here that Sony did not port the entire engine, just enough to get the support for HTML5 UIs being used by Netflix, Hulu, Lovefilm etc. The JIT I think yes, 2D cpu and 2D hardware accelerated yes also but the WebGL 3D calls possibly no.

This brings up an interesting side discussion. Netflix stated that it ported Webkit to the PS3 to support their UI. Knowing that webkit can't be ported to the PS3 without multiple PD libraries and changes to the PS3 OS I was skeptical of the claim. I assumed just the Javascript engine but they would have to have the font and Cairo routines chosen by Sony. IF Sony were porting only portions of the webkit javascript engine this creates further issues, did Netflix do the same? Patsu assumed co-operation between Netflix and Sony but Netflix claims their engineers did it independently. News articles are even assuming that others are using the Netflix engine.

I don’t expect that the first release of a webkit browser on the PS will contain WebGL. The best I expect would be hardware accelerated 2D canvas. But even there I would bet my money on the worse software implementation that webkit use in general.

The port is apparently being done in the US and is overseen by Geoff Levand, the Sony engineer in charge of the Linux port to the PS3. Does this change your viewpoint? Considering "epicfail" more responsibility may be transfered to the UK or US.

All I have in rebuttal is that Sony is supposed to have 3-D picture and video support in the browser and that requires HTML5 and WebGL.

Thanks again for the corrections, constructive criticism and new viewpoint.