Let’s play the, “How much hardware do we need to fix these issues” aka “Will Next Gen Hardware be fast enough to at least fix these issues?” As of Feb. 2nd, 2013 we do not have confirmed hardware specs/architectural details for the next Xbox or Playstation so I expect this to be a fluid discussion. I do expect the discussion to shift from "what is needed" to "can they do it" in the coming months. What I hope to accomplish is to have a running thread looking at current console issues that impact visuals and gameplay and how new hardware will/won't resolve these and, just as importantly, not just fixing the issues of today but tackling new horizons of gameplay and rendering (will there be hardware left for such?)
A great test case for this discussion is Forza Motorsport 4 (FM4) for the following reasons:
* The first reason is there will certainly be a Forza Motorsport 5 on the Xbox 3 to eventually benchmark this discussion against.
* FM4 is a technically excellent title: 720p, rock solid 60Hz, 2xMSAA. The technical are good (cars on the track, physics, etc), has a slew of assets (cars, tracks), and has robust features (online, leagues, car modifications, rivals mode, etc). FM4 stands on the podium of “best racing games” in the industry as it has both robust gameplay features and excellence in technology (not either/or), i.e. one has not been sacrificed for the other.
* FM4 (2011) is the 3rd iteration on the Xbox 360 (FM2 in 2007, FM3 in 2009). This gives us a good baseline as the same developer, working on the same hardware, iterating the same franchise was able to utilize the machine. By looking at FM4 compared to other franchises we can get a rough sense of what the system limitations are versus what Turn10’s limits/design decisions are. i.e. the negative LOD bias for textures is a performance concession many Xbox 360 titles make due to the performance impact of AF (due to texture cache issues?).
* GT6 is also a certain release on the PS4. Artistic preferences and design accents aside the two titles and developers are roughly on par in term of technical excellence (although many would give a nod to PD in terms of lighting and Turn10 in terms of world detail). As the PS4 is rumored to be both similar in base technological decisions and design constrictions (TDP, noise, size, cost) but with possible differences in memory footprint, bandwidth, and hierarchies as well as GPU balance there will be some discernible discussion points to evaluate how the technological choices (hardware and software) aided, and prohibited, the developer from addressing issues and building forward.
Goal:
(a) Discuss the various IQ issues and game issues. (By game issues I mean # cars on track, physics, etc).
(b) How can these issues be addressed.
(c) Performance Cost to address.
(d) Will the announced hardware be able to meet these estimated costs.
(e) Will the new hardware provide enough hardware to explore new rendering and gameplay enhancements? (Will we for example get "split screen" gaming via 3D glasses? Fully integrated motion controls? The long fabled RT?)
Targets: For our discussion I ask the following targets be assumed:
60Hz. Non-negotiable for a racing sim. FM4 already runs at a solid 60Hz.
1080p. Quasi-negotiable. Diminishing returns, market penetration of HD let alone Full HD sets, etc. dictate a “hard” 1080p be required. Assume that dynamic resolution (with 720p being the lowest for our discussion), scaled horizontal or vertical resolutions, and sub-1080p upscaled (but, again, resolution still above 720p) are all techniques “on the table” but less than ideal. (Personally, after watching a couple seasons of Transformers Prime at 480p I would say 720p with gobs of techniques for pristine texture and edge quality would be awesome, but I digress).
4xMSAA or equivalent. MSAA is negotiable, but an Anti-aliasing effect minimally equivalent to 4xMSAA is to be expected as a baseline. Crude techniques such as edge blur, motion blur, depth of field, as well as post-process AA (FXAA, SSMA, etc) are all acceptable solutions although how pleasant they are is dependent on the quality of the implementation. 4xMSAA is a solid baseline equivalent that, anything less, would indicate a lack of consideration for AA
Assuming the above targets 1080p alone will drive about a 2x need for increased performance on whatever element(s) are limiting current performance. (1080p is 2.25x 720p but depending on what the limiting factor is there may not need to be a pure 2.25x increase due to how GPUs work). Right out of the gate FM5 is going to require some elements of the Xbox 3 to be 2x as fast to make the jump to 1080p before any of the following are addressed. Also bear in mind that most of the following points are addressing immediately visible shortcomings; the hope would be the new hardware not only addressed these issues but had enough horsepower to allow developers to go in new directions that go beyond fixing the current issues.
Another consideration is FM4’s replay mode runs at 30Hz and employs 4xMSAA, better LOD, and improved motion blur. We don’t know if the costs associated with these features but we can assume their cost makes a stable 60Hz beyond the consoles abilities (it could run on average above 50Hz or just above 30Hz, we don’t know).
With those points in mind I present some of the issues (from memory) I believe FM4 on the Xbox 360 had that I would hope FM5 on the Xbox 3 would address.
Day/Night Cycle. FM4 has time of day presets that cannot be changed. FM5 needs day and night options for all courses and ideally dynamic time of day adjustments to race any time of day the user wishes. A plus would be for it to change in realtime and an accelerated clock.
Weather. FM4 has no weather. FM5 minimally needs to offer sunny, cloudy, overcast, light rain, heavy rain, and thunder storm weather (with wind speed and visibility being options). Ideally FM5 would offer realtime transitions (rain to sunny as an example) and dynamic location syncing (e.g. if it is raining lightly at the Ring it should in the game). Snow, while looks cool, outside of specialty racing you don’t take the cars in FM into an icy, snowy coarse.
These two options are going to require major upgrades to how FM5 handles the following…
Shadowing & Indirect Occlusion. In FM4 some direct shadows have jagged edges. Indirect light occlusion is fairly weak as can be seen in overcast settings. FM5 is going to require dynamic shadows and a robust ambient occlusion system (not all methods are as robust or have the same quality) to not just correct FM4’s issues but also to prove convincing Day/Night and Weather.
Lighting. FM4 improved its lighting but the issue is there is a lot of work left both artistically and balance (see: GT5). These issues will be compounded by the need for a more robust pipeline would help that can offer convincing dynamic lighting. Paint needs to bring the car even more into the environment while the engine will need to improve indirect lighting (GI hacks) while at the same time offering robust point light (head lights) solutions. Atmospherics (lightening, flickering street lights, sparks) all need to be addressed. Little things like the glowing brake disks and tail lights need an overhaul. Lighting, both on the cars (paint), as well as point lights and world lighting will be the areas of biggest impact (along with post processing and particles) in making the world look better. It is a tall order to ask FM5 to move up in resolution, increase lighting IQ, but also asking it to become dynamic.
Load Times. FM4, as good as the load times are compared to many other racers, still is poor. The time from accepting a Rival to the track is poor—and this is compounded by having to go back to the Rival screen to select the next rival. I always skip testing my tuning changes because the time to make changes and then get to the track takes forever. FM5 needs to be designed so that a track is always in memory so whatever activity it is—test driving cars, tuning, painting, buying new parts, changing out a couple cars after a race, selecting a new rival, etc—that as long as the track is not changed you can always, and immediately, make these basic changes and be on the track within seconds.
AA. FM4 uses 2xMSAA most of the time and the IQ is reflective of this level of AA (most major jaggies are absent although on high contrast areas or items with fine geometry stair stepping and crawling are noticeable; to Turn10’s compliment they have improved AA each iteration by addressing various artifacting). A move to 1080p will help some. A number of current console games are making good use of 2xMSAA+Post Process AA (with an unfortunate hit to texture clarity). New GPU hardware makes post-process AA much cheaper.
Texture Resolution and Filtering. FM4 had solid texture resolution and variety for the genre (racing sim) and while a step above FM2 there remains much room of improvement at 720p, let alone 1080p. Filtering was poor with a negative LOD bias attempting to “solve” the low levels of filtering back causing all sort of distortion. The combination of fast motion and the unique decal system went a long way in hiding a lot of short comings. Everything to the cars, tracks, and skybox would benefit from better texturing. “Better texturing” would include, but is not limited to, a move to 1080p, higher texture resolution, more texture variety, substantial texture filtering, more texture layers, and dynamic effects (e.g. scratches, burnout marks on the road, etc). This will require more performance and memory to address these IQ issues.
Motion Blur and DOF. FM4’s replay mode had solid motion blur and DOF effects (for this console generation standards); the realtime racing was mainly limited to blur on the wheels and road. The key with MB and DOF is to use it sparingly and subtly, with the best results coming from quality implementations. Quality MB and DOF are not cheap and Bokeh is currently very expensive. With MB there remains the issue that most affordable methods confine the effect to within the geometry of the object moving. Post processing effects can “cover a multitude of IQ sins” from low amounts of AA and AF, lower texture resolution, and so forth.
LOD. FM4 has nice models but there is a clear difference in LOD between replay and realtime, e.g. some engines disappear. With increased resolution the LOD bias will need to be less aggressive—lack of detail in distant cars will no longer be hidden behind the cloak of resolution. With 2.25 as many pixels and cleaner IQ (AA, texture filtering, reduced shader aliasing) LOD issues will be more visible.
Reflections. FM4 lost the reflections of dynamic objects. The PGR3 hack of using the previous frame’s render target was a slick move but the changes in FM4 at dedication to 60Hz led to the feature hitting the cutting room floor. It needs to come back. This will become more relevant when dynamic lights and night scenes come to life.
Foliage. FM5 may be the first version that gets the real 3D grass announced in FM2. FM4 had some dithering on distant foliage and subtle LOD transitions. Tree leaves were typically entire sections of a branch with transparancies. Leaves didn't litter the track and flutter around, grass did not fly up and leave indentations or even muddy ruts, and gravel did not shoot out and litter the track. Cones were about the most interactive parts on the track. While the trees and the little grass you saw in FM4 was not bad FM5 has a lot of room for improvement, both in quality and quantity, to make the worlds look more realistic.
Particles. The dust and smoke in FM4 fall far behind the competition. The NFS and DiRT games years ago introduced thick clouds of “burning rubber” that dynamically swirled around your car. Is it too much to ask for a car traveling 125MPH hitting a dirt shoulder to toss up a thick cloud of dust that dissipates naturally? This is the single weakest part of FM4. FM4 needs to address smoke, dust, and gravel and also have falling leaves, sparks, and an assortment of subtle effects in the air. Semi-related would be heat wave distortions on hot tracks and exhaust systems. And of course there is the aforementioned weather elemental affects.
Paint. This falls under lighting as well but since cars in a car sim are made or broken by their paint it is worth repeating. FM4 improved the paint (better lighting balance, HDR artifacts were mostly removed, less shader aliasing) but as GT5 showed artistically there is room to grow and lighting balance in general is huge. A redesigned lighting pipeline and paint shaders will always be at the top of the list. But there are also subtle things, e.g. taking on the color of paint from another car impacting yours or the accumulation of dust, mud, and water.
Damage. FM4 is essentially using the same damage system from FM2. Sure cars roll over now but the damage is not very dynamic. Games like BurnOut Paradise (released in 2007 or 2008) seem like a technological leap over Forza 4. Heck even Toca 3 allowed your wheels to burst. The damage needs to appear more dynamic (there also needs, NEEDS, to be a "simulation damage" mode for realistic damage and impact on the race--or not racing may it be). This needs to extend to structural damage, panel damage, paint scraping, glass (lights, windows) cracking and littering the track, smoke, oil, etc.
Fences. Fences have been a trouble spot all generation for most games with the links being aliased and fading in and out and generally being a crawling mess. This absolutely needs to be fixed.
Cars on track. FM4 moved up to 12 cars on track in many situations but on some tracks this still seems quite barren. 16, 20, or 24 seem like reasonable targets. But consider: improved physics, improved damage modeling, jacked up rendering, etc but there will always be certain limits (e.g. online bandwidth) as well as gameplay constraints (more isn't always better). On the other hand driving through traffic is a BLAST and this really needs more cars, as do the big oval tracks.
Windshields. The cockpit view in FM4 is great and Kinect actually works! That said the windshield... oh wait, what windshield? What is said is PGR3 was a launch title and it had some of the best HDR and effects all gen. My guess was they were too expensive.
Summary: It is going to be expensive to 2x cost for increasing resolution, + improved LOD system, + better AA, + texture resolution and filtering, + MB and DOF, + more robust lighting pipeline, + dynamic lights and shadows for dynamic time of day and night driving, + robust particle system for weather and improved dirt/dust/smoke/crashes, + more detailed cars and more cars on track, + car physics, + track detail, + better foliage and grass and gravel, + overhauled damage system, all while improving the core gameplay features, - load time and making features accessible anywhere, and so forth is very much far and above what the Xbox 360 can do.
Can a 6x-8x increase in RAW throughput (plus extra for efficiency of new architectures) hope to meet the demands of all these areas in need of improvement? Considering the resolution jump that 6x-8x is more like 3x-4x in extra power to address the other shortcomings. That sounds like a very, very tall order.
A great test case for this discussion is Forza Motorsport 4 (FM4) for the following reasons:
* The first reason is there will certainly be a Forza Motorsport 5 on the Xbox 3 to eventually benchmark this discussion against.
* FM4 is a technically excellent title: 720p, rock solid 60Hz, 2xMSAA. The technical are good (cars on the track, physics, etc), has a slew of assets (cars, tracks), and has robust features (online, leagues, car modifications, rivals mode, etc). FM4 stands on the podium of “best racing games” in the industry as it has both robust gameplay features and excellence in technology (not either/or), i.e. one has not been sacrificed for the other.
* FM4 (2011) is the 3rd iteration on the Xbox 360 (FM2 in 2007, FM3 in 2009). This gives us a good baseline as the same developer, working on the same hardware, iterating the same franchise was able to utilize the machine. By looking at FM4 compared to other franchises we can get a rough sense of what the system limitations are versus what Turn10’s limits/design decisions are. i.e. the negative LOD bias for textures is a performance concession many Xbox 360 titles make due to the performance impact of AF (due to texture cache issues?).
* GT6 is also a certain release on the PS4. Artistic preferences and design accents aside the two titles and developers are roughly on par in term of technical excellence (although many would give a nod to PD in terms of lighting and Turn10 in terms of world detail). As the PS4 is rumored to be both similar in base technological decisions and design constrictions (TDP, noise, size, cost) but with possible differences in memory footprint, bandwidth, and hierarchies as well as GPU balance there will be some discernible discussion points to evaluate how the technological choices (hardware and software) aided, and prohibited, the developer from addressing issues and building forward.
Goal:
(a) Discuss the various IQ issues and game issues. (By game issues I mean # cars on track, physics, etc).
(b) How can these issues be addressed.
(c) Performance Cost to address.
(d) Will the announced hardware be able to meet these estimated costs.
(e) Will the new hardware provide enough hardware to explore new rendering and gameplay enhancements? (Will we for example get "split screen" gaming via 3D glasses? Fully integrated motion controls? The long fabled RT?)
Targets: For our discussion I ask the following targets be assumed:
60Hz. Non-negotiable for a racing sim. FM4 already runs at a solid 60Hz.
1080p. Quasi-negotiable. Diminishing returns, market penetration of HD let alone Full HD sets, etc. dictate a “hard” 1080p be required. Assume that dynamic resolution (with 720p being the lowest for our discussion), scaled horizontal or vertical resolutions, and sub-1080p upscaled (but, again, resolution still above 720p) are all techniques “on the table” but less than ideal. (Personally, after watching a couple seasons of Transformers Prime at 480p I would say 720p with gobs of techniques for pristine texture and edge quality would be awesome, but I digress).
4xMSAA or equivalent. MSAA is negotiable, but an Anti-aliasing effect minimally equivalent to 4xMSAA is to be expected as a baseline. Crude techniques such as edge blur, motion blur, depth of field, as well as post-process AA (FXAA, SSMA, etc) are all acceptable solutions although how pleasant they are is dependent on the quality of the implementation. 4xMSAA is a solid baseline equivalent that, anything less, would indicate a lack of consideration for AA
Assuming the above targets 1080p alone will drive about a 2x need for increased performance on whatever element(s) are limiting current performance. (1080p is 2.25x 720p but depending on what the limiting factor is there may not need to be a pure 2.25x increase due to how GPUs work). Right out of the gate FM5 is going to require some elements of the Xbox 3 to be 2x as fast to make the jump to 1080p before any of the following are addressed. Also bear in mind that most of the following points are addressing immediately visible shortcomings; the hope would be the new hardware not only addressed these issues but had enough horsepower to allow developers to go in new directions that go beyond fixing the current issues.
Another consideration is FM4’s replay mode runs at 30Hz and employs 4xMSAA, better LOD, and improved motion blur. We don’t know if the costs associated with these features but we can assume their cost makes a stable 60Hz beyond the consoles abilities (it could run on average above 50Hz or just above 30Hz, we don’t know).
With those points in mind I present some of the issues (from memory) I believe FM4 on the Xbox 360 had that I would hope FM5 on the Xbox 3 would address.
Day/Night Cycle. FM4 has time of day presets that cannot be changed. FM5 needs day and night options for all courses and ideally dynamic time of day adjustments to race any time of day the user wishes. A plus would be for it to change in realtime and an accelerated clock.
Weather. FM4 has no weather. FM5 minimally needs to offer sunny, cloudy, overcast, light rain, heavy rain, and thunder storm weather (with wind speed and visibility being options). Ideally FM5 would offer realtime transitions (rain to sunny as an example) and dynamic location syncing (e.g. if it is raining lightly at the Ring it should in the game). Snow, while looks cool, outside of specialty racing you don’t take the cars in FM into an icy, snowy coarse.
These two options are going to require major upgrades to how FM5 handles the following…
Shadowing & Indirect Occlusion. In FM4 some direct shadows have jagged edges. Indirect light occlusion is fairly weak as can be seen in overcast settings. FM5 is going to require dynamic shadows and a robust ambient occlusion system (not all methods are as robust or have the same quality) to not just correct FM4’s issues but also to prove convincing Day/Night and Weather.
Lighting. FM4 improved its lighting but the issue is there is a lot of work left both artistically and balance (see: GT5). These issues will be compounded by the need for a more robust pipeline would help that can offer convincing dynamic lighting. Paint needs to bring the car even more into the environment while the engine will need to improve indirect lighting (GI hacks) while at the same time offering robust point light (head lights) solutions. Atmospherics (lightening, flickering street lights, sparks) all need to be addressed. Little things like the glowing brake disks and tail lights need an overhaul. Lighting, both on the cars (paint), as well as point lights and world lighting will be the areas of biggest impact (along with post processing and particles) in making the world look better. It is a tall order to ask FM5 to move up in resolution, increase lighting IQ, but also asking it to become dynamic.
Load Times. FM4, as good as the load times are compared to many other racers, still is poor. The time from accepting a Rival to the track is poor—and this is compounded by having to go back to the Rival screen to select the next rival. I always skip testing my tuning changes because the time to make changes and then get to the track takes forever. FM5 needs to be designed so that a track is always in memory so whatever activity it is—test driving cars, tuning, painting, buying new parts, changing out a couple cars after a race, selecting a new rival, etc—that as long as the track is not changed you can always, and immediately, make these basic changes and be on the track within seconds.
AA. FM4 uses 2xMSAA most of the time and the IQ is reflective of this level of AA (most major jaggies are absent although on high contrast areas or items with fine geometry stair stepping and crawling are noticeable; to Turn10’s compliment they have improved AA each iteration by addressing various artifacting). A move to 1080p will help some. A number of current console games are making good use of 2xMSAA+Post Process AA (with an unfortunate hit to texture clarity). New GPU hardware makes post-process AA much cheaper.
Texture Resolution and Filtering. FM4 had solid texture resolution and variety for the genre (racing sim) and while a step above FM2 there remains much room of improvement at 720p, let alone 1080p. Filtering was poor with a negative LOD bias attempting to “solve” the low levels of filtering back causing all sort of distortion. The combination of fast motion and the unique decal system went a long way in hiding a lot of short comings. Everything to the cars, tracks, and skybox would benefit from better texturing. “Better texturing” would include, but is not limited to, a move to 1080p, higher texture resolution, more texture variety, substantial texture filtering, more texture layers, and dynamic effects (e.g. scratches, burnout marks on the road, etc). This will require more performance and memory to address these IQ issues.
Motion Blur and DOF. FM4’s replay mode had solid motion blur and DOF effects (for this console generation standards); the realtime racing was mainly limited to blur on the wheels and road. The key with MB and DOF is to use it sparingly and subtly, with the best results coming from quality implementations. Quality MB and DOF are not cheap and Bokeh is currently very expensive. With MB there remains the issue that most affordable methods confine the effect to within the geometry of the object moving. Post processing effects can “cover a multitude of IQ sins” from low amounts of AA and AF, lower texture resolution, and so forth.
LOD. FM4 has nice models but there is a clear difference in LOD between replay and realtime, e.g. some engines disappear. With increased resolution the LOD bias will need to be less aggressive—lack of detail in distant cars will no longer be hidden behind the cloak of resolution. With 2.25 as many pixels and cleaner IQ (AA, texture filtering, reduced shader aliasing) LOD issues will be more visible.
Reflections. FM4 lost the reflections of dynamic objects. The PGR3 hack of using the previous frame’s render target was a slick move but the changes in FM4 at dedication to 60Hz led to the feature hitting the cutting room floor. It needs to come back. This will become more relevant when dynamic lights and night scenes come to life.
Foliage. FM5 may be the first version that gets the real 3D grass announced in FM2. FM4 had some dithering on distant foliage and subtle LOD transitions. Tree leaves were typically entire sections of a branch with transparancies. Leaves didn't litter the track and flutter around, grass did not fly up and leave indentations or even muddy ruts, and gravel did not shoot out and litter the track. Cones were about the most interactive parts on the track. While the trees and the little grass you saw in FM4 was not bad FM5 has a lot of room for improvement, both in quality and quantity, to make the worlds look more realistic.
Particles. The dust and smoke in FM4 fall far behind the competition. The NFS and DiRT games years ago introduced thick clouds of “burning rubber” that dynamically swirled around your car. Is it too much to ask for a car traveling 125MPH hitting a dirt shoulder to toss up a thick cloud of dust that dissipates naturally? This is the single weakest part of FM4. FM4 needs to address smoke, dust, and gravel and also have falling leaves, sparks, and an assortment of subtle effects in the air. Semi-related would be heat wave distortions on hot tracks and exhaust systems. And of course there is the aforementioned weather elemental affects.
Paint. This falls under lighting as well but since cars in a car sim are made or broken by their paint it is worth repeating. FM4 improved the paint (better lighting balance, HDR artifacts were mostly removed, less shader aliasing) but as GT5 showed artistically there is room to grow and lighting balance in general is huge. A redesigned lighting pipeline and paint shaders will always be at the top of the list. But there are also subtle things, e.g. taking on the color of paint from another car impacting yours or the accumulation of dust, mud, and water.
Damage. FM4 is essentially using the same damage system from FM2. Sure cars roll over now but the damage is not very dynamic. Games like BurnOut Paradise (released in 2007 or 2008) seem like a technological leap over Forza 4. Heck even Toca 3 allowed your wheels to burst. The damage needs to appear more dynamic (there also needs, NEEDS, to be a "simulation damage" mode for realistic damage and impact on the race--or not racing may it be). This needs to extend to structural damage, panel damage, paint scraping, glass (lights, windows) cracking and littering the track, smoke, oil, etc.
Fences. Fences have been a trouble spot all generation for most games with the links being aliased and fading in and out and generally being a crawling mess. This absolutely needs to be fixed.
Cars on track. FM4 moved up to 12 cars on track in many situations but on some tracks this still seems quite barren. 16, 20, or 24 seem like reasonable targets. But consider: improved physics, improved damage modeling, jacked up rendering, etc but there will always be certain limits (e.g. online bandwidth) as well as gameplay constraints (more isn't always better). On the other hand driving through traffic is a BLAST and this really needs more cars, as do the big oval tracks.
Windshields. The cockpit view in FM4 is great and Kinect actually works! That said the windshield... oh wait, what windshield? What is said is PGR3 was a launch title and it had some of the best HDR and effects all gen. My guess was they were too expensive.
Summary: It is going to be expensive to 2x cost for increasing resolution, + improved LOD system, + better AA, + texture resolution and filtering, + MB and DOF, + more robust lighting pipeline, + dynamic lights and shadows for dynamic time of day and night driving, + robust particle system for weather and improved dirt/dust/smoke/crashes, + more detailed cars and more cars on track, + car physics, + track detail, + better foliage and grass and gravel, + overhauled damage system, all while improving the core gameplay features, - load time and making features accessible anywhere, and so forth is very much far and above what the Xbox 360 can do.
Can a 6x-8x increase in RAW throughput (plus extra for efficiency of new architectures) hope to meet the demands of all these areas in need of improvement? Considering the resolution jump that 6x-8x is more like 3x-4x in extra power to address the other shortcomings. That sounds like a very, very tall order.