Emergent's Dan Amerson Opens The Floodgates

["Nerve-Damage"]

Didn't see this posted

Can you talk about how the Floodgate initiative began?

Dan Amerson: Late in 2005, we were working on Gamebryo 2.2 and received an invitation to present at Sony DevCon and DevStation 2006. Two of our senior engineers, Ed Holzwarth and Vincent Scheib, decided to put together a presentation that offered a lot of technical value and was not just a demonstration of our technology.

Given that we, like many other developers, were faced with the problem of converting our mostly single-threaded engine to run well on the PS3, they started to research techniques for leveraging the SPUs on the PS3 while reducing the workload on programmers.

The presentation that they put together offered several proposals for how to implement various systems on SPUs. Those proposals evolved into a design that we later implemented during Gamebryo 2.3. Our goal was to put together a system that both simplified our internal development and allowed our licensees to easily parallelize their game logic. There are a few features from that presentation that we have not implemented yet, but the design comes largely from that presentation.

While throwing ideas around with marketing a few months later, I came up with the name "Floodgate," since the technology controls and unleashes the pent up power on the consoles.

What is Floodgate used for?

At its core, Floodgate is a stream-processing engine. Developers provide execution kernels that are just code that runs per element of a stream and specify the inputs and outputs to the kernel. Inputs can be fixed across all elements or can be a stream of elements. These items allow developers to build up tasks that Floodgate will execute asynchronously.

On top of that, developers can link tasks together by specifying outputs from one task as inputs to another task. Floodgate uses that information to schedule tasks allowing complex sets of behavior to emerge. Most importantly, all of this work occurs through a single API allowing developers to target a new platform just by recompiling their kernels and application code.

How does Floodgate operate differently between Xbox 360 and PS3?

Once a Floodgate task is submitted, the system determines the most efficient way to execute that task. On PS3, the task is divided up into workloads and sent to the SPUs for execution. The system automatically determines the optimum size for each workload to minimize the number of DMA transfers. When the results are ready, they are recombined by Floodgate and made available to the application.

On Xbox 360 and multicore PCs, a similar process happens. The task is divided up into workloads. In this case, though, we distribute the workloads across multiple threads. Floodgate also determines the optimum size for these workloads to maximize data pre-fetching.

We've seen and heard about a number of high profile delays and performance issues recently with the PS3, and you've presumably had to work quite closely with its architecture compared to the others -- what's your take? Is working with the Cell truly as difficult as the anecdotal evidence?

Programming for Cell requires managing a different set of problems than the shared memory machines that most developers are used to using. You have to make sure your data fits into the local stores of the SPUs, and you need to get that data to those SPUs.

If you write a multithreaded engine for a multicore or PC, you will almost definitely have to re-architect significant pieces to run well on the Cell architecture. They will run well if you invest enough development time though, there’s certainly power in that chip.

Since developer time is expensive and limited, we either see delays or performance issues when there isn’t enough time to get everything running correctly on the Cell architecture. Fundamentally, this goes back to your first question about how Floodgate got started. We recognized that there are very significant costs to rewriting systems for each platform. With Floodgate, you effectively get to write to one platform, Floodgate, and the port to other platforms is eased significantly by the technology under the hood.

 

[Crossbar]

Fix link, please.

 

[AlNom]

Gamasutra:
http://www.gamasutra.com/php-bin/news_index.php?story=16924

 

[Arwin]

It’s absolutely a more difficult problem, but no one buys software because it solves simple problems.

Heh. Nice.