One concern for maintaining consistent performance with the current formulation of GCN is the latency in getting a kernel set up and running on the GPU.
If a new task is requested, it has to get on a queue and the GPU's allocation hardware has to allocate the necessary resources, such as buffers and CUs with enough register space and local memory avaliable to handle the kernel.
If the system is under load, queues may be filling up, or the CUs are running other shaders that occupy most of their storage. There are signs in the SI ISA manual (or whatever it gets renamed to) that AMD is increasing the amount queues to help with that possibility.
In terms of a kernel having to wait on resources and CUs to be freed up, preemption is not yet possible. If you don't want your high-priority functions sitting around until a CU finally gets released, in the absence of preemption you can instead reserve those resources ahead of time.
If you control what jobs can get to those resources, you can keep them freed up enough that your high-priority tasks don't find themselves stuck waiting on shaders that can run for an indeterminate period of time.
The price is an underutilization of the reserved resources, since some portion of their capabilities must be kept off-limits even when they're not needed.
The system functions also can't dream too big, since they only get this slice of the resources guaranteed.
