I always thought, that OpenCL does not make any assumptions about the underlying hardware (or as few as possible). This is also the reason why designations like "threads" were avoided. Work items are not necessarily individually scheduled. The smallest really individual entities OpenCL knows of are the workgroups (how the work items are handled is up to the implementation and not specified, some types of workflow is explicitly not guaranteed to work within a workgroup to allow execution in SIMD fashion). My characterization with the local memory as private to the workgrops therefore exactly fits the definition of a resource only visible to a single invocation/"thread of execution" you just came up with.
And as you just wrote, the OpenCL API leaves huge holes, so we actually can't consider it to deliver a rigorous perspective.
You can't have multiple CS kernels from different processes in flight at the same time? What is nVidia's HyperQ doing, btw.? I guess AMD's ACEs (you have two), especially with the extensions coming with GCN 1.1 can basically achieve something similar. And there is also the device fission OpenCL extension. Hmm.As I said, you can't corrupt other processes' (shared local) memory on *any* GPUs right now because you can't have multiple DMA buffers from different processes in flight at the same time. This is all an academic discussion
True, but they *can* be. Indeed even single work items can be scheduled individually subject to barrier operations. Or an entire workgroup can be scheduled together. Code is not allowed to make any assumptions either way.
Nah but that only works if I could write code as if the workgroups were indeed the smallest schedulable entity, but I'd wreck GPUs if I write all my code with 1 item/groupMy characterization with the local memory as private to the workgrops therefore exactly fits the definition of a resource only visible to a single invocation/"thread of execution" you just came up with.
Honestly at this stage I don't buy that the additional abstraction of workitem is particularly useful compared to something like ISPC where I can just write my code with a parameterized SIMD size (which enables more efficient expressions of algorithms), but that's the way it is unfortunately...Sure, but DX is pretty good in that regard, which is why I tend to use it in examples. I only switched to CL because OpenGL Guy rejected the relevance of DX (for self-evident reasons I guess?
).It's coming, but it's not all here yet on consumer solutions, and as you make clear, it's all extensions and not required by any specs yet. And yeah I definitely agree that process isolation is a requirement (which is what started the entire subthread here), I just don't think going beyond that is particularly useful. An ideal implementation would crash and tell me where if I access outside of bounds within my single process.
They probably took it down to do the changes Dave suggested. Btw., you should be aware that this appears to be a bit preliminary and contains a few more errors than usual.
Yes, you are no allowed to make assumption which makes the workgroup the smallest individually scheduled entity you can rely on.
The smallest individually scheduled items on GPUs are the wavefronts/warps which almost coincide with the workgroups. Where is the problem?Nah but that only works if I could write code as if the workgroups were indeed the smallest schedulable entity, but I'd wreck GPUs if I write all my code with 1 item/group
Does the description of the group shared memory of DX deviate in a significant way from the local memory of OpenCL? Doesn't DX specify what should happen for out of bounds accesses to texture or buffers (but lets it undefined for shared memory)? I don't get the self-evident reasons part.Sure, but DX is pretty good in that regard, which is why I tend to use it in examples. I only switched to CL because OpenGL Guy rejected the relevance of DX (for self-evident reasons I guess?
Okay, now you backtracked from *any* GPU to available consumer solutions and being an extension and not required by spec.
Because it's not only visible to a single schedulable entity. If I create a workgroup of size 1024 on SI that'll be multiple different scheduled entities that are looking at it. Otherwise I wouldn't need barriers. That's fundamentally a different programming model than one in which my single serial thread of execution (even if it uses SIMD) is accessing a consistent view of some resource (registers).
DirectX specifies that OOB accesses for local memory can only affect other local memory (but on the whole machine). OCL leaves it open to any sort of undefined behavior or arbitrary global memory corruption.
I was just joking about his name being "OpenGL Guy"
My statement was in the context of graphics APIs, sorry I didn't make that clear. Pure compute stuff kind of lives in its own (somewhat simpler) world.Okay, now you backtracked from *any* GPU to available consumer solutions and being an extension and not required by spec.
I think it is quite useful. Enough with the shared mutable state already. The hw checks are quite cheap. The hw can provide signals for oob access if you do oob checking in the first place, making debugging easier. The spec should mandate it. Speaking of Khronos specs, sigh...
The handle predates 7+ years of Direct3D driver work and 3 years of OpenCLI was just joking about his name being "OpenGL Guy"
Agreed for general array indexing, but this is less than that. It's for this sort of arbitrary concept of "workgroup resource boundaries"... i.e. I can still stomp all over other arrays that my workgroup uses. Definitely I won't complain about any additional debugging signals I can get though, although suppressing is less useful. Frankly though for debugging compiler-inserted bounds checking is fine too.
I know, I was just teasing because that sentence came out sounding funnyThe handle predates 7+ years of Direct3D driver work and 3 years of OpenCL
Aren't we supposed not to make assumption about the hardware and to look on it from the API perspective (that was your suggestion). The API does not make any assumption about the size of the individually scheduled entities. It could be a SIMD architecture with a vector size of 1024 for that matter. In that sense, the barriers are just there to ensure consistency for all possible implementations. If you write non portable code and use workgroups the size of the vector size of the underlying hardware, you don't have to use them. From the API side, the workgroups are the smallest schedulable entities you can rely on and OpenCL just supports execution on all kinds of hardware (with different supported sizes).
As stated several times already, I disagree. There is just a different kind of memory defined which is supposed to be private to a workgroup. This is from the basic definition much closer to registers (private to a work element, btw., as said, OpenCL also doesn't specify anything about OOB private accesses and also does not state it has to sit in registers, so where is the difference?)than to global memory (which is accessible by all work items from all work groups and also from other kernels [of the same process]).
Yes, as I said, DirectX gives specs for global memory and basically just use different words for the local/shared memory, they say the content of the shared memory (all of it) is undefined after OOB writes, i.e. it is unspecified, what operations are exactly carried out, same as with OpenCL. That they restrict corruption to the shared memory has the simple effect that you are not allowed to run shaders using shared memory without having it in hardware as a separate array (or to provide equivalent means of doing so).
I seem to have missed that.
1):
It's the same arbitrary concept as having registers private to a thread.
We're going in circles here. This entire line of questioning came out of the assertion that SLM is like registers, which makes sense from the perspective of GCN/hardware. But that's obviously not true from the programming language perspective. Registers have a consistent view from a single work item... they can't suddenly change unrelated to the work item's execution. SLM *can*. It *might* not depending on the implementation, but the fact that it can means it's a totally different beast from the programmer's perspective.
I'm not even sure of what we're arguing about or being unconvinced about at this point, so that's unsurprising
Sure, being able to debug it is great! But as I said above, it's really the most useful if it was bounds checking on *arrays*, not on SLM in general, which is pretty unlikely to bother putting in hardware. Whether or not I declare a second SLM array shouldn't affect whether or not OOB access to the first one is detected... Given that I want to fix these bugs before shipping something, I'm also fine with compiler-inserted bounds checking, but whatever.
We're going to have to agree to disagree on this. I understand your point of view, but the two have widely different semantics in the API in terms of how they relate to the execution model. But as above, we appear to be just going in circles on that. Sigh.It's the same arbitrary concept as having registers private to a thread.
Are you seriously arguing that the GCN register file/LDS design is the only one that makes sense?
I would say, we are not moving at all.
You mix things up here. APIs define the local/shared memory as private to a workgroup the same way as the private memory of OpenCL is defined as private to a work item. That makes it fundamentally different to global memory which by definition is accessible by every work item of every workgroup and in principle even other kernels. There is no restriction defined, it is not owned or private to anything (besides maybe the process).
OpenCL has no concept of registers. We have to decide if we want to talk in API terms or from the hardware point of view. You suggested the API. And there is the private and the local memory in OpenCL with defined ownership. One should just accept this concept. Maybe one can keep in mind, that on GPUs private memory maps to registers and local memory to the local/shared memory arrays. On other implementations, for instance on CPUs, the data of the private memory is also mapped on usual memory locations somewhere (basically relying on the caches to keep latencies down). Is it there the task of the compiler to ensure, that the private memory keeps consistent for each work item? I asked you already what happens with out of bounds indexing in private memory. If it's not suppressed, aren't you then corrupting the private memory of other work items the same way as OOB accesses of the local memory corrupts the memory for other workgroups? Where is the fundamental difference?
It can't change unreleated to a workgroup's execution which is the scope this whole damn thing is defined on. It's very simple. And I said this before already, the API doesn't define the execution of a single work item. It states, you can't rely on an individual execution of a work item. The only thing you can rely on, is the execution of an individual workgroup. For the rest, see above.
The concept of work group private local/shared memory is actually the same from DX, through OpenCL to CUDA. It's differs quite a bit from global memory. But good that you understand my point of view.
Did I say so? I was just trying to express, that one has always choices for the implementations, some are good and have their distinctive set of pros and cons, others much more cons.
I'm not arguing *against* that design, I'm just saying I don't really care, because accessing out of bounds is an application error. I'm also completely fine with not running multiple processes on the same CU simultaneously... I doubt you'd even see a measurable performance difference outside adversarial cases.
I agree undefined behavior is undesirable, but conversely as I've said, I reject the notion that a shipping application should ever rely on the behavior of out-of-range accesses (other than texture sampling obviously, which has specific address translation baked into the semantics). Our only disagreement here is how much it matters.
From a programmer's perspective, yes. From a hardware perspective we'd need a lot more analysis to prove that a fairly different design that didn't require bounds checking at all wouldn't even up being better, but I'm willing to buy that with the small number of local memory address bits it's not the end of the world either way.
I don't find it arbitrary. Workgroups are supposed to be independent. Limiting sharing across workgroups is one way of exposing the notion of locality in the programming model. Enforcing it in hw is desirable. Workgroups are the natural units of isolation in ocl, since fine grained communication is possible only within a workgroup and inter workgroup communication/synchronization is expensive, if allowed at all.
I agree to a large extent. But I think having separate memory spaces in *programming model* is desirable. It's better to expose locality to code and let it use as much as it wants. For codes that dont care, just use all the SRAM as cache. Kepler is quite a bit like that KC is obviously fully there.
I suspect you mean that each CU's LDS is independent of the other CUs. If so, that's correct. With GCN, memory operations (local and global) are handled asynchronously from program execution. So when you issue a read from local memory, say, there's no stall until you need the data (you issue an explicit wait). This was a big change from EG/NI where global memory operations were handled in clauses and local memory operations would cause stalls. Now the compiler can better schedule instructions to hide latency.
