GPU Shared Virtual Memory

[moozoo]

What is required in the CPU/Mother board Hardware and OS to support shared virtual memory with a GPU?

From what I have read, the OS must be running a hypervisor of some kind and the hardware must support vt-d or IOMMU2.
Is this correct?

From my experiences of enabling Hyper-v on my windows 8.1 machine there are big performance problems with this.

see "Understanding High-End Video Performance Issues with Hyper-V"
http://blogs.msdn.com/b/virtual_pc_...nd-video-performance-issues-with-hyper-v.aspx

I have also read that even if an Intel cpu has vt-d support, it may not support all of the required parts of vt-d. i.e. vt-d is a set of features and not all cpu/motherboards support all of them even if you enable vt-d in the bios.

Given Intel does not make discrete graphics cards, it seems unlikely to me that they will support the level of hardware integration required to enable shared virtual memory with a discrete graphics card.

 

[pMax]

?
no, hypervisor is not needed, I believe. CPU hypervisor is just what it is - an hypervisor. A simple one can be made in few hundred lines of code, on AMD (fucking intel hypervisor is another thing).
An hypervisor has no special 'access' that your normal kernel wouldnt have (unless you restrict your kernel under an hypervisor, that is).

You need (video) driver support and some xUMA support on both CPU and GPU chip, as they have to share info.

 

[moozoo]

I have found a number of articles that clearly link virtualization hardware support to shared virtual memory.

For example
In the document "Intel Virtualization Technology for Directed I/O"

In section 2.5.1.1
"Shared Virtual Memory: For devices supporting appropriate PCI-Express capabilities. OS may use the DMA remapping hardware capabilities to share virtual address space of application processes with I/O devices. Shared virtual memory along with support for I/O page-faults enable application programs to freely pass arbitrary data-structures to devices such as graphics processors or accelerators, without the overheads of pinning and marshalling of data."

Also "AMD says IOMMU v2.5 is key for Linux HSA support"
at http://www.theinquirer.net/inquirer/news/2265844/amd-says-iommu-v25-is-key-for-linux-hsa-support

 

[DTMJ]

Part of the IOMMU spec (part of VT-d) allows the OS / hypervisor to specify a table that links process IDs (PASIDs in the spec) to page table pointers. This allows devices to store the PASID for an application context (D3D device / OpenGL context) on the GPU and use that to request VA -> PA address translations from the IOMMU.

The main motivation for this is to allow the GPU (or other devices) to do scheduling between processes without requiring the OS to set the page table pointer when the host process submitting the work changes. It also allows the GPU to execute work from multiple host processes simultaneously.

I'm not an expert here, so I'm not 100% sure how the IOMMU tables are initialized by the OS in a virtualized vs. non-virtualized environment.