Would a GPU+SSD combo card make sense?

[Mendel]

I was thinking if a device that has both a high end GPU and a (small) SSD would make sense. Like think of it in terms of a card that has a geforce gtx680 or Radeon7970 *and* 30GB of relatively fast SSD storage.

This is where you would install the game or two that you currently actively play. It would have very fast (custom lane?) transfer of textures from this SSD to card´s main memory (and of course 16x pci express speeds to main board), which should be sufficient. This with maybe $50 - 100 extra cost to the card.

Any thoughts?

 

[3dilettante]

I do not see a significant benefit to this, especially if it's merely relatively fast flash memory.

It doesn't seem like the increase in bandwidth would be much in the grand scheme of things. A very expensive PCIe SSD can read at 1.5 GB/s, less than 1% of the memory bandwidth of a 6970 and 1/5 of the bandwidth of a 16x PCIe 2.0 slot.
It's also not like there's that much free space on the card to fit an SSD.

I'm not sure if the way standard flash memory and drive controllers work allows for a bandwidth that is so high that it becomes noticeable when compared to the bandwidths the system already supports.

If games are installed to the drive, that implies it is managed by the OS and CPU. The GPU would not be equipped to read or access it.

 

[hoho]

SSD's get their speeds from RAID'ing together tons of chips with expensive chips. Without that you'll just have another USB stick with similar speeds.

 

[moozoo]

SSD backed virtual graphics memory

@Mendel
The idea would be a 16GB (say) video card that had 4GB (say) of real ram the rest would be paged in and out from/to a 16GB array of SSD chips.
A game would just see a 16GB graphics card and could upload all its textures.

@hoho
The GPU chipset would have an on chip multi port SSD controller. It would not need to emulate a hard disk interface.

 

[hoho]

How many of those SSD chips would the GPU have to provide half-decent performance? I think a single modern chip today gives you something around 40-50MB/s of sequental read bandwidth.

How much more expensive would the GPU get roughly from the added SSD controller + much more complex PCB?

 

[Grall]

Interesting idea for sure, but unworkable in reality. Graphics cards are expensive, SSDs are expensive, combine expensive + expensive = very expensive.

Also, if either one of those two separate systems develop a problem, all of it needs to be junked, which is an incredible waste of resources. Considering the relativley low life-span expectancy of high-end graphics boards, this is not a tempting prospect IMO...

 

[moozoo]

Interesting idea for sure, but unworkable in reality. Graphics cards are expensive, SSDs are expensive, combine expensive + expensive = very expensive.

I was mostly thinking of nvidia tesla cards.
It would depend on the cost of ram vs cost of SSD.
For example a tesla card with 128GB where 8Gb is real ram the rest is SSD.
The advantage over using a normal SSD drive is that the data doesn't have to travel though multiple buses under CPU control.
It can go direct at 1GB per sec from SSD to graphics ram.(http://www.tomshardware.com/news/PhotoFast-SSD-PCI-e,7719.html)

 

[hoho]

The PCIe bus has several times higher bandwidth and lower latency from main RAM over PCIe to GPU and reads/writes can be DMA'd (no need for CPU to get involved). So yeah, still not making much sense

Fastest SSD's have around 30x slower throughput than PCIe 3.0, 15x slower than PCIe 2.0

 

[jlippo]

The PCIe bus has several times higher bandwidth and lower latency from main RAM over PCIe to GPU and reads/writes can be DMA'd (no need for CPU to get involved). So yeah, still not making much sense

Fastest SSD's have around 30x slower throughput than PCIe 3.0, 15x slower than PCIe 2.0

Fastest PCIe SSDs have speeds of ~6GB/s read and ~4GB/s write..

Although it would be ridiculously expensive to put something like that on GPU, but I do kind of like the idea of having computational units with direct access to >100GB of memory.

 

[hoho]

Fastest PCIe SSDs have speeds of ~6GB/s read and ~4GB/s write..

... and cost around $50k

Although it would be ridiculously expensive to put something like that on GPU, but I do kind of like the idea of having computational units with >100GB of memory.

Get a box that has 8 memory slots per socket and fill them with 8G sticks. Should be decent for a while and cost a LOT less than that 6GB/s SSD card.

 

[jlippo]

... and cost around $50k

Well, yes it wouldn't be cheap.

Get a box that has 8 memory slots per socket and fill them with 8G sticks. Should be decent for a while and cost a LOT less than that 6GB/s SSD card.

Indeed it would and lot easier to have interface with the GPU as well.

 

[hoho]

Also if you'd really need that 5TB of space you'd still be better off just using it as a separate card in different slot than try to integrate it on GPU PCB

 

[moozoo]

Can you get plug in PCIe cards that exposes flash memory as a large contiguous block of memory that a GPU could directly DMA transfer. i.e. not exposed as a SATA hard drive.

 

[Grall]

Fastest PCIe SSDs have speeds of ~6GB/s read and ~4GB/s write..

That's not ONE SSD, though. That's actually 4-8 SSDs all RAIDed together. No SSD exists yet with enough channels to support such transfer rates.

 

[Andrew Lauritzen]

Why would you skip a level in the memory hierarchy? Why not back it by main RAM first, and back that by the HDD/SSD (i.e. how it is already done)?

 

[Davros]

If you had a gfx card with masses of ram considering cards are supposedly general purpose and support audio, could you install a game into gfx memory and have it run entirely on the gpu

 

[Andrew Lauritzen]

If you had a gfx card with masses of ram considering cards are supposedly general purpose and support audio, could you install a game into gfx memory and have it run entirely on the gpu

Lol no. GPUs are not (yet?) general purpose devices. They still require a lot of work on the CPU to feed them data and commands in a specific format for them to process. You could have done something like that with Larrabee, but alas it was not meant to be.

Furthermore even if you could do it, it would run *very* slowly. GPUs are not good at a lot of the types of work that games have to do and that would entirely bottleneck it even if the parallel bits were very fast. The CPU/GPU combo is here to stay for the foreseeable future.

 

[Mendel]

@Mendel
The idea would be a 16GB (say) video card that had 4GB (say) of real ram the rest would be paged in and out from/to a 16GB array of SSD chips.
A game would just see a 16GB graphics card and could upload all its textures.

Interesting idea to say the least. Probably a bit more workable too than what I was thinking. Wouldn´t help with the load times from where the game is installed though but after the game would be loaded there could be some pretty damn sharp textures But then there is the whole issue of game developer having to support this. If the SSD on a GPU was just another hard drive but just faster (to transfer data with the GPU), it would be transparent to the developer. User would just choose to install whichever game he wanted on that gpu ssd and would (I´m wagering) see some benefit with loading times as opposed to installing it on a regular s-ata SSD. (or god forbid an actual spinning drive)

Also I would hope such solution would work better when it runs out of framebuffer/texture ram as opposed to similar system that would have to stream textures across pci-e from the main memory of the computer. There needs to be less of a performance drop there to allow for better resolutions and textures... (my 1GB 5870 crossfire setup just comes to crawl sometimes at 2560x1600 and that´s painful) And it wouldn´t have to be insanely expensive. I think 60 GB of SSD isn´t more than 80 euros at the moment... Give it a couple years for stuff to become even cheaper and faster... I think giving extra pci-e slot to an SSD would make less sense in the future if you can just stuff a reasonable SSD on a graphics card that you´re going to install anyways. Pretty much all gpus have a sound card of sorts these days to drive the hdmi sound, I wouldn´t have thought that would happen just a few years ago. One thing I don´t think that would make sense though is OS installation on a card like I´m suggesting... I think I was on my fourth GPU on my previous os installation

 

[moozoo]

@Mendel
The idea would be a 16GB (say) video card that had 4GB (say) of real ram the rest would be paged in and out from/to a 16GB array of SSD chips.
A game would just see a 16GB graphics card and could upload all its textures.

But then there is the whole issue of game developer having to support this.

At the Graphics API level the card would just appear as having 16GB of ram.
So the developer would have no work to support it.
I assume games upload and cache textures as they use them.
Some games might preload the cache (Unreal? had an option for this)

Anyway, I don't think the idea of using flash as the backing storage for virtual memory is a new one. flash is cheaper than DDR and has the advantage of being random access and non-volatile.
The advantage of having the flash cards on a graphics card for GPGPU applications is that it scales with the number of cards.

Long term I am thinking SSD's could evolve into some kind of intelligent storage.

 

[hoho]

Also I would hope such solution would work better when it runs out of framebuffer/texture ram as opposed to similar system that would have to stream textures across pci-e from the main memory of the computer.

Once again, the throughput you can get through PCIe is around 10x more than the fastest consumer-grade SSD's offer on the market.