https://www.dsogaming.com/news/intel-and-ucsc-showcase-dlss-like-ai-upscaling-technique/
Intel and UCSC have showcased a DLSS-like AI upscaling technique that will be further detailed at SIGGRAPH ASIA in December 20th. Intel and UCSC have used the Unreal Engine 4 Infiltrator Tech Demo in order to showcase this new AI upscaling image reconstruction tech.
This new AI upscaling technique is called QW-Net. According to the teams, QW-Net is a neural network for image reconstruction, where close to 95% of the computations can be implemented with 4-bit integers.
“This is achieved using a combination of two U-shaped networks that are specialized for different tasks, a feature extraction network based on the U-Net architecture, coupled to a filtering network that reconstructs the output image. The feature extraction network has more computational complexity but is more resilient to quantization errors. The filtering network, on the other hand, has significantly fewer computations but requires higher precision. Our network uses renderer-generated motion vectors to recurrently warp and accumulate previous frames. This produces temporally stable results with significantly better quality than TAA; a widely used technique in current games.”
Unfortunately, though, this image reconstruction technique is for offline usage (at least for now). UCSC claims that an optimized implementation for real-time inference remains future
work. In other words, we may never see this technique in games. Still, it’s fascinating witnessing alternatives to DLSS. It also shows how ahead of its time NVIDIA was with its image reconstruction
technique.
Concurrent to our work, Xiao introduced a reconstruction technique based on U-Net. Using an optimized inference implementation they reconstruct a 1080p image in 18 to 20 ms on a high-end GPU. In comparison, DLSS reconstructs a 4K image in under 2 ms. Both these approaches can reconstruct images at a higher resolution than the input render.”
https://player.vimeo.com/video/456337582
Intel and UCSC have showcased a DLSS-like AI upscaling technique that will be further detailed at SIGGRAPH ASIA in December 20th. Intel and UCSC have used the Unreal Engine 4 Infiltrator Tech Demo in order to showcase this new AI upscaling image reconstruction tech.
This new AI upscaling technique is called QW-Net. According to the teams, QW-Net is a neural network for image reconstruction, where close to 95% of the computations can be implemented with 4-bit integers.
“This is achieved using a combination of two U-shaped networks that are specialized for different tasks, a feature extraction network based on the U-Net architecture, coupled to a filtering network that reconstructs the output image. The feature extraction network has more computational complexity but is more resilient to quantization errors. The filtering network, on the other hand, has significantly fewer computations but requires higher precision. Our network uses renderer-generated motion vectors to recurrently warp and accumulate previous frames. This produces temporally stable results with significantly better quality than TAA; a widely used technique in current games.”
Unfortunately, though, this image reconstruction technique is for offline usage (at least for now). UCSC claims that an optimized implementation for real-time inference remains future
work. In other words, we may never see this technique in games. Still, it’s fascinating witnessing alternatives to DLSS. It also shows how ahead of its time NVIDIA was with its image reconstruction
technique.
Concurrent to our work, Xiao introduced a reconstruction technique based on U-Net. Using an optimized inference implementation they reconstruct a 1080p image in 18 to 20 ms on a high-end GPU. In comparison, DLSS reconstructs a 4K image in under 2 ms. Both these approaches can reconstruct images at a higher resolution than the input render.”
https://player.vimeo.com/video/456337582
