This is the type of thing that gets me really exited for the near future. This will really help to improve the graphics quality and lower the production cost and development time for games. Nvidia has created the first video game demo using AI-generated graphics “It’s a new way to render video content using deep learning,” Nvidia’s vice president of applied deep learning, Bryan Catanzaro, told The Verge. “Obviously Nvidia cares a lot about generating graphics [and] we’re thinking about how AI is going to revolutionize the field.” “The structure of the world is being created traditionally,” explains Catanzaro, “the only thing the AI generates is the graphics.” He adds that the demo itself is basic, and was put together by a single engineer. “It’s proof-of-concept rather than a game that’s fun to play.” https://www.theverge.com/2018/12/3/...deo-game-graphics-nvidia-driving-demo-neurips NVIDIA Invents AI Interactive Graphics "Twenty-five years ago NVIDIA transformed the computer graphics industry by building the first GPU, the modern day tool of the da Vincis and Einsteins of our time. Now, a new deep learning-based model developed by NVIDIA researchers aims to catapult the graphics industry into the AI chapter. Using a conditional generative neural network as a starting point, the team trained a neural network to render new 3D environments, after being trained on existing videos. This AI breakthrough will allow developers and artists to create new interactive 3D virtual worlds for automotive, gaming or virtual reality by training models on videos from the real world. This will lower the cost and time it takes to create virtual worlds."..... https://news.developer.nvidia.com/nvidia-invents-ai-interactive-graphics/?ncid=so-you-ndrhrhn1-66582 This one shows how they can edit and configure a scene. Pretty awesome stuff in my opinion and really shows how flexible this technology will be.
The thing is less impressive (but still impressive!) if you see what is really doing. It's more painting than rendering based on 3d data.
Did you saw the second video? At first I understood that it was basically extracting 3D models in real time from the videos, but it's actually more than that. On the second video you can see how you can chance the street type or chance the scenery and it will adapt depending on the context of the scene. So it is more about "Learning" how a street or a tree looks, so you can then generate new content based on that data and integrate it to the scene.
If you watch the playback, it's akin to a load of cut-and-paste photos. There's clever stuff going on, for sure, but what is the image processing actually doing? TBH I don't think it's far removed from what devs already have. You can paint with object brushes and procedural texture brushes. If you want a photo style that isn't from real life or similar solid reference, you still need human imagination and creativity. ML will just make it easier to develop creative algorithms, potentially.