Research at NVIDIA: The First Interactive AI Rendered Virtual World

This is the type of thing that gets me really exited for the near future. This will really help to improve the graphics quality and lower the production cost and development time for games.

Nvidia has created the first video game demo using AI-generated graphics

“It’s a new way to render video content using deep learning,” Nvidia’s vice president of applied deep learning, Bryan Catanzaro, told The Verge. “Obviously Nvidia cares a lot about generating graphics [and] we’re thinking about how AI is going to revolutionize the field.”

“The structure of the world is being created traditionally,” explains Catanzaro, “the only thing the AI generates is the graphics.” He adds that the demo itself is basic, and was put together by a single engineer. “It’s proof-of-concept rather than a game that’s fun to play.”

https://www.theverge.com/2018/12/3/...deo-game-graphics-nvidia-driving-demo-neurips

NVIDIA Invents AI Interactive Graphics

"Twenty-five years ago NVIDIA transformed the computer graphics industry by building the first GPU, the modern day tool of the da Vincis and Einsteins of our time. Now, a new deep learning-based model developed by NVIDIA researchers aims to catapult the graphics industry into the AI chapter.

Using a conditional generative neural network as a starting point, the team trained a neural network to render new 3D environments, after being trained on existing videos.

This AI breakthrough will allow developers and artists to create new interactive 3D virtual worlds for automotive, gaming or virtual reality by training models on videos from the real world. This will lower the cost and time it takes to create virtual worlds.".....

https://news.developer.nvidia.com/nvidia-invents-ai-interactive-graphics/?ncid=so-you-ndrhrhn1-66582

This one shows how they can edit and configure a scene. Pretty awesome stuff in my opinion and really shows how flexible this technology will be.

 

Did you saw the second video? At first I understood that it was basically extracting 3D models in real time from the videos, but it's actually more than that. On the second video you can see how you can change the street type or change the scenery and it will adapt depending on the context of the scene. So it is more about "Learning" how a street or a tree looks, so you can then generate new content based on that data and integrate it to the scene.

 

This latest version has better consistency between frames.