Machine Learning: WinML/DirectML, CoreML & all things ML

Unity's done a great job integrating ML into their product so far and it makes sense for them to have a layer that can provide ML functionality without any platform specific frameworks. That said, the DirectX platform does have some unique hardware acceleration support in DirectML. At this year's GDC we announced the public release of the DirectML API and Unity announced support for DirectML, leveraging it where they can for increased performance:

https://devblogs.microsoft.com/directx/gaming-with-windows-ml/
https://devblogs.microsoft.com/directx/directml-at-gdc-2019/

Thanks,
Max McMullen
Development Manager
Compute, Graphics, & AI
Microsoft

 

nice! Unity is getting pretty awesome. What a fantastic feature for developers.

 

Is it too much to ask for some freeware where I can just input a batch of images/textures/etc., select a SuperScale amount, and presto-output-folder?

Kind of curious to see if MS can have that as an option when taking screenshots on Xbox or Win10, for example. It'd be like having Ansel (so hot right now) without needing to inject it or have developer intervention (only select games).

I'd love to see an experiment for 90s 2D/sprite games. Is performance there for real-time superscaling where older-school games are in the 320x200-800x600 range I guess they'd have to be on D3D first... and maybe there'd be crazy artefacting in motion around moving objects (e.g. isometric games). :s zoidblerg.

 

not quite just a freeware with some customizations
I suspect someone can make the tool like that. what are you looking to do?

 

well considering it took over a minute on a single 256x256 sized texture on my admittedly inboard graphics I say having this running at good framerates is a long way off.
Also it doesnt handle Jpg artifacts well, just makes them worse, so prolly not the best for photos, but old CGI yes could be good

 

I think a more workable way of doing this would be to rip the game's source art, superscale that up, and inject the higher res art into the game. Some emulators allow such fan-made texture-patch injection.

 

Very quiet about DirectML, any news?

 

its released. It's up to developers to take advantage of it.

 

Found out yesterday. .

 

Announcing AMD Support for GPU-Accelerated Machine Learning Training on Windows 10
https://community.amd.com/community...nounce-en-email&utm_medium=email&utm_term=btn

 

Pretty fun thing here.
Background is static. Only players are AI generates.

 

Hey long time lurker, first time poster. But I had a question on this whole ML/DLSS type stuff. If this isn't the right thread, I apologize and would appreciate it if you could point me to the right thread.

Anyways I was wondering that for the DLSS implementation of AI upscaling, AFAIK it starts on the low-res 2D image as the input to the algorithm, but doesn't that seem like an inefficient way to go about upscaling and filling in detail? Like it mostly works well and gives an amazing performance boost compared to native rendering, but to me as a layman it sounds too inefficient to become the standardized method of the future as the algorithm doesn't have a lot of information that it "knows" since it is just working off of a 2D image. Like if it's just seeing a pattern of pixels but doesn't have a label to say this is an office chair or the main characters pony tail or whatever.

I would think that a better implementation is if the game engine tells the algorithm which object it is working on as well as it's location, size and orientation on the screen. For things like an explosion effect, foliage, semi-transparent clouds, particle effects, etc. Those types of visuals are really dynamic and I can imagine it tricking the DLSS algorithm or at least making it less efficient. I dunno, just a question I wanted to ask as there's a lot of hype about Tensor cores and them being crucial to graphics in the future but I'm skeptical of it the same way I was of SSAA and Physx fifteen years ago. It just feels like there's a better way to go about it.

 

That's not the only input.

• Current low resolution frame
• Previous frame
• Motion vectors

Are all inputs into the ML upscaling.

 

DLSS is 2 fold. First it’s job is to anti alias. The second job is to upscale.

the AI is responsible for detecting aliasing in the image, this is how it is trained. Then using 16K images as training, it tries to figure out what the antialiasing would look like if 16K Super sampling was done on this image and reverted back to the source resolution.

per object would be too slow and it would be too difficult.
Once it’s done with the anti aliasing, it then uses a second AI NN to upscale from the target resolution up to 4K.

the speed gains comes from rendering at a lower resolution (1080p) vs native. By doing so you’re working with 4x less pixels. And I’m general 4x less workload. When we continue to layer more things on like ray tracing which has a massive
Costs with more pixels (as each pixel will
Shoot rays) the costs keep going higher. Keeping the resolution locked at 1080p and using AI to extrapolate the rest and shifting it to 4K is faster than raw calculations, if it weren’t, there wouldn’t be any speed gains.

Hope that Helps