That's what they want you to think, a toolchain, even a complex one, could be millions. But the cost of lock in is potentially almost infinite. The only "reason" old code doesn't get updated is because "well it runs on the old harware and we have the old hardware so why bother?" By your logic the systems still running Windows 3.1 do so because it's somehow too expensive to re-write the software.
DNN is extremely hardware intensive, it's not like 4 Titan X's to a case is somehow inexpensive. And the toolchain Nvidia provides wouldn't be a huge cost to replace. It's the software built on it that took all that money, and only now that the money is spent, and everyone realizes they just locked themselves into a single IHV just to save a bit of time up front, do people rationalize that it was "worth it" the whole time.
The exact same bullshit has always applied, to 3DFX Glyde, to building a Direct X based rendering engine instead of a higher level open abstraction, to etc. etc. It's brilliant on Nvidia's part, it worked. Someone, anyone, could put out a GPU with 64gb of Ram and quadruple the performance of a Nvidia card and DNN builders would still hesitate. It's why Nvidia did what it did in the first place. Vendor lock in, software, hardware, etc. pretty much always costs more than it's worth. It's only after you've gone down the hole too far to get out that most tend to realize their mistake.
