The problem with smart devices is their life span. I have a bluray player by sony which my parents have set up in their kitchen. We used to get hulu on the box and now we don't since Sony stopped supporting it a year ago. We replaced it with an amazon fire stick.None of these types of devices (Sony's, Amazon's, Google's) have exactly set the world on fire. I don't anticipate anything being different with Microsoft's. Especially considering every TV sold today has "Smart" functionality.
I can access all streaming media straight from my TV and the layout is pretty, functional, and (fairly) quick. Needing to switch to another HDMI is play basic (?) Windows games on my TV definitely doesn't appeal.
So if all of my streaming media works straight from my TV and if I want to play games, I switch to either my WiiU or my PS4, what does this device offer me that I cannot already do both more easily and probably with much more powerful hardware?
A lot of those smart tvs will get a lot dumber and once people are burnt on them they will want something that they can upgrade outside of the tv eco system.