Ah... okay, I still don't get the full picture here. I thought that you see the real world like you see it with your regular glasses (or your own eyes) so you can wear Hololens and play a game on the TV, like since time immemorial, and play it like that, so wearing Hololens would be always a seamless transition from the real world.No, maybe I didn't write that out clearly. The AR world is as large as you want it to be. From what I understand, the AR world is modelled within Unity, so anything outside the unity application won't show up unless it swaps over to a new unity level/instance. Your viewport is limited however. If you want to see everything in your viewport without having to look around, you're going to have to step back, or it's going to have to be smaller.
Say you play a strategy game like Age of Empires on the TV, and you see the TV as usual in order to play, but you could have VR windows to the sides sometimes, to check some buildings and unit creation..etc. Is it possible to play anything like that?