Can't argue that is the general thought on the subject. It seems a bit pointless, IMO, to even bother with getting bigger and bigger screens just so you can sit proportionally farther away and the screen becomes perceptibly the same size (the one exception being that you move into a bigger place). When I shop for a bigger screen, I am more than likely planning to view it at nearly the same distance I am currently so the screen is that much more immersive. That's why I'm paying for it- to see a bigger image.
The rule of thumb for viewing distance IIRC is 3X the height of the screen or something like that. Most people who go to the movie theater will get the best picture from the middle to the back seating sections not the front as your eyes will get a real workout in the horizontal and to a lessor extent the vertical direction s if you sat at the front. Same with a big tv in the living room. Also when you say you'll be sitting at nearly the same distanct when moving to a bigger tv means very little when you don't state initial and final screen sizes. If you're moving from a 32" to a 36" then you won't be sitting that much farther if at all obviously.