3D is still displayed 2D so why bother with 3D?

[Dregun]

I know the title is a little confusing and might make it sound like I'm talking about 2D side scrolling games or circa 80's tech. However I'm wondering if we are going about "rendering" all the wrong way. When you ask an artist to draw a ball they don't care what is not visible to the viewer, they do not draw the backside of the ball as well as the front side. When you ask an artist to draw something from a photograph they don't draw stuff they can't see, they don't draw an "entire" 3D world they instead draw a 2D version of it based on what they see.

You take a scene from any 3D game and really think of what is being done it seems like a lot of wasted time and effort are put into making a virtual 3D space when the output is only a 2D image. Could a process be made in which the game draws each frame as a 2D image like an artist would? When you think about film itself, its no different then a frame in any video game. The difference is that the film is only capturing what is visible, it is displayed to you showing you what you can see and
not storing data on what you can't see.

Best example I can think of is a football being thrown with almost the entire football field in the cameras view at the 10 Yard line..

3D way of drawing it would be to draw the entire field in Virtual 3D (terrain/grass, hash marks, both endzones and any deformations made to the field. Then the football is drawn with full pigskin texture, thread and any mud/grass stains. Now a "2D picture" is taken from the outputted 3D image to give you a frame of what just took place.

2D way of drawing it would be to draw in detail the grass in the foreground and slowly put less detail in the grass the farther it goes away. Draw the hash marks and slowly close the gap between them the farther they go out. Draw the football either small or big depending on its location at the time the picture is to be taken. The football will only show what would be visible so if mud is on the other side it is not drawn and you might not even see the pigskin texture if its too far.

Too me it seems as though more work is done drawing a virtual 3D image as supposed to using the power of these machines to draw a 2D image instead. If they can store 3D images in the game and have the machine interpret what they would look like combined in 2D wouldn't that make more sense?

Its very hard for me to explain my way of thinking, for that matter some might say I'm "not thinking" but hey, no IQ test required to use the internet

Dregun

 

[Hardknock]

Each frame of a 3D game is basically a 2D image sent to your television screen. Which is what you are describing.

 

[Hardknock]

2D way of drawing it would be to draw in detail the grass in the foreground and slowly put less detail in the grass the farther it goes away. Draw the hash marks and slowly close the gap between them the farther they go out. Draw the football either small or big depending on its location at the time the picture is to be taken. The football will only show what would be visible so if mud is on the other side it is not drawn and you might not even see the pigskin texture if its too far.

And this is done already in 3D. It's called LOD (level of detail).

 

[ERP]

Really what 3D is about is just finding a good way to present a 3 Dimensional scene in 2D, what to you seems wasteful, is actually pretty much the best way to do it currently.

There are other mechanisms to construct "3D scenes" from parts of 2D images, but they all have there own sets of limitations, do a search on image based rendering.

 

[inefficient]

Build engine says hi

 

[dodo3]

Really what 3D is about is just finding a good way to present a 3 Dimensional scene in 2D, what to you seems wasteful, is actually pretty much the best way to do it currently.

There are other mechanisms to construct "3D scenes" from parts of 2D images, but they all have there own sets of limitations, do a search on image based rendering.

I can't wait to see LG's 3D HDTV be made available for games.

 

[Nesh]

Its easier to do good and smooth animation with 3D than drawing frames of 2D sprites (I dont know if I understood what you are saying )

 

[Squeak]

2D way of drawing it would be to draw in detail the grass in the foreground and slowly put less detail in the grass the farther it goes away. Draw the hash marks and slowly close the gap between them the farther they go out. Draw the football either small or big depending on its location at the time the picture is to be taken. The football will only show what would be visible so if mud is on the other side it is not drawn and you might not even see the pigskin texture if its too far.

Too me it seems as though more work is done drawing a virtual 3D image as supposed to using the power of these machines to draw a 2D image instead. If they can store 3D images in the game and have the machine interpret what they would look like combined in 2D wouldn't that make more sense?

Look up:
MIP mapping
Level of Detail and Culling
Rasterization

Basically "store(ing) 3D images in the game and have the machine interpret what they would look like combined in 2D" is what is being done now, only they are stored in a very compressed way.

What you seem to forget somewhat is that the 3d scene that has to be projected on the 2d canvas has to be "calculated" in some way first, either directly by the real world or by the artists mind, imagining the 3d structure of the scene (how the human mind does 3d is actually a very very interesting question (that still has to be answered satisfactory)).

There are lots of other ways to build 3d images than the currently successful paradigm with polygons and 2d textures, but they all have the same "shortcoming" of being somewhat wasteful by using time on stuff that is never going to be seen by the viewer.
That's because the computer can never tell exactly what is going to show until the scene is actually finished.
You can collect all the data for the scene in a giant buffer, "deferring" the rendering to avoid the rasterization overhead but that's about it.

 

[archangelmorph]

I know the title is a little confusing and might make it sound like I'm talking about 2D side scrolling games or circa 80's tech. However I'm wondering if we are going about "rendering" all the wrong way. When you ask an artist to draw a ball they don't care what is not visible to the viewer, they do not draw the backside of the ball as well as the front side. When you ask an artist to draw something from a photograph they don't draw stuff they can't see, they don't draw an "entire" 3D world they instead draw a 2D version of it based on what they see.

You take a scene from any 3D game and really think of what is being done it seems like a lot of wasted time and effort are put into making a virtual 3D space when the output is only a 2D image. Could a process be made in which the game draws each frame as a 2D image like an artist would? When you think about film itself, its no different then a frame in any video game. The difference is that the film is only capturing what is visible, it is displayed to you showing you what you can see and
not storing data on what you can't see.

Best example I can think of is a football being thrown with almost the entire football field in the cameras view at the 10 Yard line..

3D way of drawing it would be to draw the entire field in Virtual 3D (terrain/grass, hash marks, both endzones and any deformations made to the field. Then the football is drawn with full pigskin texture, thread and any mud/grass stains. Now a "2D picture" is taken from the outputted 3D image to give you a frame of what just took place.

2D way of drawing it would be to draw in detail the grass in the foreground and slowly put less detail in the grass the farther it goes away. Draw the hash marks and slowly close the gap between them the farther they go out. Draw the football either small or big depending on its location at the time the picture is to be taken. The football will only show what would be visible so if mud is on the other side it is not drawn and you might not even see the pigskin texture if its too far.

Too me it seems as though more work is done drawing a virtual 3D image as supposed to using the power of these machines to draw a 2D image instead. If they can store 3D images in the game and have the machine interpret what they would look like combined in 2D wouldn't that make more sense?

Its very hard for me to explain my way of thinking, for that matter some might say I'm "not thinking" but hey, no IQ test required to use the internet

Dregun

To me the biggest problem with your suggestion is that your only considering one facet of the elements which make up a video game..

3D games work so well because they allow the developer to fully express a scene in an arbitary way.. The fact that only a fraction of the work done in building up that information to describe the "look" of the scene with respect to the point-of-view of the player is insignificant because the rest of the work has plenty of other uses..

For example, lets say we take your football pitch and place a warehouse in the centre of it.. If we place the view somewhere at the edge of the pitch looking towards the outside of the building then we can build up a description of the scene in 3D as you described (vertices, textures, co-ords, skydome etc..)
Now when we render we basically clip most of the unnecessary information away so that we only have to "Draw" what can be seen by the view.. This might be considered wasteful if the view position remained static throughout the course of this game but such a scenario is HIGHLY unlikely..

If we now move the view, the game can re-render without us having to do anymore work to consider the change in perspective.. Also we can add avatars into the scene and track there locations/actions/behaviours with respect to the scene information because we already have such information available.. We can add physics simulation rules and set them to act upon the objects/avatars/environment all with great ease because all the information relating to these entities exists in memory and is accessible.. There can be so much variation in this "model" of the scene and the vast majority can be done automatically without the need of any kind of addition/modification to the scene description because it is already "complete"..

To change the this method of "modelling" the world in favour of the one you suggested has no advantage other than if the view and all the entities in the scene were static.. As soon as somthing changes we would have to change the description of the scene and build up a new model because the information stored is incomplete and incapable of supporting such scene variation..

This is the primary reason video games moved from flat, unexpressive, heavily creatively-constrained 2D worlds to the 3D ones we have today..

Taken the approach you suggested seems more like a step backwards and would more than likely hinder the performance and amount of effort required to fully realise the kind of games developers are doing in this age..

 

[Lazy8s]

The processor wouldn't know how much smaller and more faded detail is supposed to look at a distance from the camera and what a particular orientation looks like from a perspective if it didn't calculate the 3D. As mentioned, the rendering techniques currently employed are people's best idea of how to minimize that calculation.

The back sides and fully obscured layers of detail don't have to be drawn, however. As also mentioned, that's one of the concepts behind deferred rendering.

 

[zidane1strife]

Really what 3D is about is just finding a good way to present a 3 Dimensional scene in 2D, what to you seems wasteful, is actually pretty much the best way to do it currently.

There are other mechanisms to construct "3D scenes" from parts of 2D images, but they all have there own sets of limitations, do a search on image based rendering.

A gai meta-algo could tweak the pixels like an artist and show photorealistic gphx of indefinite complexity, detail, etc, with ease. Just like a human painter but on turbo, you couldn't tell reality from fantasy once one of those was assembled, not to mention pretty much everyone'd be out of work

The world would finally open its eyes.

 

[randycat99]

The "2D approach to 3D" is essentially how fps games started, no?...Wolfenstein and Doom, anyone? If the goal was simply to achieve a static picture with various elements (such as a artist painter), the creator needs only to render the image for a single perspective and then apply all the depth cues to make 2D elements appear to exist in depth in relative to each other and to the viewer.

Now a moving, interactive game image presents some additional challenges. The perspective is no longer fixed. It can be anywhere, now that the player has full control to manuever through the game level (whereas the picture painting only has to depict one single state). So the game has to possess all of this 3D information in order to seamlessly render the image at any given viewpoint (where things can be viewed over a continuous range of angles), which changes at any given point in time according to the inputs from the player.

Potentially, this effect could be achieved with enough manipulation of simple 2D primitives, but doing this smoothly and seamlessly for any/every possible state in a game level would be far more painstaking and extensive than simply doing what we do now- store an actual mathematical 3D representation of the game level, render it from the current viewpoint, and then project that result onto a 2D plane to display on a monitor. The "3D way" is simply the most practical and natural way to derive an authentic depiction of a game world from any angle/viewpoint on-the-fly.

 

[Simon F]

The "2D approach to 3D" is essentially how fps games started, no?...Wolfenstein and Doom, anyone?

IIRC, apart from creatures, the world was 3D. Some simplications in choice of camera angle were taken that simplified the rendering, but other than those it was 3D.

 

[Bad_Boy]

I was actually thinking about this after watching the special features on Invincible (that football movie w/ marky mark ). Anyways, to make a long story short they showed how they made the football crowd fill the stadium from the footage of an empty one. Basically they had people do "cheer" movements in front of a bluescreen, and made sprites of their animation and filled the 3D stadiums with little 2D movies of each fan in the stadium. The "fake" crowd pretty much looked real thanks to movie magic. Of course in gaming, these are considered pretty much high res sprites.

But I would guess due to memory space, sprites of such high quality would not be possible? Or would it be processing power? Say we are talking about Madden 08 doing the same thing. Would it be possible to fill the stadium with 50K or so sprites that are pretty much 5 second movie clips on a 2D plane? I remember tech demos of the Cell being able to have a few dozen movie clips displaying at the same time, would this be much different?

Kinda hard to explain what I mean exactly. If you have the movie, check the special features on "building the Vet".

 

[Fafalada]

IIRC, apart from creatures, the world was 3D.

I would argue that Wolfenstein was a 2d world with 2d gameplay, displayed in 3d. The same kind of world and gameplay was displayed in 2d by some games before it (eg. Into Eagle's nest).

The "2D approach to 3D" is essentially how fps games started, no?

Depends how you define "started". There were numerous FPS and FPS adventure games released throughout 80s, most of them used vectors or filled polygons, and several actually had full 3d camera and freedom of movement. But popularity of the genre was another matter

 

[London Geezer]

I was actually thinking about this after watching the special features on Invincible (that football movie w/ marky mark ). Anyways, to make a long story short they showed how they made the football crowd fill the stadium from the footage of an empty one. Basically they had people do "cheer" movements in front of a bluescreen, and made sprites of their animation and filled the 3D stadiums with little 2D movies of each fan in the stadium. The "fake" crowd pretty much looked real thanks to movie magic. Of course in gaming, these are considered pretty much high res sprites.

But I would guess due to memory space, sprites of such high quality would not be possible? Or would it be processing power? Say we are talking about Madden 08 doing the same thing. Would it be possible to fill the stadium with 50K or so sprites that are pretty much 5 second movie clips on a 2D plane? I remember tech demos of the Cell being able to have a few dozen movie clips displaying at the same time, would this be much different?

Kinda hard to explain what I mean exactly. If you have the movie, check the special features on "building the Vet".

The problem with 2D sprites is that they are... 2D... If you ever want to see "behind", you won't see how the character looks from behind, you'll see something like a poster with a picture printed on it - even if the image is moving.

That's why this whole thread is a bit weird. If you want to explore a 3D space, even if ultimately it is displayed on a 2D screen, the only way to do that is with a 3D environment made of 3D objects. A 2D environment has no "depth" and therefore would not give you results that are "explorable in 3D", even though in the end it's all displayed in 3D.

One could argue that in the end, even what we see in reality "becomes" 2D, for our brains to then be "understood" as 3D: light hits our retina's, which is a 2D surface - the only hint for us that there is "depth" in our real 3D space is our binocular vision and how that works together with our movements so that our brains "understand" the concept of depth. But our retina's are still 2D planes which are hit by light.
Likewise, in 3D games it's our movement that makes our brains "understand" that we are exploring a 3D space. If not things would still be like those old games like Myst which were a series of 2D pictures with no 3D information whatsoever: those can't be explored like today's 3D games. If we were to explore those games, we would need one picture saved somewhere of every point of view possible in a 3D space, which obviously is an impossible task.
Polygons are the simplest way we have today of representing 3D space.

 

[Dregun]

I refrained from commenting until I had more opinions and views on the subject. I knew I wasn't explaining it very well but I didn't realize I did that bad of a job on it Anyways I'm going to quote this post and use it further explain my wary way of thinking.

Likewise, in 3D games it's our movement that makes our brains "understand" that we are exploring a 3D space. If not things would still be like those old games like Myst which were a series of 2D pictures with no 3D information whatsoever: those can't be explored like today's 3D games. If we were to explore those games, we would need one picture saved somewhere of every point of view possible in a 3D space, which obviously is an impossible task.
Polygons are the simplest way we have today of representing 3D space.

This is pretty close to what I'm talking about, its a completely new way of "rendering" as no 3D object is ever created, instead a 2D representation of the object is drawn on the screen. I'm not talking about a sprite per say, I'm talking about the machines using the power to "intelligently" draw a 3D scene in 2D like an artist would for each frame. The world would still be in 3D, the machine would just "draw" the backside of the object.

Im sure we all know enough about basic art to draw a 3D cube. You draw two off center squares and connect them with diagonal lines and remove the "non visible" lines. Spheres are simple circles to an artists with proper shading showing it has depth.. Artists have been showing 3D environments before electricity was discovered but when it comes to rendering we are focused on recreating the entire world in 3D just to take a "picture" of it to show as a frame. If an artists can draw the same scene from any view you asked on a canvas then how come we can't create software to do it on computers? We throw huge mathematical calculations at these machines to apply light and shadows to entire objects in 3D yet we can't use the same mathematics to tell the machine how to draw the image in 2D to represent it in 3D?

Too me it just seems like we are over complicating rendering for a medium that cannot display full 3D content. If we were dealing with Holographics then obviously polygons would be the best way to recreate a 3D world. But when displaying on a 2D medium why create a virtual 3D world? It seems like we "missed" a step, like going from inventing the wheel to inventing the motorcycle and completely skipping the bicycle. Just because the Motorcycle is better suited for long distances in short times doesn't mean the bicycle isn't important or serves a purpose.

If this doesn't make sense then a greater consumption of alcohol might put it in perspective.
Dregun

 

[zed]

If an artists can draw the same scene from any view you asked on a canvas then how come we can't create software to do it on computers? We throw huge mathematical calculations at these machines to apply light and shadows to entire objects in 3D yet we can't use the same mathematics to tell the machine how to draw the image in 2D to represent it in 3D?

Too me it just seems like we are over complicating rendering

(not laughing at you)
but the overcomplication would be how do want it to be, it would require some sort of AI greater than any so far devised, its like the film bladerunner, from memory where harrison ford take a photo of the back of a guys head + he saiz to the computer, calculate how he looks like from the front.
the current method of storing geometry as a collection of polygons is used since its the simplist method that exists for describing complicated geometry

 

[CosmoKramer]

http://en.wikipedia.org/wiki/Ray_tracing

 

[Shifty Geezer]

This is pretty close to what I'm talking about, its a completely new way of "rendering" as no 3D object is ever created, instead a 2D representation of the object is drawn on the screen. I'm not talking about a sprite per say, I'm talking about the machines using the power to "intelligently" draw a 3D scene in 2D like an artist would for each frame. The world would still be in 3D, the machine would just "draw" the backside of the object.

For an artist to draw a 3D scene onto a 2D page, they need a reference. Let's take a still-life as an example. The 3D objects exist in 3D space, and the artist takes that 3D data and compresses it into 2D by selecting a single viewpoint.

When rendering 3D graphics, the 3D scene is stored in computer and then compressed into 2D by selecting a viewpoint.

Both methods are identical. They require a 3D scene to exist. You can't expect a computer to draw a scene without having that scene in memory somewhere. How can it render an image of a house from any viewpoint without a 3D model of the house being stored somewhere for it to reference? What you asking is something that's utterly impossible. A human artist can use their imagination to guess at a scene, but even then if you have a somputer with an imagination and enough real-world experience to populate it with ideas, you won't get the accuracy needed in computer graphics.

Rather than thinking the current 3D graphics rendering is too complicated, think instead of how else you're going to do it. You want the computer to be able to draw a bowl with an apple and a banana in it, with a wine bottle next to the bowl, and to be able to draw that scene from any angle. How is it going to be able to do that without having the scene stored in 3D space?