RayTracing?

[Davros]

Nvidia has released a tech demo
NVIDIA Apollo 11 Moon Landing RT Tech Demo
Heres some of the blurb
With RTX, each pixel on the screen is generated by tracing, in real time, the path of a beam of light backwards into the camera (your viewing point), picking up details from the objects it interacts. That allows artists to instantaneously see accurate reflections, soft shadows, global illumination and other visual phenomena.

Can someone explain the bold part, thats not backwards if I film something photons dont come out of the camera and reflect of objects until hitting a light source

 

[Scott_Arm]

Nvidia has released a tech demo
NVIDIA Apollo 11 Moon Landing RT Tech Demo
Heres some of the blurb
With RTX, each pixel on the screen is generated by tracing, in real time, the path of a beam of light backwards into the camera (your viewing point), picking up details from the objects it interacts. That allows artists to instantaneously see accurate reflections, soft shadows, global illumination and other visual phenomena.

Can someone explain the bold part, thats not backwards if I film something photons dont come out of the camera and reflect of objects until hitting a light source

The path of the beam of light is traced backwards from the camera, vs forwards from the light source. Ray tracing is always relative from the camera vs the light sources.

 

[Davros]

As someone who doesnt know how its done in graphics only how light works in real life that seems backwards
how do you even know there is a ray to trace or if there is a light source, or its intensity or colour . Do you calculate all possible rays
that could hit the camera and discard those that dont end at a light source ?
and when you get to the light source and discover its red do you have to adjust all your previous calculations to take that into account ?

 

[pcchen]

As someone who doesnt know how its done in graphics only how light works in real life that seems backwards
how do you even know there is a ray to trace or if there is a light source, or its intensity or colour . Do you calculate all possible rays
that could hit the camera and discard those that dont end at a light source ?
and when you get to the light source and discover its red do you have to adjust all your previous calculations to take that into account ?

Basically if a photon can go from A to B, then it can go from B to A. So it does not matter to the end result if it's calculated forward or backwards.
Now if you do it forward then it's very likely that a lot of the light coming out of a light source does not actually get into the camera, so it's wasteful to compute them. On the other hand, when shooting the ray from the camera, it's easier to compare the relative location between the point where the ray hit an object and a light source, so it's easier to determine how the light source affect the color of the point.
There are some bi-directional ray tracing algorithm for handling specific situations, but they're probably not fast enough for real time rendering.

 

[Davros]

I'm sure it seems perfectly logical to you but I'm struggling with it
lets take this : "when shooting the ray from the camera,"
1 : How do you know there is a ray to trace
2 : How do you know what colour and intensity it is
3 : How do you know what angle it hit the camera at

 

[Davros]

Asking just out of pure curiosity
If I raytraced this scene
would a or b appear on my monitor?

 

[milk]

I'm sure it seems perfectly logical to you but I'm struggling with it
lets take this : "when shooting the ray from the camera,"
1 : How do you know there is a ray to trace
2 : How do you know what colour and intensity it is
3 : How do you know what angle it hit the camera at

1: What does that mean? We are trying to solve what light got to each pixel, as such, there is at least one ray per pixel, if if is completely occluded, than that's our result: no light ever reaches our pixel.
2: Shaders. The thing about physics is that, for a fully known system you have a snapshot of, with all position, mass, speed, acceleration, etc variables known, you can predict both future and past states with perfect accuracy using the very same equations. Same for pathtracing. The same equations work in reverse.
3: A propper camera does not let light coming from random directions in. They try to approximate a pin-hole effect. So does your eye by the way. As such, for each pixel, you only have to trace initial rays from the angle the gets through our virtual pin-hole position. After first hit, more rays may be spawned though.
A naive raytracer will just spawn thousands of rays from the first hit randomly, to see if it hits a light or not, and shoot another thousand random rays from the secondary hits and so on. Of course that shoots a bunch of useless rays and as the resulting image is being constructed it starts out nearly black (very few paths ultimately found a light) and noisy (the patha that did find lights vs. the ones that didn't are very randomly distributed). That's why nobody uses a completely naive path tracer.

Enter the world of pathtracing optimisations.

The first, more obvious one: After you found your first hit, first and foremost, trace against all primary lightsources, this quickly gives you initial lighting info right from the start. If you wanna be even more optimal, you can prioritize the nearest sources, and maybe have an acceleration structure that allows you to discard occluded sources early before even shooting a ray. Of course, most path tracers work with area lights, and those need hundreds or thousands of rays to approximate properly, so this optimization alone is not enough for those. These will benefit more from the same kind of optimizations that benefit secondary illumination and GI in general.

Second most obvious optimization, use the material's BRDF to prioritize where your rays are shot first. Hit a mirror? Easy, your secondary bounce ray will go straight in the specular reflection direction. Hit a diffuse surface, bad luck, your ray can go anywhere. Shoot something in between, prioritize your random rays near the specular reflection, but not exactly at it.

Third most obvious optimization, keep track of how much potential energy your path could carry. Say, you are following a path that has bouced off of 10 different surfaces by now. You've solved their BRDF function at every step along the way, you know that by now, even if your ray ends up hitting a supernova, the resulting added energy won't cary over enough light to the final pixel to increase it's intensity more than 1%. Kill it. Forget that path for now, and prioritize paths that have more chances of influencing your render meaningfully.

Fourth obvious optimization: You've already solved many paths. Your framebuffer already has an understandable but still noisy image showing up. Analyse where the most noise is in screenspace, and prioritize shooting rays from those problem pixels.

Those are just a taste of what pathtracers do. To give you an idea. You can read the hundreds of articles and papers on the subject if you wanna go in-depth. It's trully facinating.

Path tracing does shoot many rays that end up being useless, but still only a fraction of useless rays of what you propose instead. Suppose we do the oposite. Suppose we shoot rays from the lightsources towards the world. Even for a tiny-ass scene like the sponza atrium. You have a world of say a couple hundred metres squared, by a couple dozen metres high, with rays bouncing off the sun across all parts and all directions of this space. How many of those end up actually hitting our small camera in the middle of all this? A fraction. A 0.00x kind of fraction, if not less. Probably much less actually.
That's why pathtracers start paths from the camera.


In realtime land though, they have been focusing a lot on optimizations that have just recently entered the toolset of offline renderers. Specially: Spacio-temporal Screenspace filtering/denoising of the final framebuffer. It comes from the assumption that neighbor pixels probably have similar lighting conditions, so one can filter a bit of the lighting from a pixel's neighbor to get some extra paths for cheap.
When you are doing that, distributing your rays so that they produce more evenly distributed dither across the screen also becomes more useful, because it produces easier to filter results.

 

[milk]

Asking just out of pure curiosity
If I raytraced this scene
would a or b appear on my monitor?

a

Light moves like rays in ray-tracing.

 

[imerso]

Davros: in standard raytracing, things happen reversed. Instead of shooting rays from the light sources, rays are shot from the camera.

In reality, the screen (yes, the flat 2d screen plane) is scanned from top left to bottom right, pixel by pixel, and one ray is shot in the camera direction.

Those rays have no color yet. By doing intersection tests against the world, if an object is hit, then a new ray is shot in the direction of a light source, to see if that point can "see" the light. If it sees the light, then it is lit by that light, else it is in shadow.

Then if the object's material is reflective, the ray is bounced (in a new direction that is the surface normal against original direction) and if it hits another object, it adds that other object's color.

When it finishes traveling, then we found that pixel's color and can start calculating the next pixel.

This is a gross explanation for whitted raytracing, but I hope you understood that each pixel in the screen becomes a ray, shot from the camera plane in the camera direction.

 

[imerso]

Lets not confuse raw ray tracing (whitted ray tracing) with path tracing, though. Ray tracing won't shoot thousands of rays from the first hit! Path tracing will, to achieve global illumination.

Path tracing is still a very slow process, and the reason for the noisy GI, while raw ray tracing is faster but does not include GI.

 

[imerso]

Milk: you're confusing ray tracing with path tracing.

 

[milk]

Milk: you're confusing ray tracing with path tracing.

Yeah, I was describing path tracing there, and used the term ray tracing instead of it a couple of times.

 

[hughJ]

You can think of ray tracing in similar terms as rasterization. With ray tracing you've got the camera and scene geometry in a world coordinate system, and you apply 3D transformations to your camera and its projected rays, and perform ray->object intersection tests to generate color values in your frame buffer. With rasterization you're instead applying a 3D transform to move the contents of the world into the camera's coordinate system, along with the camera's frustum parameters (perspective/FOV) such that the unclipped elements of the scene get squeezed into a rectangular box with parallel sides. At this point the polygon->raster conversion sampling is equivalent to shooting parallel rays from the screen's pixels into that rectangular box, and the Z-test is equivalent to the nearest-intersection comparison that's done in ray tracing. The coloring, shading and material properties you deal with after completing those intersections are the same thing either way (ray tracing can use the phong shading model, refraction, reflection, Schlick's approximation, etc just as you see in any D3D/OGL pixel shader code.) In this sense you can view the advent of SGI,opengl and consumer 3D rasterization as the result of an algorithm optimization that maximizes the sampling rate of that initial intersection test by reducing the problem space, but as a result it's poorly suited to tasks that fall outside that problem space.

In other words: what makes ray tracing powerful is the fact that you're transforming the camera's rays rather than the world -- this means you're able to perform further intersection tests from arbitrary perspectives with no added overhead, where as with rasterization you would need to re-transform the geometry of the world into a new perspective frustum for every vantage point of reflection, refraction, etc of any surface position and surface normal.

The hard part of ray tracing is the fact that you need to sort your world geometry (and continuously re-sort the dynamic geometry) in an acceleration structure, have that structure's entire contents resident in fast VRAM, and have a fuck-ton of compute to hit a particular threshold of samples per pixel per second. And even if you do manage to achieve all of that, you're still stuck with the fact that 99.999% of consumers are well below that threshold and still will be a decade from now.

 

[Davros]

Light moves like rays in ray-tracing

Thanks, so that means ray tracing doesnt take in account constructive and destructive interference.

 

[JoeJ]

1 : How do you know there is a ray to trace

3 : How do you know what angle it hit the camera at

You shoot one ray from the eye through each pixel of the screen. (Totally independent of scene / lights - just a matter of camera projection.)

2 : How do you know what colour and intensity it is

When this ray hits something, you trace a second ray from the hitpoint to the light source. If the light is not occluded, you shade the hitpoint accordingly.

This is very basic Whitted Raytracing but there are methods that shoot rays from the lights as well:
Bidirectional Path Tracing combines paths originating form both the camerea and the lights.
Photon Mapping shoots rays from the lights, stores the hitpoints as "photons" and later you gather nearby photons froam camera ray hitpoints to get global illumination effects.
Although those methods are typical for offline rendering, we already see some similar ideas in games and demos.

Also worth to mention: In games the first step of shooting rays through screen pixels is replaced by rasterization because it's faster. (Quake RTX being the only exception AFAIK)

 

[milk]

Thanks, so that means ray tracing doesnt take in account constructive and destructive interference.

Yes. Light moves like particles in ray tracing. If a cat is inside a box, it is always dead, because a computer cat is not a real cat. It has no soul.

 

[chris1515]

https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-overview

Good definition of raytracing

One problem with discussing ray-tracing is that ray-tracing is an overloaded term.

Ray-tracing is a technique for computing the visibility between points. Light transport algorithms are designed to simulate the way light propagates through space (while interacting with objects). They are used (in short) to compute the color of a point in the scene. Do not mix ray-tracing with light transport algorithms. They are two different things. Ray-tracing is not a light-transport algorithm. It is only a technique to compute visibility between points.

 

[Cyan]

Devlog July 2023 - MARTY'S MODS

www.martysmods.com

 

[JoeJ]

Asking just out of pure curiosity
If I raytraced this scene
would a or b appear on my monitor?

It's this (because your light is an area light):

the path of a beam of light backwards into the camera

Such statements are always bullshit, or at least inaccurate.
'rays transport light physically accurate like in the real world'... or such statements always reveal the writers have no clue, or attempt to use gross simplifications for the audience, treating them dumb. I kinda hate it.

But it's not easy to express it better. I'll try in two ways...

For the double slit scene you gave, we actually expect a result like this (ignoring quantum confusions):


I tried to draw gradients of white-red-white in Paint.

The reason for the gradients is this:
As we move from pixel to pixel of out image, we can see only parts of the spherical light through the slit.
First we see only the left side of the sphere, so we see only a small area of light, and this lesser light gives us the rosy color.
Then we see the center section of the light, but still just part of it - not the whole sphere. But the section we see has larger area, so we get red.
Then we see the right side of the light and get rosy again.

Illustrating what the left and middle pixel can see through the slit, but too lazy to draw the right. Sorry it's so small, you'd need to magnify:


So we want those smooth gradients because we can see smaller or larger parts of the light.

Bit RT is a binary visibility test which can not provide this information.
We may pick a random point on the light, trace a ray and test if the point is visible through the slit. That's the only information the ray provides.
(There also is NO LIGHT associated with a ray at all. Never. WE need to implement this association on the higher level algorithm which uses the ray visibility test among other things.)
To sum it up: The ray allows us to do point samples, but those points have no aera. We get the noisy image i've shown first from the randomized point samples drawn per pixel from the light sphere.

To get to the desired results we now have two options:
Blur the noisy image.
Or do MANY point samples per pixel and take the average of all.

But at this point we already have 3 concepts to generate our image: RT (to do visibility or intersection tests), the idea to use random samples, and some way to integrate those samples somehow so we get an approximation of area.
And we need to explain / mention all those concepts to associate RT with the application of generating images.

To avoid this rhetorical shortcoming you could use the term path tracing instead ray tracing for example. Because PT defines all those concepts, but RT does not.

Now the second example:
Imagine some light hits the red wall. And if the wall has some diffuse material, the light scatters around like so (no matter where it comes from):

I have drawn some of the infinite number of rays originating from one point on the wall, illustrating the light bouncing off the point on the wall.
Their magnitudes form a lobe. In this case, because we assume perfect diffuse material, the lobe has the shape of sphere.
The important observation here is that a ray can not describe the scattering function. Light goes into many directions, not just one.
So instead talking about rays, we want to talk about the function which describes the scattering distribution on all directions.
To approximate this function, we can use many rays, but it's clear the finite number of rays only tells a part of the whole story.
So again the concept of a ray is not enough to tell anything about optics.

Now the same example again, but it's a perfect mirror now, not a wall:

Light comes in, gets reflected and goes out. Only in this special and theoretical case a single ray is enough to describe the function. Because the shape of our lobe is (coincidentally) a line.

And because that's all what very first attempts of RT have implemented, the foundation of the current rhetorical issue was set back then.
If we could figure out a single sentence to say all this, we could probably address the issue...

 

[JoeJ]

Oh, while i'm at it, i would like to ask a related question...
I have observed some phenomenon in the real world defying what i believe to know about physics and optics.
What i see hints that light travels along curvy paths, not straight lines.
Yes, i'm serious. Here's what i saw and still see:

I was out the house to smoke a cigarette. While i look at the cigarette 30cm away from my eye, i see neighbors roof in the background. The structure of the roof forms a regular grid, which is why i noticed the phenomenon.
Here is an illustration, ignoring the angle of the roof, which does not matter:


Left is the near cigarette, and right is the grid formed by the roof, maybe 20m behind.
The grid is regular. But near the cigarette, it's compressed.
It's like the light travels around the cig. obstacle. The light actually bends.
And due to that, i can actually see grid lines which should be occluded by the cig.

Try it yourself. It's not related to temperature of the cigarette. I have closed one eye as well. Light still bends around it. I can go out and do it again. It fucking bends.
The effect is limited to a small area around the obstacle, i would say few millimeters around the cig. But it's easy to see if you focus.
(Use a pencil or whatever instead. And move it slowly a little bit left and right. Then you should notice a kind of lens effect, making it easy to focus and concentrate on the compression.)

Why is this?
I know black holes can bend light. But the effect of a cig. should not be noticeable.
I lack any explanation.