Photon Mapping

Discussion in 'Rendering Technology and APIs' started by chris1515, Jun 7, 2019.

  1. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    It seems we are on the verge to have one photon mapping GPU maybe inside PS5 or if it is not PS5 maybe on an NVIDIA next iteration of RTX.

    http://casual-effects.com/research/Mara2013Photon/Mara13Photon.pdf

    Nvidia search about the different photon mapping method on a Geforce 670 in 2013, the best one is 2d tiles on a GPU

    https://patents.google.com/patent/US9633471B2/en

    Sony patent with hardware 2d tiles photon mapping, reading the patent it seems rasterizing is only used for direct illumination and visibility for shadow. Full indirect photon mapping lightning

    http://fileadmin.cs.lth.se/cs/Education/EDAN30/papers/photonMappingTutorial-SIG08.pdf

    It goes much faster than path tracing, sometimes two orders of magnitude faster. In SIGGRAPH 2008 they did a rendering demo needing 100 rays per pixel to render in 5 minutes in path tracing to be noise free, took 5 seconds and 10 rays per pixel in photon mapping without blur.

    Advantage:
    _ Much faster than path tracing/raytracing even simple Whitted raytracing and with better quality because it can render caustic. Photon tracing pass is like raymarching.
    _ fully independent of geometry scene complexity
    _ In 2D tile method resolution will be equal to what you will be able to render into the direct illumination solution rasterizing or raytracing. The limitation is the number of photons you generate during the photon tracing pass, the more you can trace, the best the quality will be.
    _ Easy to integrate inside an existing rasterizing or raytracing pipeline
    _ In 2D tile possibility to use tile rendering inside on-chip memory, I believe this what Sony is doing reading the patent
    _ No need to do some denoising when the number of photons is too low the light blur, like into the REVISION 2019 raytracing 4k winning demo.

    Problem
    _ KD-tree construction is slow(probably generation with something like scene generation in Power VR GPU)
    _ Can't do glossy effects like eye reflection or certain refraction. Can't do direct illumination. The two effects demand too much photon.
    _ Difficult to parameter the number of photon for each surface, it kills photon mapping as a global method for offline rendering. This is only used for some effect like volume rendering where it is great and much faster than path tracing. Importance sampling can be a solution to this problem.

    The year after the 2015 hardware 2D tile Photon mapping patent, Sony Interactive Entertainment hired an employee of Imagination/Power VR which did his master thesis on Progressive Photon mapping using CUDA in a GPU. He is a raytracing/Photon mapping specialist...

    https://www.linkedin.com/in/carlovloet

    https://carlovloet.wordpress.com/

    On raytracing gem book there is a chapter about Photon Mapping on RTX with a 2080 TI but it does not use the 2D tile method.

    https://github.com/ananthaks/DXR-PhotonMapper

    A photon mapper on Titan X, slowest part kd tree construction. I read the Kun Zou 2008 paper about KD-tree construction on GPU I don't think this is fast enough, the GPU is only as good as a multicore CPU.

    If someone resolve the KD-tree construction this a much better solution than raytracing and probably the only way to solve it is to use something fast enough and able to build BVH every frame like the scene hierarchy generator of Power VR but for KD-tree.

    http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.118.6575&rep=rep1&type=pdf

    Photon mapping pass will be slower with more photon on a PS1 scene at same resolution than a scene of the Avatar movie, this is not a joke because this is fully decoupled from scene complexity, you can use tesselated, displacement shading or any form of complex geometry or primitive this will not be the problem and the limitation of the photon mapping pass. It will be the number of photons linked to the number of ray.

    2D tile Photon mapping has less dependancy on resolution than other photon mapping method and probably other global illumination method. The resolution will be the one you can reach with the direct illumination rendering pass.

    In offline rendering the biggest problem of Photon mapping it was too difficult to parameter automatically to know the number of photon you need to use for each surface and it took too much artist time . But the inventor of the method told it is possible to solve problem with importance sampling but I think offline rendering choose path tracing without doing more research about photon mapping only more power was needed after all. Artist time is too precious.
     
    #1 chris1515, Jun 7, 2019
    Last edited: Jun 7, 2019
  2. JoeJ

    Regular Newcomer

    Joined:
    Apr 1, 2018
    Messages:
    394
    Likes Received:
    475
    Maybe it's worth to add the incompatibility between PT and Photon Mapping has been resolved: https://cgg.mff.cuni.cz/~jaroslav/papers/2012-vcm/

    But more complex geometry makes tracing photons more expensive as always?
    Being independent of screen resultion also is an advantage that can be brought to almost any other technique as well. But i need to read about this 2D tile tech which is new to me.
     
    BRiT and chris1515 like this.
  3. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    It seems reading all paper I read nothing just the number of photons is the limitation. I hope the next step is too solve the low geometry on realtime graphics from my point of view.

    Edit: A guy with enough mathematical and physics knowledge explains me photon tracing pass is like raymarching tracing inside a volume. I was by intuition knowing it is like tracing inside a volume but he was knowing much better than me.
     
  4. Globalisateur

    Globalisateur Globby
    Veteran Regular

    Joined:
    Nov 6, 2013
    Messages:
    2,823
    Likes Received:
    1,576
    Location:
    France
    I don't know, I am very skeptical about the potential use of that tech for Ray tracing in a console.

    Usually in those patents they identify every possible use cases, like for videogames, gaming console, computer, any devices and such. Here they only talk about 'image processing device', 'computer-readable recording medium' and 'computer program'.

    Shouldn't it be more suited for camera recording devices and such ?
     
    PSman1700 likes this.
  5. JoeJ

    Regular Newcomer

    Joined:
    Apr 1, 2018
    Messages:
    394
    Likes Received:
    475
    I think different people may have different ideas of what makes PM distinct from other techniques, personally i see it as a mix of multiple ideas for optimization, which where in the original paper:

    Can approximate indirect diffuse by gathering nearby photons. This gathering can be done like the initial proposal of a kD tree just for the photons or using any other spatial data structure. If we do it in screenspace the process would be very similar to current denoising if not the same.

    Can help Monte Carlo methods to find better ray directions, e.g. with a hint like: Nearby photon knows it is in shadow form light source X, so ignore that one. (At this pint we need somthing like PT and PM, likely that's too much to be interesting for realtime?)

    Can do caustics easier than naive PT because it stats at the lights not at the eye. (But if we aim for a generalization of all RT algortihms we end up at works like the paper i've linked above, while 2080 Ti barely manages naive PT in Quake 2 - so likely that's not the focus yet.)



    So i conclude it's mainly indirect diffuse that's interesting here for games.

    The Sony patent (i do not really understand the language and refuse to try harder :) ) talks about 'an image processing method to construct virtual 3D space'. Maybe this just means 'build AS and do RT' but make it sound as familiar as a traditional draw call.
    PM is technically RT, so i don't see anything fancy behind those words other than accelerating RT.
    But maybe PM raises the need for flexibility in a good way: You need to bin photons to the scene, so maybe accelerating binning or opening up AS to be reused (adding region queries not just ray intersection). Could be all FF and blackboxed as well ofc. But could also be a set of very nice building blocks.

    The point 'detect low luminance regions' is quite uncler to me. Maybe it is used to find regions that require more photons to become stable: Likely i get this wrong.

    For the gathering they mention something like getting a reduced set of photons per pixel. I'm confused here too, but it all sounds like focus on the indirect diffuse thing.


    A black box to accelerate just PM but nothing else sounds not reasonable to me - much less than RTX, which allows to implement PM, PT and whatever.
    But if it would be just the building blocks they would not need to mention PM at all in the patent, so that's all a bit odd.

    That's a bit off - firstly it does not matter much which form of spatial acceleration structure you use, and second Photon Mapping is raytracing, not something else. So both Path Tracing and Photon mapping require the same RT, and maybe you want to reuse acceleration structure silicon for both.
     
  6. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    I did not say you don't need ray intersection hardware for doing Photon Tracing efficiently It is a two passes algorithm with a raytracing pass. Maybe they will do the ray intersection with other things. Maybe they will take Power VR technology for example. It is possible to license the technology

    Edit: and when I talk about KD tree, It all about the KD tree Photon maps looking like a point cloud. Last year Sony was hiring for R&D people with GI knowledge and Volume rendering knowledge it was the two first point of the job offer.

    Edit: In the Nvidia paper, it looks like exactly the same algorithm. The 2D tile part has much more explanation inside the Nvidia study. And this for Nvidia is the best Photon Mapping method on a GPU.

    Edit2: The patent only looks like one pass the one describe inside Nvidia study. Nvidia study bypasses all the raytracing part. And there is no raytracing document reference inside the Sony patent. And they talk about another rendering part not describe inside the patent. The patent and the Nvidia study are only about the shading part. I think Sony will let other people doing the raytracing part.
     
    #6 chris1515, Jun 7, 2019
    Last edited: Jun 7, 2019
  7. JoeJ

    Regular Newcomer

    Joined:
    Apr 1, 2018
    Messages:
    394
    Likes Received:
    475
    Ok, it sounded like you would see PM as a faster alternative to RT so i pointed it out - did not want to sound smart.

    I don't think they would license tech, but maybe that's the reason the patent only covers PM but not related RT?
    Technically it should be possible to accelerate both RT and PM on a small chip because both is about AS and binning, and they had enough time.
     
  8. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    Maybe they will try to do it themselves but it is a possibility now at least than Power VR open a licensing program.

    EDIT: I think Nvidia will maybe add the possibility in another iteration of RTX because it is an implementation of the 2D tiles describe in Nvidia study...

    EDIT2: and when I said decoupled of geometry this the photon map pass...

    The raytracing pass is dependant of scene complexity but it seems you can go with less ray than path tracing for example...
     
    #8 chris1515, Jun 7, 2019
    Last edited: Jun 7, 2019
  9. JoeJ

    Regular Newcomer

    Joined:
    Apr 1, 2018
    Messages:
    394
    Likes Received:
    475
    Unlikely. First they will add HW BVH build to support more dynamic scenes, LOD and compatibility mesh shaders... as i hope. Would be super at least :)
    And second they might add ray reordering if they don't have yet.
    PM would benefit from both.

    Yeah, but you still need incoherent paths, and again each one provides only incomplete information so you need many.
    But that's surely the main argument. Never implemented PM so i don't know how less rays it might be, but...

    "It provides the same flexibility as general Monte Carlo ray tracing methods using only a fraction of the computation time."

    Is this a claim from the time when the PM paper came out?
    Veachs works came later and brought all those improvements on multiple importance sampling, bidirectional PT, metropolis light transport. And they made PT a whole lot faster.
    Even if those improvements are not practical for low realtime ray budgets but PM is, PMs advantage over naive PT is that it combines multiple samples (or photons) - exactly what denoising enables as well.

    So i remain doubtful. HW RT is one thing, but HW tuned to one certain algorithm out of many? Would be something new. We'll se...
     
    OCASM and PSman1700 like this.
  10. PSman1700

    Newcomer

    Joined:
    Mar 22, 2019
    Messages:
    65
    Likes Received:
    23
    So Nvidia ditched the idea all the way back to Kepler, after testing and exploring this method, and never developed this tech any further even today, and went with another tech for a reason?
     
  11. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    https://aras-p.info/blog/2018/03/21/Random-Thoughts-on-Raytracing/

    From what I read for HW build BVH and ray reordering, they will probably need license some patent from Power VR. Reading the site of Imgtec patentable is a word used everywhere. I am not sure you can use HW build BVH or Kd tree without needed to pay Imgtec.

    https://graphics.stanford.edu/courses/cs348b-00/course8.pdf

    Because after collecting the photon you need to organize it inside a kd tree and this part is slow... If you can generate it fast enough the problem is solved.

    The definition of photon mapping inside road to path tracing from Pixar

    The reason it fail as a general global method in offline rendering
    Edit:But I agree with you for correctness bidirectional path tracing mixed with photon mapping is the best solution after we are in vdieogames I think Photon mapping can be a solution before we have more power.

    https://www.ci.i.u-tokyo.ac.jp/~hachisuka/starpm2013a/Combining_PM_and_BPT_annotated.pdf
     
    #11 chris1515, Jun 7, 2019
    Last edited: Jun 7, 2019
  12. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    Double post
     
    #12 chris1515, Jun 7, 2019
    Last edited: Jun 7, 2019
  13. PSman1700

    Newcomer

    Joined:
    Mar 22, 2019
    Messages:
    65
    Likes Received:
    23
    So why Nvidia didn't bother with it since?
     
  14. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    Because Imgtec seems to have patent for HW build BVH or KD-tree data structures in hardware you need to find some financial arrangements with Imgtec and it is officially possible since 3 May 2019 if I remember well.

    This is probably the reason Nvidia use compute for build it BVH for raytracing currently and doesn't have ray reordering. Nvidia engineer are not stupid, they know they need to reorder the ray for coherent memory access.

    Edit: Imgtec launched a raytracing GPU 4 years before Nvidia :wink:

    Edit: And read the photon mapper they tried to do on Titan X slowest part build this fucking KD-tree.

    Edit3: I wait the SIGGRAPH 2008 document about the path tracing demo against photon mapping. We can see the type of path tracing used if this is pre or post Veach 1997 thesis.
     
    #14 chris1515, Jun 7, 2019
    Last edited: Jun 7, 2019
  15. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    This is a video from SIGGRAPH 2008 one of the guy Henrik Wann Jensen is the creator of Photon Mapping 02 hours 48 it is midnight here I will see it tomorrow.

     
    #15 chris1515, Jun 7, 2019
    Last edited: Jun 7, 2019
  16. JoeJ

    Regular Newcomer

    Joined:
    Apr 1, 2018
    Messages:
    394
    Likes Received:
    475
    Makes sense, but then you have to consider other alternatives as well like radiosity method which i am using: No need for a gathering step so no additional kD tree required. Iterative solver is a natural fit for anmiated scenes not just single images. Memory requirements justified by getting infinite bounces for free. Shading just texture lookup. Directionality can be added to support rough glossy specular. No noise or blotches.
    It's surely faster, but harder to implement in both engines and tools. (Remember Enlighten implementation in Battlefield 3: They used static light mapped proxy geometry - that's more work than just RTX on.)

    There are too much options to be certain, and picking one for FF HW would be a risky and limiting decission.
     
  17. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    Yes probably on PC side maybe less risky for a console hardware where the most important is performance for the next 5 years. If it is the Sony choice.
     
  18. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    One point for you this is against naive path tracing and the best case with caustics. But I was knowing this was a scene with caustics because I see some more complex scene with photon mapping this faster than path tracing but only 4 to 10 times times for example. Not bad but not as good than with caustic.
     
  19. milk

    Veteran Regular

    Joined:
    Jun 6, 2012
    Messages:
    2,858
    Likes Received:
    2,384
    What makes PM "independant of scene complexity" is the second pass that sorts out all the photons statistically across many screen pixels. Effectivelly, that is exactly what new denoising techniches are doing in PT now. So, PM does need less rays than naive PT, but in real time, we are already doing sparse PT with a denoising pass, not naive brute-forcey PT, so the amount of rays needed by a hypothetical PM solution can't be much less. In short, modern PT learned a few tricks that were once exclusive to PM, evening them out along the way.
     
  20. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    3,342
    Likes Received:
    1,937
    It means you don't need tensor core to doing it but you use other things. You trade tensor core against another hardware.

    It is more a question of which one will have the more efficient die size.
     
    #20 chris1515, Jun 8, 2019
    Last edited: Jun 8, 2019
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...