Writing a CPU Raytracer

Discussion in 'Beginners Zone' started by Scott_Arm, Mar 25, 2018.

  1. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    I'm going to write a cpu ray-tracer by following the Ray Tracing in One Weekend series of ebooks by Peter Shirley. I may pick a different language from C++ (Rust, Apple Swift?), and make some implementation changes as I go along. I may try to implement it in a data-oriented (SOA) way, if I can. I'll post in this thread as I work on it. The books are really cheap for Kindle, and they look great.
     
    Rodéric, Lightman, AlBran and 5 others like this.
  2. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    I initially started in C++ because I was reading Swift docs and unable to figure out how to write to a file. Later on I realized it's because file io is not included int the Swift docs. Had to read about the Foundations framework from Apple. Now I think I know how it'll work, so I'll move on in Swift, just for a first experience with the language. I'll probably try to do everything with structs, but Swift is interesting in that all structs are pass by value, and all classes are pass by reference. Apparently structs are much much faster in most cases. Edit: structs on stack, classes in heap. Makes sense. Still not sure about passing all structs by value. If you pass by reference, there doesn't seem to be an equivalent to const to protect the reference from being modified.

    Another thing I'll run into is some implementation changes. The ray tracing books use a material system with a base abstract material class, so I'll probably implement that as a protocol in swift instead.

    The book also includes all of the math you need, but I've been reading Eric Lengyel's Foundations of Game Engine Development vol 1: Mathematics, so I'll probably implement a lot of what's in that book, even if it doesn't end up being needed to complete the first basic ray tracer.
     
    #2 Scott_Arm, Mar 26, 2018
    Last edited: Apr 3, 2018
    Billy Idol and Rodéric like this.
  3. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Lol, so I messed around with writing to a file in swift. Figured out how to use FileManager to get a URL to a file, and append my new filename. The thing I don't get is how to append to a file. It's probably obvious, but I wanted to move on with working on the actual ray tracer.

    If I do something like the following, it'll overwrite the file each time it goes through the loop.

    var text: String

    for loop in 0..9 {
    text = String(loop)

    do {
    try text.write(to: fileURL, atomically: true, encoding: String.Encoding.utf8)
    }
    catch {
    }
    }

    I ended up just appending my data to a String and writing once at the end. Writing my ray tracer output to a ppm file.
     
    Rodéric likes this.
  4. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Implementing some of the vector math I'll need. Operator overloading seems to work as expected. You can inline with the @inline attribute in Swift, so I'm inlining a lot of functions. Not sure of the performance implications of all of the structs being passed by value. They're allocated on the stack, where classes are allocated in heap memory. No real hangups. Just going slow and reviewing the vector math and comparing the implementations in Eric Langyel's math book vs Peter Shirley's ray tracing book. Look pretty much identical, and even overload operators in the same way. Haven't tried overloading the [] operator in Swift yet.

    When I get to materials I'll probably implement a protocol and structs, instead of an abstract class and sub-classes. Not sure if that is really the best way to do it. In the back of my mind I'm thinking about data-oriented and SoA. I'm not sure if this book series will touch on multi-threading at any point.
     
    Rodéric likes this.
  5. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    And I have a ray-traced sphere. 200x100 pixels. Anti-aliasing is next.

    upload_2018-3-27_23-39-6.png
     
    Rodéric, Lightman, AlBran and 2 others like this.
  6. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    So far it's been surprisingly straight forward. I can already tell that the real hangup for me will be on the math side. This tutorial is using pure mathematical calculation of a sphere, vs an actual mesh or other form of geometry. Not sure where the book series is going to end up.
     
    Rodéric likes this.
  7. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Quickly added calculating surface normals to the sphere. Next steps will be multiple spheres, anti-aliasing. Then I think the materials chapter is after. Nice easy book. Pretty interesting.

    upload_2018-3-28_19-58-16.png
     
  8. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Swift is an interesting language. I kind of like the syntax. I think it's a little more clear than C++. Xcode seems to be very helpful in pointing out errors, so I've gotten away without making any dumb mistakes. I'm a little unsure about the performance of things like their dynamically scaling Array. I got to a point where the C++ sample in my book was using a obj **ptr, for a list of objects, so I'm substituting a dynamic Array type in Swift instead. Maybe I'll play around with profiling at some point to see how performance between Swift and C++ compares. I think the source code for this book is available online, so I should be able to dig into it. The only major departures in my implementation right now is I'm doing everything with structs instead of classes. Found out I should be able to pass a struct by reference explicitly: func dilly(x: inout megastruct) -> void {}, then call the function as dilly(x: &mystruct), or at least that's how I think it should work.

    Tonight I should have the next step working, which is rendering multiple spheres in the scene. After that I'm on to anti-aliasing and then materials. Materials will be a big decision point, because I'll have to choose between an abstract base class with multiple sub-classes for each material, a Swift protocol with structs implementing the protocol, or one big material function with all of the available material parameters. The book uses the abstract base class.

    I'm definitely struggling a bit to see how I'd implement this in the data-oriented way. I've been thinking about it as I've gone along, but it's definitely hard not to think in an object-oriented manner, since that was what the professors evangelized to me in school. Maybe I'll go through once, and then see if I can start over in a data-oriented implementation, after fully understanding everything,
     
    Rodéric likes this.
  9. Arwin

    Arwin Now Officially a Top 10 Poster
    Moderator Legend

    Joined:
    May 17, 2006
    Messages:
    17,568
    Likes Received:
    1,126
    Location:
    Maastricht, The Netherlands
    Functional programming is a good way to help you to learn to think in a more data driven manner. Basically though in general it is a matter of thinking about where data originates and where it goes, and then imagine that there is a conveyer belt in a factory that does all sorts of things to the data before it reaches the end-point. Once you visualise that, you can then think about if there are parts of the data that can go on separate conveyor belts so that work can be done in parallel.
     
    Rodéric, iroboto and Scott_Arm like this.
  10. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Lol, latest MacOS preview corrupted and I lost my project. Good times. Shouldn't take more than a few hours to write what I lost. Years and years of betas abd I've never had that happen. Got too comfortable.
     
    Rodéric likes this.
  11. milk

    Veteran Regular

    Joined:
    Jun 6, 2012
    Messages:
    2,336
    Likes Received:
    1,716
    My condolences.
     
    Rodéric likes this.
  12. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    I think it's one of those things, where the universe fucks with you. Only time I've ever had a hard drive fail was once in school when I stayed up all night working on an assignment and then my hard drive died in the morning when I was going to print out.

    But I did manage to save my work. Forgot I had it saved to icloud.
     
    Rodéric, milk and Grall like this.
  13. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Just did a late night session where I fixed this thing up to handle multiple spheres, finding the nearest hit to the camera, and also anti-aliasing. All that's left in this short book is materials (diffuse, metal, dielectrics), more work on the camera and depth of field (defocus blur, as the book calls it). I've been doing this in probably 30-45 minute sessions. It's pretty quick to go through. Before I move on to the next book I'll probably play around a bit and see what I can come up with on my own. I may play around with the apple apis, see if I can use grand central dispatch to do some of the ray calculations concurrently.

    upload_2018-4-1_2-7-7.png
     
    Rodéric, Lightman and BRiT like this.
  14. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Now I have diffuse materials with 2.2 gamma correction. Working on implementing a metal material type. The book uses an abstract class which is inherited to define materials. I think I'm just going to do one material struct with an enum to define the type, and have a single function to define the material function.
     
    Rodéric and BRiT like this.
  15. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Now I get to learn the xcode debugger. When I went to bed I was implementing my material system and all of my hit pixels were coming out black.

    It really sucks to have to go to work in the morning, when I know the only things I'm going to be thinking about all day are those black pixels.
     
    #15 Scott_Arm, Apr 2, 2018
    Last edited: Apr 2, 2018
    Rodéric and BRiT like this.
  16. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Found some really dumb bugs. The first bug was forgetting to assign the material of the hit result for the ray, which meant the result was just storing a hit with the default material with (0,0,0) albedo. This is why all of my sphere's were drawing as black.

    upload_2018-4-2_19-15-34.png

    Then I had this ...

    upload_2018-4-2_18-51-40.png

    Wtf ... Took a look at my function that iterates through my spheres and tests for a hit. It's supposed to record the value t, which is the nearest point along the ray that hits the sphere. Then check the next sphere with a ray max length of t ... except I was always checking against the max value of Int. This bug never appeared because the middle sphere was first in the list and there was another bug.

    I had a phantom return true in the code that would have returned out of the function, on any hit, before iterating to the next sphere. That's why the large sphere is drawing oddly over the left and right spheres. The large sphere is defined earlier in the list. It also prevented my previous bug from appearing in the output.

    But now it's fixed and I have this ...

    upload_2018-4-2_18-56-27.png

    Middle and bottom sphere are diffuse. Left and right are metal with different roughness factors.
     
    #16 Scott_Arm, Apr 2, 2018
    Last edited: Apr 3, 2018
    Rodéric, BRiT and milk like this.
  17. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Having some issues with dielectric materials. Not going to lie; I need to review the math to fully understand how this material works. That reflection in the dielectric sphere on the left is not correct, and it should look like a bubble (there is a second slightly-smaller sphere inside with a negative radius)

    upload_2018-4-2_22-13-18.png

    Edit: Decided to give it one last look. I was reflecting all rays and never refracting any. There is a probability check for reflection. In the true condition I should be creating a ray that's a reflection, and in the false condition I should be creating a ray that's a refraction. I was creating a ray that was a reflection in both cases. Simple typo, using the vector "reflected" in both cases. Here's the corrected output:

    upload_2018-4-2_22-23-43.png
     
    #17 Scott_Arm, Apr 3, 2018
    Last edited: Apr 3, 2018
    Rodéric, Malo and BRiT like this.
  18. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Got my camera positioning working. It's pretty straightforward. De-focus blur is the last topic in the book. I'll do that tomorrow. For now, I'm going back and figuring out how to properly write to a file in swift.

    Here's a higher-res output of my camera test:

    upload_2018-4-4_21-9-37.png
     
  19. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    I hate to say it, but the most complicated part of this program is figuring out how the fuck to write to files correctly on a mac. Lol. There seem to be so many ways to do it, but oddly no real information out there other than a few posts on stack overflow. I've read through some of the reference on the apple developer site, and part of it is unclear to me. I can make a file handle for writing, but writing requres a Data object, and I'm not sure sure how the Data object works. I'm just trying to write plain text to a file.

    Maybe I'll try NSString instead of a regular String type. Not sure of the differences.

    I'm thinking the best way to do this might be to allocate a large buffer up front, sized to fit the contents of the file. Write the pixel info into the buffer and then write once to file at the end. Right now I'm just using a String object and I append the next line of the file to the string. Not sure how the dynamic allocation is handled when I do that, but I believe that when I did my 1000x500 render, it was about a 6mb file. That's a long ass String.
     
    #19 Scott_Arm, Apr 5, 2018
    Last edited: Apr 5, 2018
    iroboto likes this.
  20. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    12,931
    Likes Received:
    3,094
    Finished de-focus blur and the randomized scene at the end. Here's a low resolution image, because it takes a long time to render.

    upload_2018-4-6_0-52-11.png

    I'm going to play around with profiling and file writing before I move on to the next book. I want to play around with some things in swift to see how it affects performance.
     
    BRiT likes this.

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...