I finished the book just last month! Super happy with the results. To celebrate I wrote an article and posted about my experience.
For a few thoughts on the book: it was great to learn ray tracing through the test-first approach. I learned a lot about both graphics and C++ programming through the process.
About optimizations and the end-of-chapter challenges: It’s worth it to make some performance optimizations earlier on in the development process, especially once you’re starting to render more complex scenes. It makes the iteration process a lot smoother. The two biggest optimizations I found were adding multithreading (straightforward because ray tracing is embarrassingly parallel) and precomputing matrix inversions. I highly recommend implementing this for your scene’s objects as that computation was by far the heaviest method, and it’s called many times per pixel.
GitHub repo:
Medium Article: