Sunday, March 27, 2016
Learn Quartz Composer in One Weekend
Quartz composer (QC) is an interactive dataflow language from Apple (you need MacOS to run it). I prototype many things in QC because it is very easy to do many things like test image processing functions by feeding the web cam into it. Facebook prototypes their apps on a version of QC they have extended for their purposes. Lots of people, including me, love it. However, visual dataflow a pretty different way of programming and there are also conventions in QC that that are easy, but somebody needs to show them to you. I spent about a week learning QC to the point where I could do something useful because I was a bit overwhelmed by all of the documentation out there. However, key was I found an almost perfect set of youtube videos by Rob Duarte. Go through these videos on Day 1, and be sure to follow each one and actually duplicate his compositions as you go. On Day 2, do your own project. I, for example, did face detection on the webcam and then added devil horns. That was a really easy project in QC because there is a face detection module build in.
Thursday, March 24, 2016
Ray Tracing: The Rest of Your Life
Ray Tracing: The Rest Of Your Life (Ray Tracing Minibooks Book 3)
Chapter 0 Overview
This page is for further reading and a page to comment on. This book is for people who have already written a ray tracer as an entry portal into the world of graphics research. So this book has a narrower audience than the previous two mini-books. It covers a path tracer's probabilistic sampling in enough detail to get people up to speed to follow the literature and rendering trends.
Chapter 1: A Simple Monte Carlo Program
Don Mitchell has a nice paper on how jittering changes convergence rates.
Chapter 2: Monte Carlo Integration
Chapter 3: Light Scattering
Chapter 4: Importance Sampling Methods
Chapter 5: Generating Random Directions
Chapter 6: Ortho-normal Bases
A very nice little trick on quickly generating ONBs
Chapter 7: Sampling Lights Directly
Chapter 8: Mixture Densities
Chapter 9: Some Architectural Decisions
Chapter 10: Cleaning Up Pdf Management
Chapter 11: The Rest of Your Life
Rendering is a mature enough field that there are several good book on it now. First I would get yourself a copy of this book:
Physically Based Rendering, Third Edition: From Theory To Implementation
I used it as a text in my course instead of my own book. It is that good!
This is another extremely good book on physically based rendering:
Advanced Global Illumination, Second Edition
This book is older, but it mostly covers topics that don't age and I would definitely get a copy. It's treatment of the math topics useful in graphics are my favorite (for example the basis function discussion is amazing): (update-- it is free online!)
Principles of Digital Image Synthesis (The Morgan Kaufmann Series in Computer Graphics) 2 Volume Set
You will want to start putting out high dynamic range images. You are already computing them! This book is a very good discussion of high dynamic range issues:
High Dynamic Range Imaging, Second Edition: Acquisition, Display, and Image-Based Lighting
This is another older book. It's still got some gold in it, and if you're gonna stay in the area long-haul then I would buy it:
An Introduction to Ray Tracing (The Morgan Kaufmann Series in Computer Graphics)
Henrik Jensen has a very nice book on photon mapping that is also just a good book on physically-based rendering. Realistic Image Synthesis Using Photon Mapping 2nd Revised edition by Jensen, Henrik Wann (2001) Paperback
Sunday, January 31, 2016
Ray Tracing: the Next Week
There's been a lot of interest and positive feedback on my mini-book on ray tracing.
This page is for the sequel Ray Tracing: the Next Week
, available on Kindle. This page also gives links and pointers for each chapter. The features covered are those in this picture:
Chapter 1: Motion Blur
This method was introduced in 1984 by Rob Cook.
Chapter 2: A Bounding Volume Hierarchy (BVH)
The construction method in the book can be improved by using the surface area heuristic (SAH). When evaluating potential partitions, the one that minimized the surface area of the sum of volumes of the sub-trees is almost always good. Here is a SAH-based build that cuts on the longest axis.
Chapter 3: Solid Texture Mapping
Chapter 4: Perlin Noise
A fantastic tool explaining how it works by Andrew Kensler
Chapter 5: Image Texture Mapping
Chapter 6: Rectangles and Lights
The program in the book implicitly samples lights so there are no shadow rays. If you want to get more efficient direct lighting you can either send shadow rays, or importance sample by sending more rays toward the lights.
Chapter 7: Instances
A general instance usually stores transformation matrices. Composite transforms can all be in one node. When scales are allowed handling the surface normals must be done with care.
Chapter 8: Volumes
Here's a derivation of the ray-constant-medium intersection.
It is straightforward to add nonuniform densities by adding a more sophisticated intersection method. This is covered in this blog post. It's pretty common knowledge in the ray tracing community, but not really in the intersection.
Wednesday, January 27, 2016
Ray Tracing in One Weekend
The books are now available in unprotected pdf for PAY WHAT YOU WISH, with 50% going to not for profit programming education organizations. Here are the files.
Ray tracing was invented by Turner Whitted around 1980. His classic paper is really amazing; it had bounding volume hierarchies, adaptive sampling, and it's future work suggested randomized reflection. Since then it has branched into a ton of variations and is used in most movies for the indirect "bounce" lighting.
I've was assigned a 2D ray tracer in a physics class in 1984 and I was hooked. Since then I have written a bunch of ray tracers in Pascal, Fortran, Scheme, C, C++, Java, and most recently Swift (which is awesome for writing ray tracers). I've also taught classes with ray tracing about a dozen times, most recently at Westminster College in 2015. My approach over the years has evolved to do what I think are the most fun parts of ray tracing that are still in the direction of writing a production-quality ray tracer.
Ray Tracing in One Weekend
is a kindle book that goes through all of the details to generate a rudimentary ray tracer. It's $2.99 on amazon. It uses C plus classes plus operator overloading. I have heard this referred to as "C plus" which I now call it. Most production renderers are written in C++ so I opted for that as the driving language. I put the most primal set of these assignments so that it's very doable in a weekend, the fundamental unit of coding self-improvement :) Here is the book:
The resulting program generates this image:
Here's my MacOS code for the book. Should be semi-portable but drand48() will need to be written on some systems and the fileio may be too. I also have a little post on the tools I used to write the book for those interested in doing a little book of their own. Please let me know if you do!
Links for further reading and exploration related to the book:
Chapter 0: Overview
Here is a list of basic graphics texts that cover the vector background the book assumes:
Fundamentals of Computer Graphics, Fourth Edition
Computer Graphics: Principles and Practice (3rd Edition)
The Graphics Codex
Real-Time Rendering, Third Edition
Chapter 1: Output an image
For output of most LDR and HDR images I love stb_image.
For more info on HDR images and tone mapping the I like the book High Dynamic Range Imaging, Second Edition: Acquisition, Display, and Image-Based Lighting
Chapter 2: The vec3 class
I advocate using the same class for points, displacements, colors, etc. Some people like more structure and type checking (so for example multiplying two locations would be rejected by the compiler). An example article where points and vectors are different is here. Jim Arvo and Brian Smits experimented with not only distinguishing points and vectors, but using the compiler to do dimensional analysis (so velocity, length, and time would be different types for example). They found this to be too cumbersome in 1990s C++ but I'd love to hear about anybody's experience. Researchers at Dartmouth have taken a really serious effort at this and their code and paper are available at github.
The first thing you might add to your background function is an environment map. Paul Debevec has a terrific history of environment mapping. The easiest mapping for this uses a single image for the entire sphere of directions. Paul also provides some good images to use for your environment map.
Chapter 4: Adding a sphere
There are a bunch of other object types you can add. Triangles are usually first and I am a fan of barycentric methods. After triangles, many people quit adding primitives because graphics has such a big infrastructure for triangles. Ellipses are an easy thing to add but instancing is usually a more "ray tracey" approach (let the software do the heavy lifting). Composite objects via CSG are surprisingly straightforward.
Box filtering as done in the book suffices for most situations. However, Guassian-like filters can have advantages and are pretty easy. You can either uniformly sample the whole screen and weight the samples, or use non-uniform samples. All approaches work.
"Ideal" diffuse materials, also called "Lambertian" are used 99% of the time in graphics. The wikipedia article on this approximation is good. Real diffuse surfaces do not behave exactly as Lambertian (for example they get specular at grazing angle) but especially with interreflection in the mix the appearance differences are minor. So this is probably not where you should push your renderer until many many other features are addressed.
The first improvement you might make is to have the color of the metal go to white at grazing angle. The Schlick approximation (used in Chapter 9 for glass where grazing behavior matters more) works for that. Full-bore Fresnel equations will describe color variation with angle, but in my experience getting normal incident color is good enough.
The first thing you might add is filtering of light within the dielectric (like the subtle "greenness" of much glass). This is a classic exponential decay and is covered well in the Beer's Law section of this nice page.
If you ever need to match a real camera, there is a nice survey of models used in graphics. And here is a very good presentation on real lenses written for a technical audience.
Ray tracing was invented by Turner Whitted around 1980. His classic paper is really amazing; it had bounding volume hierarchies, adaptive sampling, and it's future work suggested randomized reflection. Since then it has branched into a ton of variations and is used in most movies for the indirect "bounce" lighting.
I've was assigned a 2D ray tracer in a physics class in 1984 and I was hooked. Since then I have written a bunch of ray tracers in Pascal, Fortran, Scheme, C, C++, Java, and most recently Swift (which is awesome for writing ray tracers). I've also taught classes with ray tracing about a dozen times, most recently at Westminster College in 2015. My approach over the years has evolved to do what I think are the most fun parts of ray tracing that are still in the direction of writing a production-quality ray tracer.
Ray Tracing in One Weekend
is a kindle book that goes through all of the details to generate a rudimentary ray tracer. It's $2.99 on amazon. It uses C plus classes plus operator overloading. I have heard this referred to as "C plus" which I now call it. Most production renderers are written in C++ so I opted for that as the driving language. I put the most primal set of these assignments so that it's very doable in a weekend, the fundamental unit of coding self-improvement :) Here is the book:
The resulting program generates this image:
Here's my MacOS code for the book. Should be semi-portable but drand48() will need to be written on some systems and the fileio may be too. I also have a little post on the tools I used to write the book for those interested in doing a little book of their own. Please let me know if you do!
Links for further reading and exploration related to the book:
Chapter 0: Overview
Here is a list of basic graphics texts that cover the vector background the book assumes:
Fundamentals of Computer Graphics, Fourth Edition
Computer Graphics: Principles and Practice (3rd Edition)The Graphics Codex
Real-Time Rendering, Third Edition
Chapter 1: Output an image
For output of most LDR and HDR images I love stb_image.
For more info on HDR images and tone mapping the I like the book High Dynamic Range Imaging, Second Edition: Acquisition, Display, and Image-Based Lighting
Chapter 2: The vec3 class
I advocate using the same class for points, displacements, colors, etc. Some people like more structure and type checking (so for example multiplying two locations would be rejected by the compiler). An example article where points and vectors are different is here. Jim Arvo and Brian Smits experimented with not only distinguishing points and vectors, but using the compiler to do dimensional analysis (so velocity, length, and time would be different types for example). They found this to be too cumbersome in 1990s C++ but I'd love to hear about anybody's experience. Researchers at Dartmouth have taken a really serious effort at this and their code and paper are available at github.
Chapter 3: Rays, a simple camera, and background
The first thing you might add to your background function is an environment map. Paul Debevec has a terrific history of environment mapping. The easiest mapping for this uses a single image for the entire sphere of directions. Paul also provides some good images to use for your environment map.
Chapter 4: Adding a sphere
There are a bunch of other object types you can add. Triangles are usually first and I am a fan of barycentric methods. After triangles, many people quit adding primitives because graphics has such a big infrastructure for triangles. Ellipses are an easy thing to add but instancing is usually a more "ray tracey" approach (let the software do the heavy lifting). Composite objects via CSG are surprisingly straightforward.
Chapter 5: Surface normals and multiple objects.
If you want your code to be more efficient for large numbers of objects, use a BVH-- they are as good as any other in efficiency and are the most robust and easiest to implement.
If you want your code to be more efficient for large numbers of objects, use a BVH-- they are as good as any other in efficiency and are the most robust and easiest to implement.
Chapter 6: Antialiasing
Box filtering as done in the book suffices for most situations. However, Guassian-like filters can have advantages and are pretty easy. You can either uniformly sample the whole screen and weight the samples, or use non-uniform samples. All approaches work.
Chapter 7: Diffuse Materials
"Ideal" diffuse materials, also called "Lambertian" are used 99% of the time in graphics. The wikipedia article on this approximation is good. Real diffuse surfaces do not behave exactly as Lambertian (for example they get specular at grazing angle) but especially with interreflection in the mix the appearance differences are minor. So this is probably not where you should push your renderer until many many other features are addressed.
Chapter 8: Metal
The first improvement you might make is to have the color of the metal go to white at grazing angle. The Schlick approximation (used in Chapter 9 for glass where grazing behavior matters more) works for that. Full-bore Fresnel equations will describe color variation with angle, but in my experience getting normal incident color is good enough.
Chapter 9: Dielectrics
The first thing you might add is filtering of light within the dielectric (like the subtle "greenness" of much glass). This is a classic exponential decay and is covered well in the Beer's Law section of this nice page.
Chapter 10: Positionable camera
Camera parameter setting is just plain ugly. The system used in the book is relatively common and is in my opinion the prettiest. I avoid the matrix formulation wherever possible because I never understand my code when I am done. But it is very elegant and works.
Chapter 11: Defocus blur
If you ever need to match a real camera, there is a nice survey of models used in graphics. And here is a very good presentation on real lenses written for a technical audience.
Chapter 12: Where next?
You should have the core of a very serious ray tracer now. I would now take it in one of three directions. They are not mutually exclusive but explicitly deciding your goals will simplify architectural decisions.
- Make it physically accurate. This will imply using spectra instead of RGB (I like just using a big array of wavelengths) and get something where you know the reflectances. Popular is to get a X-Rite MSCCC ColorChecker Classic
whose data is available online.
- Make it good for generating animations. Lots of movies use a ray traced system, and Disney, Pixar, and the Solid Angle teams have both disclosed a remarkable amount about their code. Work on features and then efficiency. I think you will be amazed how soon you can produce amazing images.
- Make it fast. Here you can roll your own, or start using a commercial API. To see exactly where that community is now, go to the 2016 HPG conference. Or backtrack in their previous papers. They are a very open community and the papers go into much detail relative to many sub-fields in computer graphics.