Page 7: Multi-Pass Rendering

We will talk about the concept of multi-pass rendering in lecture. On this page, we’ll focus on a particular kind of multi-pass rendering: where we draw the scene from some other viewpoint than the camera that is ultimately making the picture, store that picture somewhere, and use it (often as a texture) when we make the final picture.

This is useful for many things, for example:

• Reflections (we take a picture of what the mirror sees and put that on the mirror, this is kindof a simple form of Environment Map)
• Shadows (we take a picture of what the light sees, and only light those parts)
• Environments (rather than pre-painting an environment map, we render a picture of the world so the correct things show up)
• Picture in Picture effects - you could put a television showing what a camera elsewhere in the scene is pointing at

From an implementation perspective these things involve:

1. Setting up an additional camera
2. Rendering a picture from that camera’s viewpoint and storing the picture in a texture. This is called render to texture.
3. Using that rendered texture when we ultimately draw the scene.

For shadows (on the previous page), this was built into THREE. The camera was connected to the light source, the image (shadow map) was stored somewhere by three, and its shader used it.

As you might expect, THREE makes it really easy to do this. Cameras in THREE can easily render in a texture that is used by some material. In fact, for the important special case of environment maps, it is the default. You can make a CubeCamera to take a picture of the scene, and use its renderTarget as an environment map.

A dynamic environment map is cool: by taking the picture where an object is, you can create reflections (or lighting) based on what the object actually sees - not what was pre-loaded in the texture map. And, they are really easy because they are practically built-in to THREE.

There are a bunch of steps, all simple:

1. You need to create the CubeCamera at the beginning (for example, when you create the object)
2. You need to attach the CubeCamera’s “render target” as the environment map of the texture
3. At the appropriate time, you need to have the CubeCamera do an update - which is basically, take the picture of the world and put it into the environment map.

There is a little bit of a catch using the class software framework: you need to take a picture with the “environment camera” before you use the picture that it takes. If you have a separate environment map for each object, you can do this in the object’s stepWorld function (which is called before drawing). Or, you can do this using a callback in the animation loop (useful if you use the same map for multiple objects).

There are a couple of other potential problems. One problem is that a camera cannot take a picture of itself (for example, if it is looking at itself in a mirror). If a camera takes a picture of an object that uses a texture that is that camera’s render target, there is a “feedback loop” (the camera is looking at itself in the mirror). This generates an error message. It isn’t disasterous, but usually we take steps to avoid it (for example, not rendering the objects that use the texture map that we are creating) - you can see examples in the framework demo code.

For this exercise, I’d like you to make a dynamic environment map. Create a scene with a moving object in it. Create a reflective object. Use a dynamic environment map so that you can see the reflection of the moving object. There are examples: one uses object centered maps (where there is a camera for each object), and the other just uses a single cube camera at the center of the world. This is the 2020 version of the framework (things will not work without modification). You shouldn’t just copy my code: make sure you understand what is going on, and make something that at least looks different.

You can find the starter code in 09-07-01.js ( 09-07-01.html)

For an advanced point challenge, create a “video screen” in the scene that shows the image that is taken by a camera that is moving around in the scene (you’ll need to have a moving object that we can see). To get the points, it should be obvious what object the camera is “attached to”, and that the video screen is showing the image from that camera.

A Final Note: These kind of multi-pass things involve all kinds of “under-the-hood” details that can be painful to make work. Trying to get the graphics hardware to draw pictures into the right forms so it can be re-used in other ways can be tricky. Fortunately, THREE takes care of that for us. You don’t need to know the details like how it manages memory to make sure the output of one thing can be the input of another. But you do need to understand conceptually how we use this idea of making multiple pictures allows us to achieve effects such as shadows and reflections.

The End…

That’s the end of this workbook.

Check that you have filled in all of the text boxes explaining the textures you used and attributions 09-workbook.txt.

Be sure to add all of your textures and objects, commit and push your work, and complete the Canvas handin Assignment.

A warning: the biggest mistake students make in this workbook is to forget to add their texture files to the repo! Next: Hand-in and Gallery