Prev: W9, Next: W11
Links: Zoom, TopHat (993679, or Form), Calculator:
Slide:

# Review of 3D Graphics: Meshes

📗 Triangles:
➩ The only primitive shape is triangles (sometimes points and lines).
➩ Projection of a triangle is a triangle.
➩ Barycentric coordinates.
➩ Fast algorithms to draw.

# Review of 3D Graphics: Materials

📗 The color of a point is determined by:
➩ Surface color (and properties).
➩ Light color (and properties).
📗 Local lighting:
➩ Given what light arrives, what is the color of the point.
➩ Compute each triangle (each point on each triangle) independently.
📗 Global lighting:
➩ What light gets to the point.
➩ How do different points interact.
➩ Shadows, reflections, refraction, ... (use texture hacks).



# Process of Drawing in 3D

📗 Triangles in 3D.
📗 Viewing: transform the triangles to 2D.
📗 Rasterization: convert triangles to pixels.
📗 Shading (lighting, texturing, ...): color each pixel.
➩ Clipping: decide if the triangle is on the screen.
➩ Visibility: decide if another triangle blocks the pixel.

# Culling

📗 Culling is skipping primitives based on clipping and visibility.
➩ Skip triangles outside of the camera frustum, outside of near and far planes.
➩ Skip farther triangles that are block by near ones (assume objects are solid and not transparent).
📗 Backface culling (given the normal) is determining whether the triangle is facing the camera: Wikipedia.

# Visibility Algorithms

📗 Painter's Algorithm: Wikipedia.
➩ Not used in practice.
➩ Collect all objects.
➩ Sort from back to front.
➩ Draw objects in order (back to front).
📗 Z-Buffer Algorithm: Wikipedia.
➩ Used in practice.
➩ Add an extra number per pixel: color buffer (RGB) and z-buffer (Z).
➩ Start with all pixels at max distance (far distance of the camera).
➩ When drawing a pixel \(\left(x, y\right)\) with color \(c\) at depth \(z\), if \(z\) value is smaller than the current \(\left(x, y\right)\) z-value (closer to the camera), replace the pixel color with \(c\) and z-value with \(z\).



# Painter's Algorithm

📗 Problems with painter's algorithm:
➩ Need all objects to sort (not immediate mode).
➩ If two triangles have ties or intersect, need to cut them.
➩ Inefficiency: resort when camera moves (can use Binary Space Partitioning trees or BSP trees): Wikipedia.
➩ Inefficiency: draw things that get covered.

# Z-Buffer

📗 Problems with Z-buffer:
➩ Z-fighting problems: when z values can have ties (or numerical issues).
➩ Cannot handle transparency: can sort objects (painter's algorithm) and draw transparent objects last.
➩ Efficiency issue: pixels are thrown away after their colors are computed.
📗 It is used in practice since:
➩ It is simple.
➩ It is generally order-independent (immediate mode).
➩ It is easy to implement in hardware.

# Drawing Order [TopHat]

📗 If the drawing order is "red", "green", "blue", how is the frame buffer updated?
Demo painter



# Drawing Process: Pipeline

📗 Draw each triangle (order independent, can be done in parallel):
➩ (1) Transform triangles into 2D (with Z values).
➩ (2) Rasterize triangles (into pixels called fragments), with Z values).
➩ (3) Figure out the color of those fragments.
➩ (4) Write those fragments to the image (with Z test).
📗 Shaders program steps (1) and (3): Wikipedia.

# Group of Triangles

📗 A shader program draws a group of triangles every frame.
📗 A group of triangles are sent to the program together.
📗 All triangles in the group share some common properties (scene, material), and they are called "uniforms".

# Shader Program

Step Process Shader
1 Transform vertices to 2D Vertex shader
2 Rasterize triangles (fixed)
3 Figure out the color of fragments (pixels) Fragment shader
4 Write fragments to image (fixed)




# Inputs and Outputs

Process Host program (JavaScript) Vertex shader Fragment shader
Inputs - Per object info Per object info
- - Per vertex info Per fragment info
Outputs Per object info Per vertex info Per fragment info
- Per vertex info - -


📗 Per object information is stored in uniform variables.
📗 Per vertex information is stored in attribute variables.
📗 Per fragment information is stored in varying variables.

# Required Inputs and Outputs

📗 The purpose of the vertex shader is to output the screen space position (for rasterization and Z-test), and anything needed for coloring.
📗 The purpose of the fragment shader is to output the color of the fragment (pixel).

Process Host program (JavaScript) Vertex shader Fragment shader
Inputs - uniform uniform
- - attribute varying (per fragment)
Outputs uniform varying (per vertex) gl_FragColor
- attribute gl_Position -


📗 The rasterizer (Barycentric) interpolates the varying variables for the vertices to get the variable for each fragment.



# OpenGL Shading Language (GLSL)

📗 OpenGL GLSL is the shading language based on C: Link.
📗 WebGL (Web Graphics Library) is a variant of OpenGL that can be used with JavaScript: Wikipedia.
➩ Syntax is similar to C: strict typing; operator overloading.
➩ Features for graphics: math data types (vectors and matrices), built-in functions.
📗 Each shader for a material is a separate program (compiled at runtime).
➩ Vertex shader: uniform, attribute -> gl_Position, varying
➩ Fragment shader: uniform, varying -> gl_FragColor

# Where to Put Shaders

📗 THREE.ShaderMaterial requires the following parameters:
➩ uniforms: a dictionary of variables (Number, THREE.Vector2, THREE.Vector3, THREE.Color or THREE.Matrix3, THREE.Matrix4, ...): Doc.
➩ vertexShader: a String containing the vertex shader program.
➩ fragmentShader: a String containing the fragment shader program.
➩ Other options: Doc.
📗 The shader programs can be written in JavaScript as a text string (lecture demos read the text in a text box).
➩ Shader programs can also be written in HTML a sa script.
➩ Shader programs can also have their own .vs .fs files (workbooks do this, and CS559 Framework code helps with loading these files): Doc.



# THREE Built-in Uniforms

📗 Some uniforms and attributes are built-in (no need to declare them in shader programs): Doc.
Type Variable Math Note
Vertex shader uniform mat4 modelMatrix 4 x 4 matrix -
- uniform mat4 modelViewMatrix 4 x 4 matrix viewMatrix * modelMatrix
- uniform mat4 projectionMatrix 4 x 4 matrix -
- uniform mat4 viewMatrix 4 x 4 matrix -
- uniform mat3 normalMatrix 3 x 3 matrix inverse transpose of modelViewMatrix
- uniform vec3 cameraPosition 3 x 1 vector -
- attribute vec3 position 3 x 1 vector -
- attribute vec3 normal 3 x 1 vector -
- attribute vec3 uv 3 x 1 vector -


📗 There are some uniforms and attributes that are conditional depending on the material and geometry.
📗 For other built-in uniform, for example THREE.UniformLib['lights'], there is a list: Link, but the documentation is not good at the moment: Doc.


# Affine in 3D as Linear in 4D

📗 Affine transformations in 2D were represented by linear transformations in 3D (3 by 3 matrices in HTML Canvas \(\begin{bmatrix} a & c & e \\ b & d & f \\ 0 & 0 & 1 \end{bmatrix}\)), similarly affine transformations in 3D can be represented by linear transformations in 4D (modelMatrix is 4 by 4).
➩ Translation in 3D can be viewed as shearing in 4D on the \(z = 1\) hyperplane.
➩ Rotation and scaling in 3D can still be rotation and scaling in 4D.
➩ A 3D point \(\begin{bmatrix} x \\ y \\ z \end{bmatrix}\) can be represented by 4D point with \(w = 1\), \(\begin{bmatrix} x \\ y \\ z \\ 1 \end{bmatrix}\) (homogeneous coordinate system)
➩ Arbitrary 4D linear transformations can be applied to \(\begin{bmatrix} x \\ y \\ z \\ 1 \end{bmatrix}\), and the result can be projected back onto \(w = 1\) hyperplane (\(\begin{bmatrix} x' \\ y' \\ z' \\ w' \end{bmatrix}\) can be projected back as \(\begin{bmatrix} \dfrac{x'}{w'} \\ \dfrac{y'}{w'} \\ \dfrac{z'}{w'} \\ 1 \end{bmatrix}\)).
Demo homogeneous



# Basic Vertex Shader

📗 The simplest (the one used by THREE.js) vertex shader computes the position of a vertex projected onto the screen: gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0).
➩ projectionMatrix projects a point in 3D camera space onto 2D screen space.
➩ modelViewMatrix projects a point in the 3D scene space onto 3D camera space.
➩ vec4(position, 1.0) is the homogeneous coordinates of the position attribute of the vertex.
📗 Note: 1.0 is a float and 1 is an int, they are different in shader program.
📗 For most of the workbook exercises, vertex shader does not need to be changed.

# Displacement Map with Vertex Shader [TopHat]

📗 displacementMap does not alter the geometry, it just renders it differently with a different vertext shader.
📗 Change the vertex shader so that there are bumps on the surface.
Demo shader
Demo shader_texture

# Some Useful Built-in Functions

📗 If and else: if, else, ? :.
📗 Operators: +, -, *, \.
📗 Comparisons: <, >, <=, >=, ==, !=.
📗 Logical: &&, ||.
📗 Trig: sin, cos, tan, asin, acos, atan.
📗 Other math: abs, floor, ceil, fract, mod, sqrt, pow, exp, log.
📗 Vector: length, distance, cross, normalize, reflect.
📗 To replace conditions (faster): max, min.
➩ sign: sign(x) = x > 0.0 ? 1.0 : (x < 0.0 ? -1.0 : 0.0)
➩ clamp: clamp(x, a, b) = min(max(x, a), b).
➩ mix: mix(x, y, t) = x * (1 - t) + y * t.
➩ step: step(t, x) = x < t ? 0.0 : 1.0.
➩ smoothstep: smoothstep(t1, t2, x) is the smooth version of step, goes from 0.0 to 1.0 smoothly for x between t1 and t2. This is useful to get smooth transition between two colors.
Math
📗 smoothstep performs smooth Hermite interpolation: for x between t1 and t2, t = (x - t0) / (t1 - t0); and smoothstep(t1, t2, x) = t * t * (3.0 - 2.0 * t).



# Fragment Shader Basic Example

📗 The simplest fragment shader uses a constant color for every fragment (pixel): gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0) colors every pixel of the object (group of triangles), the last 1.0 is transparency.
📗 Interesting appearances (shading) can be created using changing the way the pixels are colored: all of these happen in the fragment shader.

# Passing Varying Variable [TopHat]

📗 Find out what the "original" color of the sphere is (what is material.getAttribute("color")?).
Demo shader_variable

# Passing Uniform Variable [TopHat]

📗 Uniforms can be created by JavaScript and passed in the THREE.ShaderMaterial, for example, THREE.ShaderMaterial(uniforms = {t: {value: 0.0}, u: {value: 0.0}}, ...).
📗 Attributes can be created and set for THREE.BufferGeoemtry bg too, for example, bg.setAttribute("v", ...). 
📗 Change the vertex displacement or fragment color as a function of t (time) or u (slider value).
Demo shader_variable

# Passing Uniform Texture Variable [TopHat]

📗 One special type of uniform variable is uniform sampler2D texture.
📗 It can be passed in as THREE.ShaderMaterial(uniforms = {texture: {value: THREE.Texture()}}, ...).
📗 To get its color, use the special function texture2D(texture, uv, ...).
📗 Change the vertex displacement or fragment color using the texture uniform variable tex.
Demo shader_texture



# Implement Lighting

📗 THREE.js has default shaders to implement lighting:
➩ THREE.MeshBasicMaterial does not implement lighting.
➩ THREE.MeshLambertMaterial implements the simplest Lambertian model.
➩ THREE.MeshPhongMaterial implements the Phong model (some physics next lecture).
➩ THREE.MeshStandardMaterial implements an improved Phong model (not sure what it really does).
➩ THREE.MeshPhysicalMaterial implements a more complex model.
➩ THREE.ShaderMaterial allows a custom shader.

# Lecture Summary

📗 Graphics pipeline.
📗 Painter's algorithm.
📗 Z-Buffer algorithm.
📗 Vertex and fragment shaders.
📗 WebGL shaders.
📗 Uniforms, attributes, varyings.
📗 Projection matrices.


📗 Notes and code adapted from the course taught by Professor Michael Gleicher.
📗 Please use Ctrl+F5 or Shift+F5 or Shift+Command+R or Incognito mode or Private Browsing to refresh the cached JavaScript: Code.
📗 You can expand all the examples and demos: , or print the notes: .
📗 If there is an issue with TopHat during the lectures, please submit your answers on paper (include your Wisc ID and answers) or this Google Form at the end of the lecture.

Prev: W9, Next: W11





Last Updated: April 05, 2025 at 10:41 PM