Prev: W9, Next: W11, Quiz of the week: Q10
Links: Zoom, TopHat (300987), Calculator:
Slide:

# Review of 3D Graphics: Meshes

📗 Triangles:
➩ The only primitive shape is triangles (sometimes points and lines).
➩ Projection of a triangle is a triangle.
➩ Barycentric coordinates.
➩ Fast algorithms to draw.

# Review of 3D Graphics: Materials

📗 The color of a point is determined by:
➩ Surface color (and properties).
➩ Light color (and properties).
📗 Local lighting:
➩ Given what light arrives, what is the color of the point.
➩ Compute each triangle (each point on each triangle) independently.
📗 Global lighting:
➩ What light gets to the point.
➩ How do different points interact.
➩ Shadows, reflections, refraction, ... (use texture hacks).



# Process of Drawing in 3D

📗 Triangles in 3D.
📗 Viewing: transform the triangles to 2D.
📗 Rasterization: convert triangles to pixels.
📗 Shading (lighting, texturing, ...): color each pixel.
➩ Clipping: decide if the triangle is on the screen.
➩ Visibility: decide if another triangle blocks the pixel.

# Culling

📗 Culling is skipping primitives based on clipping and visibility.
➩ Skip triangles outside of the camera frustum, outside of near and far planes.
➩ Skip farther triangles that are block by near ones (assume objects are solid and not transparent).
📗 Backface culling (given the normal) is determining whether the triangle is facing the camera: Wikipedia.

# Visibility Algorithms

📗 Painter's Algorithm: Wikipedia.
➩ Not used in practice.
➩ Collect all objects.
➩ Sort from back to front.
➩ Draw objects in order (back to front).
📗 Z-Buffer Algorithm: Wikipedia.
➩ Used in practice.
➩ Add an extra number per pixel: color buffer (RGB) and z-buffer (Z).
➩ Start with all pixels at max distance (far distance of the camera).
➩ When drawing a pixel \(\left(x, y\right)\) with color \(c\) at depth \(z\), if \(z\) value is smaller than the current \(\left(x, y\right)\) z-value (closer to the camera), replace the pixel color with \(c\) and z-value with \(z\).



# Painter's Algorithm

📗 Problems with painter's algorithm:
➩ Need all objects to sort (not immediate mode).
➩ If two triangles have ties or intersect, need to cut them.
➩ Inefficiency: resort when camera moves (can use Binary Space Partitioning trees or BSP trees): Wikipedia.
➩ Inefficiency: draw things that get covered.

# Z-Buffer

📗 Problems with Z-buffer:
➩ Z-fighting problems: when z values can have ties (or numerical issues).
➩ Cannot handle transparency: can sort objects (painter's algorithm) and draw transparent objects last.
➩ Efficiency issue: pixels are thrown away after their colors are computed.
📗 It is used in practice since:
➩ It is simple.
➩ It is generally order-independent (immediate mode).
➩ It is easy to implement in hardware.

# Drawing Order [TopHat]

📗 If the drawing order is "red", "green", "blue", how is the frame buffer updated?
Demo painter



# Drawing Process: Pipeline

📗 Draw each triangle (order independent, can be done in parallel):
➩ (1) Transform triangles into 2D (with Z values).
➩ (2) Rasterize triangles (into pixels called fragments), with Z values).
➩ (3) Figure out the color of those fragments.
➩ (4) Write those fragments to the image (with Z test).
📗 Shaders program steps (1) and (3): Wikipedia.

# Group of Triangles

📗 A shader program draws a group of triangles every frame.
📗 A group of triangles are sent to the program together.
📗 All triangles in the group share some common properties (scene, material), and they are called "uniforms".

# Shader Program

Step Process Shader
1 Transform vertices to 2D Vertex shader
2 Rasterize triangles (fixed)
3 Figure out the color of fragments (pixels) Fragment shader
4 Write fragments to image (fixed)




# Inputs and Outputs

Process Host program (JavaScript) Vertex shader Fragment shader
Inputs - Per object info Per object info
- - Per vertex info Per fragment info
Outputs Per object info Per vertex info Per fragment info
- Per vertex info - -


📗 Per object information is stored in uniform variables.
📗 Per vertex information is stored in attribute variables.
📗 Per fragment information is stored in varying variables.

# Required Inputs and Outputs

📗 The purpose of the vertex shader is to output the screen space position (for rasterization and Z-test), and anything needed for coloring.
📗 The purpose of the fragment shader is to output the color of the fragment (pixel).

Process Host program (JavaScript) Vertex shader Fragment shader
Inputs - uniform uniform
- - attribute varying (per fragment)
Outputs uniform varying (per vertex) gl_FragColor
- attribute gl_Position -


📗 The rasterizer (Barycentric) interpolates the varying variables for the vertices to get the variable for each fragment.



# OpenGL Shading Language (GLSL)

📗 OpenGL GLSL is the shading language based on C: Link.
📗 WebGL (Web Graphics Library) is a variant of OpenGL that can be used with JavaScript: Wikipedia.
➩ Syntax is similar to C: strict typing; operator overloading.
➩ Features for graphics: math data types (vectors and matrices), built-in functions.
📗 Each shader for a material is a separate program (compiled at runtime).
➩ Vertex shader: uniform, attribute -> gl_Position, varying
➩ Fragment shader: uniform, varying -> gl_FragColor

# Where to Put Shaders [Updated]

📗 THREE.ShaderMaterial requires the following parameters:
➩ uniforms: a dictionary of variables (Number, THREE.Vector2, THREE.Vector3, THREE.Color or THREE.Matrix3, THREE.Matrix4, ...): Doc.
➩ vertexShader: a String containing the vertex shader program.
➩ fragmentShader: a String containing the fragment shader program.
➩ Other options: Doc.
📗 The shader programs can be written in JavaScript as a text string (lecture demos read the text in a text box).
➩ Shader programs can also be written in HTML a sa script.
➩ Shader programs can also have their own .vs .fs files (workbooks do this, and CS559 Framework code helps with loading these files): Doc.



# THREE Built-in Uniforms [Updated]

📗 Some uniforms and attributes are built-in (no need to declare them in shader programs): Doc.
Type Variable Math Note
Vertex shader uniform mat4 modelMatrix 4 x 4 matrix -
- uniform mat4 modelViewMatrix 4 x 4 matrix viewMatrix * modelMatrix
- uniform mat4 projectionMatrix 4 x 4 matrix -
- uniform mat4 viewMatrix 4 x 4 matrix -
- uniform mat3 normalMatrix 3 x 3 matrix inverse transpose of modelViewMatrix
- uniform vec3 cameraPosition 3 x 1 vector -
- attribute vec3 position 3 x 1 vector -
- attribute vec3 normal 3 x 1 vector -
- attribute vec3 uv 3 x 1 vector -


📗 There are some uniforms and attributes that are conditional depending on the material and geometry.
📗 For other built-in uniform, for example THREE.UniformLib['lights'], there is a list: Link, but the documentation is not good at the moment: Doc.



# Basic Vertex Shader

📗 The simplest (the one used by THREE.js) vertex shader computes the position of a vertex projected onto the screen: gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0).
➩ projectionMatrix projects a point in 3D camera space onto 2D screen space.
➩ modelViewMatrix projects a point in the 3D scene space onto 3D camera space.
➩ vec4(position, 1.0) is the homogeneous coordinates of the position attribute of the vertex.
📗 Note: 1.0 is a float and 1 is an int, they are different in shader program.
📗 For most of the workbook exercises, vertex shader does not need to be changed.

# Displacement Map with Vertex Shader [TopHat]

📗 CORRECTION from an earlier lecture: displacementMap does not alter the geometry, it just renders it differently with a different vertext shader.
📗 Change the vertex shader so that there are bumps on the surface.
Demo shader

# Some Useful Built-in Functions

📗 If and else: if, else, ? :.
📗 Operators: +, -, *, \.
📗 Comparisons: <, >, <=, >=, ==, !=.
📗 Logical: &&, ||.
📗 Trig: sin, cos, tan, asin, acos, atan.
📗 Other math: abs, floor, ceil, fract, mod, sqrt, pow, exp, log.
📗 Vector: length, distance, cross, normalize, reflect.
📗 To replace conditions (faster): max, min.
➩ sign: sign(x) = x > 0.0 ? 1.0 : (x < 0.0 ? -1.0 : 0.0)
➩ clamp: clamp(x, a, b) = min(max(x, a), b).
➩ mix: mix(x, y, t) = x * (1 - t) + y * t.
➩ step: step(t, x) = x < t ? 0.0 : 1.0.
➩ smoothstep: smoothstep(t1, t2, x) is the smooth version of step, goes from 0.0 to 1.0 smoothly for x between t1 and t2. This is useful to get smooth transition between two colors.
Math
📗 smoothstep performs smooth Hermite interpolation: for x between t1 and t2, t = (x - t0) / (t1 - t0); and smoothstep(t1, t2, x) = t * t * (3.0 - 2.0 * t).



# Fragment Shader Basic Example

📗 The simplest fragment shader uses a constant color for every fragment (pixel): gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0) colors every pixel of the object (group of triangles), the last 1.0 is transparency.
📗 Interesting appearances (shading) can be created using changing the way the pixels are colored: all of these happen in the fragment shader.

# Passing Varying Variable [TopHat]

📗 Find out what the "original" color of the sphere is (what is material.getAttribute("color")?).
Demo shader_variable

# Passing Uniform Variable [TopHat]

📗 Uniforms can be created by JavaScript and passed in the THREE.ShaderMaterial, for example, THREE.ShaderMaterial(uniforms = {t: {value: 0.0}, u: {value: 0.0}}, ...).
📗 Attributes can be created and set for THREE.BufferGeoemtry bg too, for example, bg.setAttribute("v", ...). 
📗 Change the vertex displacement or fragment color as a function of t (time) or u (slider value).
Demo shader_variable

# Passing Uniform Texture Variable [TopHat]

📗 One special type of uniform variable is uniform sampler2D texture.
📗 It can be passed in as THREE.ShaderMaterial(uniforms = {texture: {value: THREE.Texture()}}, ...).
📗 To get its color, use the special function texture2D(texture, uv, ...).
📗 Change the vertex displacement or fragment color using the texture uniform variable tex.
Demo shader_texture



# Implement Lighting

📗 THREE.js has default shaders to implement lighting:
➩ THREE.MeshBasicMaterial does not implement lighting.
➩ THREE.MeshLambertMaterial implements the simplest Lambertian model.
➩ THREE.MeshPhongMaterial implements the Phong model (some physics next lecture).
➩ THREE.MeshStandardMaterial implements an improved Phong model (not sure what it really does).
➩ THREE.MeshPhysicalMaterial implements a more complex model.
➩ THREE.ShaderMaterial allows a custom shader.

# Lecture Summary

📗 Graphics pipeline.
📗 Painter's algorithm.
📗 Z-Buffer algorithm.
📗 Vertex and fragment shaders.
📗 WebGL shaders.
📗 Uniforms, attributes, varyings.
📗 Projection matrices.


📗 Notes and code adapted from the course taught by Professor Michael Gleicher.
📗 Please use Ctrl+F5 or Shift+F5 or Shift+Command+R or Incognito mode or Private Browsing to refresh the cached JavaScript: Code.
📗 You can print the notes: .
📗 Anonymous feedback can be submitted to: Form.

Prev: W9, Next: W11, Quiz of the week: Q10





Last Updated: November 30, 2024 at 4:35 AM