# Review of 3D Graphics: Meshes

📗 Triangles:
➭ The only primitive shape is triangles (sometimes points and lines).
➭ Projection of a triangle is a triangle.
➭ Barycentric coordinates.
➭ Fast algorithms to draw.

# Review of 3D Graphics: Materials

📗 The color of a point is determined by:
➭ Surface color (and properties).
➭ Light color (and properties).
📗 Local lighting:
➭ Given what light arrives, what is the color of the point.
➭ Compute each triangle (each point on each triangle) independently.
📗 Global lighting:
➭ What light gets to the point.
➭ How do different points interact.
➭ Shadows, reflections, refraction, ... (use texture hacks).

# Process of Drawing in 3D

📗 Triangles in 3D.
📗 Viewing: transform the triangles to 2D.
📗 Rasterization: convert triangles to pixels.
📗 Shading (lighting, texturing, ...): color each pixel.
➭ Clipping: decide if the triangle is on the screen.
➭ Visibility: decide if another triangle blocks the pixel.

# Culling

📗 Culling is skipping primitives based on clipping and visibility.
➭ Skip triangles outside of the camera frustum, outside of near and far planes.
➭ Skip farther triangles that are block by near ones (assume objects are solid and not transparent).
📗 Backface culling (given the normal) is determining whether the triangle is facing the camera: Wikipedia.

# Visibility Algorithms

📗 Painter's Algorithm: Wikipedia.
➭ Not used in practice.
➭ Collect all objects.
➭ Sort from back to front.
➭ Draw objects in order (back to front).
📗 Z-Buffer Algorithm: Wikipedia.
➭ Used in practice.
➭ Add an extra number per pixel: color buffer (RGB) and z-buffer (Z).
➭ Start with all pixels at max distance (far distance of the camera).
➭ When drawing a pixel \(\left(x, y\right)\) with color \(c\) at depth \(z\), if \(z\) value is smaller than the current \(\left(x, y\right)\) z-value (closer to the camera), replace the pixel color with \(c\) and z-value with \(z\).

# Painter's Algorithm

📗 Problems with painter's algorithm:
➭ Need all objects to sort (not immediate mode).
➭ If two triangles have ties or intersect, need to cut them.
➭ Inefficiency: resort when camera moves (can use Binary Space Partitioning trees or BSP trees): Wikipedia.
➭ Inefficiency: draw things that get covered.

# Z-Buffer

📗 Problems with Z-buffer:
➭ Z-fighting problems: when z values can have ties (or numerical issues).
➭ Cannot handle transparency: can sort objects (painter's algorithm) and draw transparent objects last.
➭ Efficiency issue: pixels are thrown away after their colors are computed.
📗 It is used in practice since:
➭ It is simple.
➭ It is generally order-independent (immediate mode).
➭ It is easy to implement in hardware.

# Drawing Order [TopHat]

📗 If the drawing order is "red", "green", "blue", how is the frame buffer updated?

Demo painter

# Drawing Process: Pipeline

📗 Draw each triangle (order independent, can be done in parallel):
➭ (1) Transform triangles into 2D (with Z values).
➭ (2) Rasterize triangles (into pixels called fragments), with Z values).
➭ (3) Figure out the color of those fragments.
➭ (4) Write those fragments to the image (with Z test).
📗 Shaders program steps (1) and (3): Wikipedia.

# Group of Triangles

📗 A shader program draws a group of triangles every frame.
📗 A group of triangles are sent to the program together.
📗 All triangles in the group share some common properties (scene, material), and they are called "uniforms".

# Shader Program

# Inputs and Outputs

📗 Per object information is stored in uniform variables.
📗 Per vertex information is stored in attribute variables.
📗 Per fragment information is stored in varying variables.

# Required Inputs and Outputs

📗 The purpose of the vertex shader is to output the screen space position (for rasterization and Z-test), and anything needed for coloring.
📗 The purpose of the fragment shader is to output the color of the fragment (pixel).



📗 The rasterizer (Barycentric) interpolates the varying variables for the vertices to get the variable for each fragment.

# OpenGL Shading Language (GLSL)

📗 OpenGL GLSL is the shading language based on C: Link.
📗 WebGL (Web Graphics Library) is a variant of OpenGL that can be used with JavaScript: Wikipedia.
➭ Syntax is similar to C: strict typing; operator overloading.
➭ Features for graphics: math data types (vectors and matrices), built-in functions.
📗 Each shader for a material is a separate program (compiled at runtime).
➭ Vertex shader: uniform, attribute -> gl_Position, varying
➭ Fragment shader: uniform, varying -> gl_FragColor

# Where to Put Shaders [Updated]

📗 THREE.ShaderMaterial requires the following parameters:
➭ uniforms: a dictionary of variables (Number, THREE.Vector2, THREE.Vector3, THREE.Color or THREE.Matrix3, THREE.Matrix4, ...): Doc.
➭ vertexShader: a String containing the vertex shader program.
➭ fragmentShader: a String containing the fragment shader program.
➭ Other options: Doc.
📗 The shader programs can be written in JavaScript as a text string (lecture demos read the text in a text box).
➭ Shader programs can also be written in HTML a sa script.
➭ Shader programs can also have their own .vs .fs files (workbooks do this, and CS559 Framework code helps with loading these files): Doc.

# THREE Built-in Uniforms [Updated]

📗 Some uniforms and attributes are built-in (no need to declare them in shader programs): Doc.


📗 There are some uniforms and attributes that are conditional depending on the material and geometry.
📗 For other built-in uniform, for example THREE.UniformLib['lights'], there is a list: Link, but the documentation is not good at the moment: Doc.

# Basic Vertex Shader

📗 The simplest (the one used by THREE.js) vertex shader computes the position of a vertex projected onto the screen: gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0).
➭ projectionMatrix projects a point in 3D camera space onto 2D screen space.
➭ modelViewMatrix projects a point in the 3D scene space onto 3D camera space.
➭ vec4(position, 1.0) is the homogeneous coordinates of the position attribute of the vertex.
📗 Note: 1.0 is a float and 1 is an int, they are different in shader program.
📗 For most of the workbook exercises, vertex shader does not need to be changed.

# Displacement Map with Vertex Shader [TopHat]

📗 CORRECTION from an earlier lecture: displacementMap does not alter the geometry, it just renders it differently with a different vertext shader.
📗 Change the vertex shader so that there are bumps on the surface.

Demo shader

# Some Useful Built-in Functions

📗 If and else: if, else, ? :.
📗 Operators: +, -, *, \.
📗 Comparisons: <, >, <=, >=, ==, !=.
📗 Logical: &&, ||.
📗 Trig: sin, cos, tan, asin, acos, atan.
📗 Other math: abs, floor, ceil, fract, mod, sqrt, pow, exp, log.
📗 Vector: length, distance, cross, normalize, reflect.
📗 To replace conditions (faster): max, min.
➭ sign: sign(x) = x > 0.0 ? 1.0 : (x < 0.0 ? -1.0 : 0.0)
➭ clamp: clamp(x, a, b) = min(max(x, a), b).
➭ mix: mix(x, y, t) = x * (1 - t) + y * t.
➭ step: step(t, x) = x < t ? 0.0 : 1.0.
➭ smoothstep: smoothstep(t1, t2, x) is the smooth version of step, goes from 0.0 to 1.0 smoothly for x between t1 and t2. This is useful to get smooth transition between two colors.

Math

📗 smoothstep performs smooth Hermite interpolation: for x between t1 and t2, t = (x - t0) / (t1 - t0); and smoothstep(t1, t2, x) = t * t * (3.0 - 2.0 * t).

# Fragment Shader Basic Example

📗 The simplest fragment shader uses a constant color for every fragment (pixel): gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0) colors every pixel of the object (group of triangles), the last 1.0 is transparency.
📗 Interesting appearances (shading) can be created using changing the way the pixels are colored: all of these happen in the fragment shader.

# Passing Varying Variable [TopHat]

📗 Find out what the "original" color of the sphere is (what is material.getAttribute("color")?).

Demo shader_variable

# Passing Uniform Variable [TopHat]

📗 Uniforms can be created by JavaScript and passed in the THREE.ShaderMaterial, for example, THREE.ShaderMaterial(uniforms = {t: {value: 0.0}, u: {value: 0.0}}, ...).
📗 Attributes can be created and set for THREE.BufferGeoemtry bg too, for example, bg.setAttribute("v", ...). 
📗 Change the vertex displacement or fragment color as a function of t (time) or u (slider value).

Demo shader_variable

# Passing Uniform Texture Variable [TopHat]

📗 One special type of uniform variable is uniform sampler2D texture.
📗 It can be passed in as THREE.ShaderMaterial(uniforms = {texture: {value: THREE.Texture()}}, ...).
📗 To get its color, use the special function texture2D(texture, uv, ...).
📗 Change the vertex displacement or fragment color using the texture uniform variable tex.

Demo shader_texture

# Implement Lighting

📗 THREE.js has default shaders to implement lighting:
➭ THREE.MeshBasicMaterial does not implement lighting.
➭ THREE.MeshLambertMaterial implements the simplest Lambertian model.
➭ THREE.MeshPhongMaterial implements the Phong model (some physics next lecture).
➭ THREE.MeshStandardMaterial implements an improved Phong model (not sure what it really does).
➭ THREE.MeshPhysicalMaterial implements a more complex model.
➭ THREE.ShaderMaterial allows a custom shader.

# Lecture Summary

📗 Graphics pipeline.
📗 Painter's algorithm.
📗 Z-Buffer algorithm.
📗 Vertex and fragment shaders.
📗 WebGL shaders.
📗 Uniforms, attributes, varyings.
📗 Projection matrices.


📗 Notes and code adapted from the course taught by Professor Michael Gleicher.
📗 Please use Ctrl+F5 or Shift+F5 or Shift+Command+R or Incognito mode or Private Browsing to refresh the cached JavaScript: Code.
📗 You can print the notes: Print.
📗 Anonymous feedback can be submitted to: Form.

Prev: W9, Next: W11, Quiz of the week: Q10