Page 2: Hello Triangle! (introduction to APIs)

The goal of the next few pages is to give you a sense of the abstractions of 3D graphics and how these appear in APIs. This should give you a sense of why we are going to use three.js in class, but also why you might look for a different API in different situations.

We are going to start with a simple demo: drawing a single triangle. This is as simple a graphics program as I can imagine. We choose a triangle because it is “the main primitive” of interactive 3D graphics: it is what the graphics hardware actually works with.

Most web browsers include four built-in APIs: HTML5 Canvas, SVG, WebGL, and WebGPU.

We are adding a 5th API (three.js) which is a layer on top of either WebGL or WebGPU. In class, we won’t actually ask you to write programs with WebGL or WebGPU directly. We will always access them through a higher-level API.

You are already familiar with Canvas and SVG from previous workbooks. These are (primarily) 2D APIs - but we should try to understand why we make that distinction.

The programs in this section were written by Gemini. I try to avoid writing WebGL programs myself. I also have no experience with WebGPU.

The lecture videos take a slightly different path, focusing on how APIs connect to 3D cues. They use a cube instead of a triangle and do not cover the other APIs.

A Triangle

Here are 4 different programs that all draw the same triangle. You should see 4 different “windows,” each with a black background and a single yellow triangle.

If you want to see the actual programs, they are:

• HTML5 Canvas: triangle_canvas2d.js
• Three.JS: triangle_threejs.js
• WebGL: triangle_webgl.js
• WebGPU: triangle_webgpu.js

If the WebGPU version doesn’t work in your web browser, don’t worry: we won’t use WebGPU again in this workbook. But you should be aware of this if you encounter problems using WebGPU.

What all these programs do is draw a single yellow triangle. They do it in quite different ways.

With Canvas, we directly describe things in 2D. We make a path, draw the corners of a triangle, and fill it with yellow. You can guess what the program looks like (since you’ve written similar ones), or you can look at triangle_canvas2d.js .

A Triangle in Three.js

With Three.js, we need to create the triangle in 3D and draw a picture of it. We need to use all of the 3D machinery. Each part of the program corresponds to an idea of what we need to make 3D pictures (from the previous page). We’ll discuss the details later, but for now make sure you see the connection between the abstractions in the API and the things we want to think about when doing 3D graphics:

1. We need to create a Scene - the 3D world that will have our objects in it.
2. We define the triangle in 3D space. In three, our objects will be Mesh objects (that contain collections of triangles, which three calls Geometry). These objects encapsulate the notion of transformations that put them into the right place in the scene.
3. We need to make the triangle interact with light. In three, we do this by giving it a Material - the stuff the object is made from. In this case, the material is a special “basic” material that just shows up as yellow, no matter what the lighting is.
4. We need to map from positions in the 3D scene to positions in the 2D screen. We can think of this as taking a picture of the 3D world with a camera that is placed in it. Three abstracts the viewing of the 3D world in a special kind of object called a Camera.
5. We need to tell three that we want to actually make a picture of the scene using a camera, coloring the pixels appropriately. This step will also take care of occlusions (making sure closer objects cover farther ones). Three encapsulates the drawing process in something it calls a Renderer.

Here is the code from triangle_threejs.js with the comments removed. Notice how the abstractions line up with what we have been discussing. The program is a little long (since we need to make 3D structures that we aren’t going to use).

 1import * as THREE from 'three';
 2const canvas = document.getElementById('threejsCanvas');
 3
 4const scene = new THREE.Scene();
 5
 6const vertices = new Float32Array([
 7     0.0,  0.5, 0.0, // Top
 8    -0.5, -0.5, 0.0, // Bottom Left
 9     0.5, -0.5, 0.0  // Bottom Right
10]);
11
12const geometry = new THREE.BufferGeometry();
13geometry.setAttribute('position', new THREE.BufferAttribute(vertices, 3));
14
15const material = new THREE.MeshBasicMaterial({ color: 0xffff00 });
16const triangle = new THREE.Mesh(geometry, material);
17
18scene.add(triangle);
19
20const camera = new THREE.OrthographicCamera(-1, 1, 1, -1, 0.1, 10);
21camera.position.z = 1;
22
23const renderer = new THREE.WebGLRenderer({ canvas, antialias: true });
24renderer.setSize(canvas.clientWidth, canvas.clientHeight, false);
25renderer.render(scene, camera);

You can see that we create a scene (line 4), make a 3D object (each vertex has 3 coordinates) (lines 6-13). Lines 15-16 make the material and the mesh object (we need to have the material to make the object out of before we can make the object). Line 18 puts the triangle into the scene. Lines 20-21 create and position the camera. Lines 23-25 create the renderer, and render the picture.

We’ll explore all of this in more detail in this workbook.

If you look at the three program, you will observe that most of the “graphics” is hidden:

1. Hardware Setup (prepare “window”)
   • Automatically creates canvas with settings
2. Triangle Drawing
   • Done by the hardware (or low-level drivers)
3. Shader Programs (program hardware for colors)
   • Standard shaders for common (fancy) lighting
   • Easy to write your own
4. Procedural drawing (specifying to draw the triangles)
   • Sends the “world” to the hardware
   • hardware takes care of occlusion (in an order-independent way)
5. Viewing Transformation
   • Camera Object
   • Matrices inside
6. Modeling Transformations
   • Matrices inside of objects
   • Can specify in many different ways
7. Making Triangles
   • High-level “Graphics Objects”
   • Meshes (collections of triangles)
   • Automatically organizes for the hardware

Basically, three.js gives us nice abstractions for the key graphics concepts:

The video walks through a similar example.

WebGL: A Hardware-Oriented API

The original “built-in” API in the web browser is WebGL. You might wonder “why don’t we just use that directly?” Looking at the triangle example will give us some clues.

If you look at the program (in triangle_webgl.js ), you will notice a few things:

1. It is much longer than the Three.js program. There is much more that we have to do, because WebGL provides a much lower level of abstraction than THREE.
2. The abstractions of WebGL correspond to how the hardware operates, rather than the graphics concepts. All of the graphics work still needs to be done, but it needs to be described in terms of the hardware (rather than the graphics abstractions).

Let me give you some examples…

In three, we made a “geometry object” that contained the triangles. With WebGL, we need to figure out how to put the geometric information into appropriate organizations of the data in memory, and to transfer that memory to the graphics hardware so it can access it directly.

In three, we made a “camera” that mapped from the positions in the scene to positions on the screen. In WebGL, we need to write a program that runs directly on the graphics hardware to do this.

In three, we made a “material” that colored our object yellow. In WebGL, we must write a program for the graphics hardware that colors the pixels yellow.

These “programs for the graphics hardware” are written in a different language (it is GLSL, not JavaScript). Notice how the program’s text is contained in a string, and we need to run the compiler and move the program to the hardware:

 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// Vertex shader just passes the position directly to clip space
const vsSource = `
    attribute vec2 aPosition;
    void main() {
        gl_Position = vec4(aPosition, 0.0, 1.0);
    }
`;
// Fragment shader hardcodes solid yellow
const fsSource = `
    void main() {
        gl_FragColor = vec4(1.0, 1.0, 0.0, 1.0);
    }
`;
function compileShader(gl, type, source) {
    const shader = gl.createShader(type);
    gl.shaderSource(shader, source);
    gl.compileShader(shader);
    return shader;
}

With WebGL you need to program the graphics hardware directly in this special programming language. This is called shader programming and we will learn about it in a future workbook. With three.js, we can write our own shader programs - or we can use the ones that the API provides. Also, the shaders are encapsulated into appropriate abstractions that match the concepts of 3D graphics: cameras, objects, lights, materials, etc.

In fact, WebGL doesn’t provide 3D abstractions at all. If you want to do 3D computations, you need to program them yourself. The example program does everything in 2D.

Why WebGL? (or hardware-oriented APIs)

WebGL gives the programmer low-level control of the graphics hardware. It allows programmers to do everything themselves, so they can make choices on how to achieve good performance. The downside is that the programmer has to make all of these choices. Most of the time, we can allow the experienced developers of “high level APIs” like three to make those choices for us.

While WebGL was designed to map directly to the hardware, it was designed a long time ago when the hardware was simpler. As graphics hardware got better, WebGL was stuck in roughly 2012.

WebGPU is a newer API that also is designed to provide direct access to the hardware. Like WebGL, it requires programmers to think in terms of hardware abstractions (like memory buffers, state machines and shader programs). However, it is designed to be flexible to account for more modern graphics hardware, and has some design choices that allow it to use the hardware extremely efficiently. Unfortunately, this makes it even more complex.

If you look at the WebGPU triangle program in triangle_webgpu.js , you will see that it is even longer than the WebGL program, and uses even more complex (hardware-oriented) concepts like “command queues” and “pipelines”.

But there is good news with this: the developers of three.js have designed Three.js so it can map to different “back-ends”. You can switch three to use WebGPU (rather than WebGL) and get the advantages of the newer API. For class, we’ll stick with the old way (since it is more reliable). But I think there is an important lesson: the core abstractions of 3D graphics remain the same, even if the “under-the-hood” implementation details change.

On the next page, we’ll consider an even more complex example to appreciate the abstractions of a high-level graphics API.