The goal of optimization is to produce better code (fewer instructions, and, more importantly, code that runs faster). However, it is important not to change the behavior of the program (what it computes)!

We will look at the following ways to improve a program:

1. Peephole Optimization. This is done after code generation. It involves finding opportunities to improve the generated code by making small, local changes.
2. Moving Loop-Invariant Computations. This is done before code generation. It involves finding computations inside loops that can be moved outside, thus speeding up the execution time of the loop.
3. Strength-Reduction in for Loops. This is done before code generation. It involves replacing multiplications inside loops with additions. If it takes longer to execute a multiplication than an addition, then this speeds up the code.
4. Copy Propagation. This is done before code generation. It involves replacing the use of a variable with a literal or another variable. Copy propagation can sometimes uncover more opportunities for moving loop-invariant computations. It may also make it possible to remove some assignments from the program, thus making the code smaller and faster.

Peephole Optimization

The idea behind peephole optimization is to examine the code "through a small window," looking for special cases that can be improved. Below are some common optimizations that can be performed this way. Note that in all cases that involve removing an instruction, it is assumed that that instruction is not the target of a branch.

1. Remove a redundant load (fewer instructions generated, and fewer executed):

          store Rx, M	\	store Rx, M
          load  M, Rx	/
          

2. Remove a redundant push/pop (fewer instructions generated, and fewer executed):

          push Rx         \
          pop into Rx     /        ... nothing ...
          

3. Replace a jump to a jump (same number of instructions generated, but fewer executed):

              goto L1     \		  goto L2
                :	        >		    :
          L1: goto L2     /	      L1: goto L2
          

4. Remove a jump to the next instruction (fewer instructions generated, and fewer executed):

         goto L1    \                   L1: ...
     L1: ...        /
     

5. Replace a jump around jump (fewer instructions generated; possibly fewer executed):

             if T0 = 0 goto L1       \       if T0 != 0 goto L2
             else goto L2             >  L1: ......
          L1: ....                   /
          

6. Remove useless operations (fewer instructions generated and fewer executed):

          add T0, T0, 0    // changes nothing; remove it
          mul T0, T0, 1    // changes nothing; remove it
          

7. Reduction in strength (same number of instructions, but faster):

           mul T0, T0, 2       -> shift-left T0
           add T0, T0, 1       -> inc T0
   	

       load  Tx, M     \       load  Tx, M   \  load  Tx, M
       add   Tx, 0      >      store Tx, M   /
       store Tx, M     /

The ideas behind this optimization are:

• For greatest gain, optimize "hot spots" i.e., inner loops.
• An expression is "loop invariant" if the same value is computed for that expression on every iteration of the loop.
• So instead of computing the same value over and over, compute the value once outside the loop and reuse it.

Example:

for (i=0; i<100; i++) {
    for (j=0; j<100; j++) {
        for (k=0; k<100; k++) {
            A[i][j][k] = i*j*k
        }
    }
}

• A is a 3D array
• each element requires 4 bytes
• elements are stored in the current activation record in row-major order (note: in Java, arrays are allocated from the heap, not stored on the stack; however, in other languages they may be stored on the stack)

address = FP + <offset of A> - (i*10,000*4)-(j*100*4)-(k*4)

So the code for the inner loop is actually something like:

T0 = i*j*k
T1 = FP + <offset of A> - i*40000 -j*400 - k*4
Store T0, 0(T1)

invariant to i loop: FP + <offset of A>
invariant to j loop: i*40000
invariant to k loop: i*j, j*400

We can move the computations of the loop-invariant expressions out of their loops, assigning the values of those expressions to new temporaries, and then using the temporaries in place of the expressions. When we do that for the example above, we get:

tmp0 = FP + <offset of A>
for (i=0; i<100; i++) {
    tmp1 = tmp0 - i*40000
    for (j=0; j<100; j++) {
        tmp2 = tmp1 - j*400
        tmp3 = i*j
        for (k=0; k<100; k++) {
            T0 = tmp3 * k         // T0 is i*j*k
            T1 = tmp2 - k*4       // T1 is the address of A[i][j][k]
            store T0, 0(T1)       // store i*j*k into A[i][j][k]
        }
    }
}

Here is a comparison of the original code and the optimized code (the number of instructions performed in the innermost loop, which is executed 1,000,000 times):

Questions:

1. How do we recognize loop-invariant expressions?
2. When and where do we move the computations of those expressions?

1. An expression is invariant with respect to a loop if for every operand, one of the following holds:
   1. It is a literal, or
   2. It is a variable that gets its value only from outside the loop.

2. To answer question 2, we need to consider safety and profitability.

Safety:

If evaluating the expression might cause an error, then there is a possible problem if the expression might not be executed in the original, unoptimized code. For example:

b = a;
while (a != 0) {
    x = 1/b; // Possible divide by zero if 1/b is moved out of the loop
    a--;
}
  

What about preserving the order of events? e.g. if the unoptimized code performed output then had a runtime error, is it valid for the optimized code to simply have a runtime error? Also note that changing the order of floating-point computations may change the result, due to differing precisions.

Profitability:

If the computation might not execute in the original program, moving the computation might actually slow the program down!

Moving a computation is both safe and profitable if one of the following holds:

1. It can be determined that the loop will execute at least once and the code is guaranteed to execute if the loop does:
   • it isn't inside any condition, or
   • it is on all paths through the loop (e.g., it occurs in both branches of an if-then-else).
2. The expression is in (a non short-circuited) part of the loop test / loop bounds, e.g.:
   while (x < i + j * 100) // j*100 will always be evaluated

Optimization #2: Strength reduction in for loops

1. Again, concentrate on "hot spots"
2. Replace expensive operations (*) with cheaper ones (+)

Given a loop of the form: "for i from low to high do ..."

An induction expression is an expression that is used inside the loop, and has the form: "i * k1 + k2"

where "i" is the loop index, and "k1" and "k2" are constant with respect to the loop.

Consider the sequences of values for variable i and for the induction expression: 

• computing low * k1 + k2 once before the loop,
• storing that value in a temporary,
• using the temporary instead of the expression inside the loop, and
• incrementing the temporary by k1 at the end of the loop.

temp = low * k1 + k2   // initialize temp to first value of induction exp
for i from low to high do
    ... temp ...      // Use temp in place of i * k1 + k2
    temp = temp + k1  // increment temp at the end of the loop by adding k1
end

Optimization #3: Copy propagation

1. no other definition of x reaches u, and
2. y can't change between d and u

Examples:

x = y                     x = y                           x = y
a = x + z // x can be     if (...) x = 2                  if (...) y = 3
          // replaced     a = x + z  // x cannot be       a = x + z  // x cannot
          // with y                  // replaced with                // be replaced
				     // y; violates                  // with y;
				     // condition 1                  // violates
				                                     // condition 2

Question: Why is this a useful transformation?

Answers:

• If all uses of x reached by d are replaced, then definition d is useless, and can be removed.

• Even if the definition cannot be removed, copy propagation can lead to improved code:
  1. If the definition is actually of the form: x = literal, then copy propagation can create opportunities for better code:

     	    x = 5;       ==>   x = 5;
     	    a = b + x;         a = b + 5;
     	    

     For a machine like the MIPS, that allows an operand to be an "immediate" (literal) value but not a memory location, better code can be generated for the transformed version (that uses a = b + 5) than for the original version (that uses a = b + x), because we can avoid loading the value of x into a register. Furthermore, this kind of copy propagation can lead to opportunities for constant folding: evaluating, at compile time, an expression that involves only literals. For example:

     	    x = 5;       ==>   x = 5;      ==>  x = 5;
     	    a = 3 + x;         a = 3 + 5;       a = 8;
     	    

  2. Sometimes copy propagation can be combined with moving loop-invariant computations out of the loop, to lead to a better overall optimization. For example:

     	   while (...) {
     	      x = a * b; // loop-inv
     	      y = x * c;
     	      ...
     	   }
     	   

     Move "a * b" out of the loop:

     	   tmp1 = a * b;
     	   while (...) {
     	      x = tmp1;
     	      y = x * c;
     	      ...
     	   }
     	   

     Note that at this point, even if c is not modified in the loop, we cannot move "x * c" out of the loop, because x gets its value inside the loop. However, after we do copy propagation:

     	   tmp1 = a * b;
     	   while (...) {
     	      x = tmp1;
     	      y = tmp1 * c;
     	      ...
     	   }
     	   

     "tmp1 * c" can also be moved out of the loop:

     	   tmp1 = a * b;
     	   tmp2 = tmp1 * c;
     	   while (...) {
     	      x = tmp1;
     	      y = tmp2;
     	      ...
     	   }
     	   

Given a definition d that is a copy statement: x = y, and a use u of x, we must determine whether the two important properties hold that permit the use of x to be replaced with y.

The first property (use u is reached only by definition d) is best solved using the standard "reaching-definitions" dataflow-analysis problem, which computes, for each definition of a variable x, all of the uses of x that might be reached by that definition. Note that this property can also be determined by doing a backward depth-first or breadth-first search in the control-flow graph, starting at use u, and terminating a branch of the search when a definition of x is reached. If definition d is the only definition encountered in the search, then it is the only one that reaches use u. (This technique will, in general, be less efficient than doing reaching-definitions analysis.)

The second property (that variable y cannot change its value between definition d and use u), can also be verified using dataflow analysis, or using a backwards search in the control-flow graph starting at u, and quitting at d. If no definition of y is encountered during the search, then its value cannot change, and the copy propagation can be performed. Note that when y is a literal, property 2 is always satisfied.

Below is our running example (after doing reduction in strength). The copy statements are shown in bold red font. In this particular example, each variable x that is defined in a copy statement reaches only one use. Comments indicate which of them cannot be propagated (because of a violation of property (1) -- in this example there are no instances where property (2) is violated).

tmp0 = FP + offset A
temp1 = tmp0            // cannot be propagated
for (i=0; i<100; i++) {
    tmp1 = temp1
    temp2 = tmp1        // cannot be propagated
    temp3 = 0           // cannot be propagated
    for (j=0; j<100; j++) {
        tmp2 = temp2
        tmp3 = temp3
	temp4 = 0       // cannot be propagated
	temp5 = tmp2    // cannot be propagated
        for (k=0; k<100; k++) {
            T0 = temp4
            T1 = temp5
            store T0, 0(T1)
	    temp4 = temp4 + tmp3
	    temp5 = temp5 - 4
        }
	temp2 = temp2 - 400
	temp3 = temp3 + i
    }
    temp1 = temp1 - 40000
}

tmp0 = FP + offset A
temp1 = tmp0
for (i=0; i<100; i++) {
    temp2 = temp1
    temp3 = 0
    for (j=0; j<100; j++) {
	temp4 = 0
	temp5 = temp2
        for (k=0; k<100; k++) {
            store temp4 0(temp5)
	    temp4 = temp4 + temp3
	    temp5 = temp5 - 4
        }
	temp2 = temp2 - 400
	temp3 = temp3 + i
    }
    temp1 = temp1 - 40000
}