# CS 368-2 (2012 Spring) — Introduction to Automated Testing

It is always a good idea to test a piece of software before releasing it, or in classroom terms, before turning it in for grading. Testing of software is a complex topic, and not one that I can do justice to in a brief web page.

One kind of testing is particularly useful: automated testing. That is, one uses a computer program (e.g., a Python script) to test another computer program (e.g., another Python script). The basic idea is to write test code that expresses how the code-under-test should perform. Then, you periodically run the test script to see if anything has broken.

## Trivial Example

Suppose we have a function that calculates the sum of two numbers:

```def sum_two(a, b):
return a + b```

Now, suppose we want to make sure this function works correctly. We could do something like this:

`print 'sum_two(42, 13) = %d' % (sum_two(42, 13))`

Then, we run the script, and make sure the output is correct. But there are several disadvantages to this approach:

• The testing output is printed every time the script runs, which probably gets in the way of real output
• A person must verify that the output is correct, and we make mistakes sometimes
• Tests are interleaved with regular code, which makes the script hard to understand

So instead, we want to write a function like this (the actual details will be described below):

```def test_sum_two():
assert(sum_two(42, 13) == 55)```

Essentially, this function says, “test that `sum_two(42, 13)` returns 55, and if it does not, raise an exception.”. You can see the actual function call and comparison directly, and the `assert()` function simply raises an exception when passed `False`.

Then, of course, we need a way to run all of our test functions and tally the results.

Let’s see how this works for real in Python.

## Python Unit Testing

This kind of testing — one function at a time — is called “unit testing” (because we are testing small units of code). Python comes with built-in support for creating and running unit tests.

### Preparing the Main Script

Before we can test a Python module (i.e., a script), it must follow certain guidelines:

• Most code should be contained in functions. It does not matter whether the functions are within a class or not.
• Functions should be as self-contained as possible; i.e., minimize use of global variables, print statements, etc.
• All code that is not in functions should go in a block like this:
```if __name__ == "__main__":
# Your non-function, main code goes here
# Keep it all inside of this if block
# Or, better yet, put this code in a function called main()```

The goal here is to make your module a real module, in the sense that it can be safely imported into another script. And that script will be our test script, which comes next…

### Writing the Test Script

The basic format for the test script is this:

```#!/usr/bin/python

import unittest

import MODULE-TO-BE-TESTED

class TestYourModule(unittest.TestCase):

def test_something(self):
# As many as you like
# But typically, no more than a handful

def test_something_else(self):
# Include as many test functions as you like.
# Typically, these functions are organized by
# the function being tested, or by the test case.

def test_more(self):
# etc.

# ...

if __name__ == '__main__':
unittest.main()```

All of the text that is highlighted in yellow are identifiers that you may name as you wish.

### Writing the Tests

Within each test function, as shown above, you write code that runs code in your module and declares (or “asserts”) the expected outcomes. In most cases, running code from your module means calling functions in the module being tested.

Assertions are written using a handful of functions defined in the standard `unittest` Python module. Note that, because tests are written as class functions, that each assertion function call begins with the `self.` prefix. Here is a list of common assertion functions:

• `assert_(expression, message)` — Assert that the (Boolean) expression is `True`
• `assertEqual(first, second, message)` — Assert that the `first` and `second` expressions are equal (`==`)
• `fail(message)` — Always raise an exception

Technically, all you need is the first function; all others can be written in terms of it. And also technically, the message argument is optional in each case. However, this is the message that is printed when a test fails, and so it is critical for understanding your test results; do not omit it.

So, for example, here are some concrete assertions:

`self.assert_(sum_two(42, 13) == 55, 'Two positive nonzero integers')   # OK way to do this test`
`self.assertEqual(sum_two(42, 13), 55, 'Same as above but using assertEqual()')   # Best way to do this test`
```if sum_two(42, 13) != 55:
self.fail('Same again, but using fail()')   # Horrible way to do this test```
`self.assert_(my_string.isupper(), 'Should be all uppercase')   # Good use of assert_()`

### A Complete (If Basic) Example

Here is a simple module that says hello to a user. Note how the critical function, which creates the final output string from user input, does not do the input or output by itself; this makes it testable.

```#!/usr/bin/python

# say_hello.py

def make_greeting(the_name):
return 'Hello, ' + the_name + '!'

if __name__ == '__main__':
print make_greeting(user_input)```

And now, the testing script:

```#!/usr/bin/python

# test_say_hello.py

import unittest
import say_hello

class TestSayHello(unittest.TestCase):

def test_make_greeting(self):
self.assertEqual(say_hello.make_greeting('Tim'), 'Hello, Tim!', 'normal case')
self.assertEqual(say_hello.make_greeting(''), 'Hello, !', 'empty string')

def test_make_greeting_2(self):
try:
say_hello.make_greeting(None)
except TypeError, e:
pass
else:
self.fail('None as argument')

if __name__ == '__main__':
unittest.main()```

Running `say_hello.py` itself works as expected:

```Please enter your name: Tim
Hello, Tim!```

Running `test_say_hello.py` runs the tests and produces much different output:

```..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK```

Note that the tests did not run the “main” code in `say_hello.py`, which is good, because we do not want to require user interaction during the (automated) test run. Instead, it ran the test functions in the test script. Each “.” in the output corresponds to one test function being run. Then, following the dashed line, we get a summary of the entire test run.

Suppose I introduce a formatting bug in `make_greeting()`:

`return 'Hello, ' + the_name`

Now, running the tests shows that there is a problem:

```F.
======================================================================
FAIL: test_make_greeting (__main__.TestSayHello)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./test_say_hello.py", line 11, in test_make_greeting
self.assertEqual(say_hello.make_greeting('Tim'), 'Hello, Tim!', 'normal case')
AssertionError: normal case

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (failures=1)```

Note the “F” instead of the first “.”, and the long FAIL message below the line of equal signs. This information helps you identify which test failed and what went wrong.