It is always a good idea to test a piece of software before releasing it, or in classroom terms, before turning it in for grading. Testing of software is a complex topic, and not one that I can do justice to in a brief web page.
One kind of testing is particularly useful: automated testing. That is, one uses a computer program (e.g., a Python script) to test another computer program (e.g., another Python script). The basic idea is to write test code that expresses how the code-under-test should perform. Then, you periodically run the test script to see if anything has broken.
Suppose we have a function that calculates the sum of two numbers:
def sum_two(a, b):
return a + b
Now, suppose we want to make sure this function works correctly. We could do something like this:
print 'sum_two(42, 13) = %d' % (sum_two(42, 13))
Then, we run the script, and make sure the output is correct. But there are several disadvantages to this approach:
So instead, we want to write a function like this (the actual details will be described below):
def test_sum_two():
assert(sum_two(42, 13) == 55)
Essentially, this function says, “test that sum_two(42, 13) returns 55, and if it does not, raise
an exception.”. You can see the actual function call and comparison directly, and the assert()
function simply raises an exception when passed False.
Then, of course, we need a way to run all of our test functions and tally the results.
Let’s see how this works for real in Python.
This kind of testing — one function at a time — is called “unit testing” (because we are testing small units of code). Python comes with built-in support for creating and running unit tests.
Before we can test a Python module (i.e., a script), it must follow certain guidelines:
if __name__ == "__main__":
# Your non-function, main code goes here
# Keep it all inside of this if block
# Or, better yet, put this code in a function called main()
The goal here is to make your module a real module, in the sense that it can be safely imported into another script. And that script will be our test script, which comes next…
The basic format for the test script is this:
#!/usr/bin/python
import unittest
import MODULE-TO-BE-TESTED
class TestYourModule(unittest.TestCase):
def test_something(self):
# Add test assertions here
# As many as you like
# But typically, no more than a handful
def test_something_else(self):
# Include as many test functions as you like.
# Typically, these functions are organized by
# the function being tested, or by the test case.
def test_more(self):
# etc.
# ...
if __name__ == '__main__':
unittest.main()
All of the text that is highlighted in yellow are identifiers that you may name as you wish.
Within each test function, as shown above, you write code that runs code in your module and declares (or “asserts”) the expected outcomes. In most cases, running code from your module means calling functions in the module being tested.
Assertions are written using a handful of functions defined in the standard unittest Python module.
Note that, because tests are written as class functions, that each assertion function call begins with
the self. prefix. Here is a list of common assertion functions:
assert_(expression, message) —
Assert that the (Boolean) expression is True
assertEqual(first, second, message) —
Assert that the first and second expressions are equal
(==)
fail(message) —
Always raise an exception
Technically, all you need is the first function; all others can be written in terms of it. And also technically, the message argument is optional in each case. However, this is the message that is printed when a test fails, and so it is critical for understanding your test results; do not omit it.
So, for example, here are some concrete assertions:
self.assert_(sum_two(42, 13) == 55, 'Two positive nonzero integers') # OK way to do this test
self.assertEqual(sum_two(42, 13), 55, 'Same as above but using assertEqual()') # Best way to do this test
if sum_two(42, 13) != 55:
self.fail('Same again, but using fail()') # Horrible way to do this test
self.assert_(my_string.isupper(), 'Should be all uppercase') # Good use of assert_()
Here is a simple module that says hello to a user. Note how the critical function, which creates the final output string from user input, does not do the input or output by itself; this makes it testable.
#!/usr/bin/python
# say_hello.py
def make_greeting(the_name):
return 'Hello, ' + the_name + '!'
if __name__ == '__main__':
user_input = raw_input('Please enter your name: ')
print make_greeting(user_input)
And now, the testing script:
#!/usr/bin/python
# test_say_hello.py
import unittest
import say_hello
class TestSayHello(unittest.TestCase):
def test_make_greeting(self):
self.assertEqual(say_hello.make_greeting('Tim'), 'Hello, Tim!', 'normal case')
self.assertEqual(say_hello.make_greeting(''), 'Hello, !', 'empty string')
def test_make_greeting_2(self):
try:
say_hello.make_greeting(None)
except TypeError, e:
pass
else:
self.fail('None as argument')
if __name__ == '__main__':
unittest.main()
Running say_hello.py itself works as expected:
Please enter your name: Tim Hello, Tim!
Running test_say_hello.py runs the tests and produces much different output:
.. ---------------------------------------------------------------------- Ran 2 tests in 0.000s OK
Note that the tests did not run the “main” code in say_hello.py, which is good, because we
do not want to require user interaction during the (automated) test run. Instead, it ran the test functions in the
test script. Each “.” in the output corresponds to one test function being run. Then, following the
dashed line, we get a summary of the entire test run.
Suppose I introduce a formatting bug in make_greeting():
return 'Hello, ' + the_name
Now, running the tests shows that there is a problem:
F.
======================================================================
FAIL: test_make_greeting (__main__.TestSayHello)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./test_say_hello.py", line 11, in test_make_greeting
self.assertEqual(say_hello.make_greeting('Tim'), 'Hello, Tim!', 'normal case')
AssertionError: normal case
----------------------------------------------------------------------
Ran 2 tests in 0.000s
FAILED (failures=1)
Note the “F” instead of the first “.”, and the long FAIL message below the line of equal signs. This information helps you identify which test failed and what went wrong.