The Test Framework

Throughout the class we’ll be using a test framework to check your code. Here’s what you need to know to make it all work.

If you haven’t done so already, create a directory to use for this class in your home directory or desktop (or wherever you want to save your work). We will refer to this folder as python_class. You can name it something else, just don’t forget where you put it as you will need it! Download the Exercise test files and put it in your python_class directory.

This file you downloaded (exercises.zip), when unzipped, should create a new folder called exercises. It contains the test framework we will be using, so you will be working from this folder. It also contains the files you will need to modify for most of the exercises in the class.

Confirm the Framework is Working

Note

You may need to write python3 or py instead of python to launch Python.

If that’s the case, replace python in all commands on this page with the correct command for your machine.

Change directory to your exercises directory, and run the following:

$ python test.py

You should see output like this:

Welcome to the Test Framework! ✨

To show all testable exercises run:
python test.py --list

To test an exercise run:
python test.py <EXERCISE_NAME>

The message confirms the Test Framework is working! Yay! 🎉

How to do the Exercises

There are 2 kinds of exercises:

Functions and Classes: You will implement a specific function in an existing .py file in the exercises folder. These .py files already contain skeleton functions that you will implement when you get to the corresponding exercise. Sometimes the exercise will have you change a function you previously implemented.
Modules: You will create a new .py file in the modules subfolder of the exercises folder. Sometimes the exercise will have you change a file you previously created.

It’s important that you write your code in the right place, or the tests won’t work!

How to Run the Tests

Each exercise will tell you the way to test it. You will always start by typing in the command window python test.py followed by the name given by the exercise. Don’t add arguments, only give the name of the test as shown in the exercise.

Always run the test program from your exercises folder.

Here is an example of how to test the first kind of exercise. To run tests for the get_hypotenuse function exercise, once you have edited the function in the file functions.py, you’ll type:

$ python test.py get_hypotenuse

If you try it now, you should get an error message that looks something like this:

$ python test.py get_hypotenuse
Testing get_hypotenuse

Running GetHypotenuseTests test class in functions_test.py

test_3_and_4 (functions_test.GetHypotenuseTests) ... ERROR
test_5_and_12 (functions_test.GetHypotenuseTests) ... ERROR

======================================================================
ERROR: test_3_and_4 (functions_test.GetHypotenuseTests)
----------------------------------------------------------------------
assert get_hypotenuse(3, 4) == 5.0

Traceback (most recent call last):
  File "/home/trey/exercises/functions_test.py", line 18, in test_3_and_4
    self.assertEqual(get_hypotenuse(3, 4), 5.0)
TypeError: get_hypotenuse() takes 0 positional arguments but 2 were given

HINT: Your function doesn't accept any arguments yet.
More on arguments at: https://trey.io/args

======================================================================
ERROR: test_5_and_12 (functions_test.GetHypotenuseTests)
----------------------------------------------------------------------
assert get_hypotenuse(5, 12) == 13.0

Traceback (most recent call last):
  File "/home/trey/exercises/functions_test.py", line 21, in test_5_and_12
    self.assertEqual(get_hypotenuse(5, 12), 13.0)
TypeError: get_hypotenuse() takes 0 positional arguments but 2 were given

HINT: Your function doesn't accept any arguments yet.
More on arguments at: https://trey.io/args

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=2)

Note that the test code that’s failing is shown in the output (see the assert get_hypotenuse(...) == ... lines above).

Below the test code is a traceback. Tracebacks are read from the bottom up and the last line is the most important line. The two lines above that represent either a line in your code where an error occurred or the line in the test module where an assertion failed.

If your code has passed the tests, you’ll see some variation of this message:

Testing get_hypotenuse

Running GetHypotenuseTests test class in functions_test.py

test_3_and_4 (functions_test.GetHypotenuseTests) ... ERROR
test_5_and_12 (functions_test.GetHypotenuseTests) ... ERROR

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

That last OK line means all tests pass.

For the second kind of exercise, for example where you will create the greetings.py file in the modules directory, you’ll type:

$ python test.py greetings.py

And if you run it now, you will get an error message that looks something like this:

$ python test.py greetings.py
Testing greetings.py

Running GreetingsTests test class in modules_test.py

ERROR

======================================================================
ERROR: setUpClass (modules_test.GreetingsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/trey/exercises/helpers.py", line 92, in setUpClass
    raise ValueError(
ValueError: You need to make a file called greetings.py in the modules subdirectory.

----------------------------------------------------------------------
Ran 0 tests in 0.000s

FAILED (errors=1)

And again, if this test were to pass, we’ll see that OK line at the end - just what we want!

Additional Testing Techniques

The test framework is meant to help you quickly test many different inputs against your code and quickly assert that your code performs the correct actions.

You may find using the test framework challenging to use at times. When that’s the case, you may want to try one of these approaches:

Continue using the test framework, but add a breakpoint call to your code to drop into a Python debugger (PDB) where you can type debugging commands to inspect the state of your code. The 3 most useful PDB commands are: 1. l to list the code line you’re currently on 2. n to go to the next line in your code 3. interact to enter an interactive Python prompt (a.k.a. the Python REPL) 4. dir() (not a PDB command, but a built-in function) to list your current local variables
Copy-paste your code from your text editor into a Python REPL. You’ll need to make sure you don’t have any blank lines in your blocks of code for this to work.
Copy-paste this function into your Python REPL and then call paste_code to paste a block of code and immediately run it.
Create your own file (called testme.py for example) which interacts with your function or class in specific ways and manually check the output.