Testing with pytest - part 1

Why to write tests?

  • Who wants to perform manual testing?
  • When you fix a bug or add a new feature, tests are a way to verify that you did not break anything on the way
  • If you have clear requirements, you can have matching test(s) for each requirement
  • You don't have to be afraid of refactoring
  • Tests document your implementation - they show other people use cases of your implementation
  • This list is endless...

Test-driven development aka TDD

In short, the basic idea of TDD is to write tests before writing the actual implementation. Maybe the most significant benefit of the approach is that the developer focuses on writing tests which match with what the program should do. Whereas if the tests are written after the actual implementation, there is a high risk for rushing tests which just show green light for the already written logic.

Tests are first class citizens in modern, agile software development, which is why it's important to start thinking TDD early during your Python learning path.

The workflow of TDD can be summarized as follows:

  1. Add a test case(s) for the change / feature / bug fix you are going to implement
  2. Run all tests and check that the new one fails
  3. Implement required changes
  4. Run tests and verify that all pass
  5. Refactor

Running pytest inside notebooks

These are the steps required to run pytest inside Jupyter cells. You can copy the content of this cell to the top of your notebook which contains tests.

In [1]:
# Let's make sure pytest and ipytest packages are installed
# ipytest is required for running pytest inside Jupyter notebooks
import sys
!{sys.executable} -m pip install pytest
!{sys.executable} -m pip install ipytest

import ipytest.magics
import pytest

# Filename has to be set explicitly for ipytest 
__file__ = 'testing1.ipynb'
Requirement already satisfied: pytest in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (3.5.0)
Requirement already satisfied: more-itertools>=4.0.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (4.1.0)
Requirement already satisfied: setuptools in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (39.0.1)
Requirement already satisfied: attrs>=17.4.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (17.4.0)
Requirement already satisfied: six>=1.10.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (1.11.0)
Requirement already satisfied: py>=1.5.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (1.5.3)
Requirement already satisfied: pluggy<0.7,>=0.5 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (0.6.0)
Requirement already satisfied: ipytest in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (0.2.2)

pytest test cases

Let's consider we have a function called sum_of_three_numbers for which we want to write a test.

In [2]:
# This would be in your e.g. implementation.py
def sum_of_three_numbers(num1, num2, num3):
    return num1 + num2 + num3

Pytest test cases are actually quite similar as you have already seen in the exercises. Most of the exercises are structured like pytest test cases by dividing each exercise into three cells:

  1. Setup the variables used in the test
  2. Your implementation
  3. Verify that your implementation does what is wanted by using assertions

See the example test case below to see the similarities between the exercises and common structure of test cases.

In [3]:
%%run_pytest[clean]
# Mention this at the top of cells which contain test(s)
# This is only required for running pytest in Jupyter notebooks


# This would be in your test_implementation.py
def test_sum_of_three_numbers():
    # 1. Setup the variables used in the test
    num1 = 2
    num2 = 3
    num3 = 5
    
    # 2. Call the functionality you want to test
    result = sum_of_three_numbers(num1, num2, num3)
    
    # 3. Verify that the outcome is expected
    assert result == 10
=============================================================== test session starts ================================================================
platform darwin -- Python 3.5.4, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
rootdir: /Users/jerry/github/jerry-git/learn-python3, inifile: pytest.ini
plugins: nbval-0.9.0
collected 1 item

testing1.py .                                                                                                                                [100%]

============================================================= 1 passed in 0.01 seconds =============================================================

Now go ahead and change the line assert result == 10 such that the assertion fails to see the output of a failed test.