12 Testing

It is of great importance to test our code. This allows us not only to avoid errors and problems while running our code but can save in total a lot of time.

Writing tests is hardly ever the main focus of a developer but it should be included as an essential part of every piece of code. There exist a style of developing called test driven development summarized in Figure 12.1.

Figure 12.1: Lifecycle of the Test-Driven Development method - Xarawn, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons

There are different types of tests that are appropriate for different types of software and are necessary in different kinds of projects, the following descriptions are adopted from Atlassian, accessed 22.11.2024.

Unit tests
Unit tests are very low level and close to the source of an application. They consist in testing individual methods and functions of the classes, components, or modules used by your software. Unit tests are generally quite cheap to automate and can run very quickly by a continuous integration server.
Integration tests
Integration tests verify that different modules or services used by your application work well together. For example, it can be testing the interaction with the database or making sure that microservices work together as expected. These types of tests are more expensive to run as they require multiple parts of the application to be up and running.
Functional tests
Functional tests focus on the business requirements of an application. They only verify the output of an action and do not check the intermediate states of the system when performing that action.

There is sometimes a confusion between integration tests and functional tests as they both require multiple components to interact with each other. The difference is that an integration test may simply verify that you can query the database while a functional test would expect to get a specific value from the database as defined by the product requirements.
End-to-end tests
End-to-end testing replicates a user behavior with the software in a complete application environment. It verifies that various user flows work as expected and can be as simple as loading a web page or logging in or much more complex scenarios verifying email notifications, online payments, etc…

End-to-end tests are very useful, but they’re expensive to perform and can be hard to maintain when they’re automated. It is recommended to have a few key end-to-end tests and rely more on lower level types of testing (unit and integration tests) to be able to quickly identify breaking changes.
Acceptance testing
Acceptance tests are formal tests that verify if a system satisfies business requirements. They require the entire application to be running while testing and focus on replicating user behaviors. But they can also go further and measure the performance of the system and reject changes if certain goals are not met.
Performance testing
Performance tests evaluate how a system performs under a particular workload. These tests help to measure the reliability, speed, scalability, and responsiveness of an application. For instance, a performance test can observe response times when executing a high number of requests, or determine how a system behaves with a significant amount of data. It can determine if an application meets performance requirements, locate bottlenecks, measure stability during peak traffic, and more.
Smoke testing
Smoke tests are basic tests that check the basic functionality of an application. They are meant to be quick to execute, and their goal is to give you the assurance that the major features of your system are working as expected.

Smoke tests can be useful right after a new build is made to decide whether or not you can run more expensive tests, or right after a deployment to make sure that they application is running properly in the newly deployed environment.

We are going to look primarily on Unit tests here.

12.1 Unit Tests

As mentioned above unit tests are meant to make sure single sections of your code provide the functionality as they should.

12.1.1 Via `assert`

One way to write a unit test is via assert statements, as they can also be deactivated during runtime, see Section 10.3, for this kind of unit tests an AssertionError signals a not successful test. Usually we write a separate function to perform the tests with the prefix test_

import numbers
import numpy as np
import traceback

def add(a: numbers.Number, b: numbers.Number) -> numbers.Number:
    return a + b

def test_add():
    assert add(2, 1) == 3
    assert add(0, 0) == 0
    assert add(0.4, 1/3) == 0.7333333333333334
    assert add(1, 1e-16) == 1
    np.testing.assert_almost_equal(add(1, 1e-16), 1)

try:
  test_add()
except AssertionError as ae:
  print(f"Test failed!, {traceback.print_exception(ae)}")
else:
  print("Test passed!")

Test passed!

This is tedious and not very convenient to handle and work with.

12.1.2 Via `unittest`

The module unittest is a much more comprehensive way of running your test. This is also where the directory tests from our introduction to pdm comes into play, see Chapter 2.

Tests are structured in this directory. We constructed a dummy project to show how testing can be done.

$ tree module/
module/
├── pdm.lock
├── pyproject.toml
├── src
│   └── rectangle
│       ├── __init__.py
│       └── rectangle.py
└── tests
1    └── rectangle
2        ├── __init__.py
3        └── test_rectangle.py

1: create a directory per module (directory) in your src folder with the same name
2: optionally make the test folder a module as well
3: create a file containing the tests with the same name

Code for the example

module/src/rectangle/__init__.py

from .rectangle import *

module/src/rectangle/rectangle.py

class Rectangle:
    def __init__(self, width, height):
        self.width = width
        self.height = height

    def get_area(self):
        return self.width * self.height

    def set_width(self, width):
        self.width = width

    def set_height(self, height):
        self.height = height

module/tests/rectangle/__init__.py

from .test_rectangle import *

module/tests/rectangle/test_rectangle.py

import unittest
from src.rectangle import Rectangle


class TestGetAreaRectangle(unittest.TestCase):
    def test_area_correct(self):
        rectangle = Rectangle(2, 3)
        self.assertEqual(rectangle.get_area(), 6, "incorrect area")

    def test_area_incorrect(self):
        rectangle = Rectangle(2, 3)
        self.assertNotEqual(rectangle.get_area(), 5, "correct area")

    def test_set_width(self):
        rectangle = Rectangle(1, 1)
        rectangle.set_width(0.7)
        self.assertEqual(rectangle.get_area(), 0.7, "set_width not working")

    def test_set_height(self):
        rectangle = Rectangle(1, 1)
        rectangle.set_width(0.7)
        self.assertEqual(rectangle.get_area(), 0.7, "set_width not working")
    
    def test_deliberate_error(self):
        rectangle = Rectangle(1, 1)
        self.assertEqual(rectangle.get_area(), 2, "deliberate error")

We can run the tests from the terminal (note the change of directory is only necessary for these notes)

pdm run python -m unittest -v tests.rectangle

INFO: Inside an active virtualenv 
/home/runner/work/MECH-M-DUAL-1-SWD/MECH-M-DUAL-1-SWD/.venv, reusing it.
Set env var PDM_IGNORE_ACTIVE_VENV to ignore it.
test_area_correct (tests.rectangle.test_rectangle.TestGetAreaRectangle.test_area_correct) ... ok
test_area_incorrect (tests.rectangle.test_rectangle.TestGetAreaRectangle.test_area_incorrect) ... ok
test_deliberate_error (tests.rectangle.test_rectangle.TestGetAreaRectangle.test_deliberate_error) ... FAIL
test_set_height (tests.rectangle.test_rectangle.TestGetAreaRectangle.test_set_height) ... ok
test_set_width (tests.rectangle.test_rectangle.TestGetAreaRectangle.test_set_width) ... ok

======================================================================
FAIL: test_deliberate_error (tests.rectangle.test_rectangle.TestGetAreaRectangle.test_deliberate_error)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/MECH-M-DUAL-1-SWD/MECH-M-DUAL-1-SWD/_assets/errorhandling/module/tests/rectangle/test_rectangle.py", line 26, in test_deliberate_error
    self.assertEqual(rectangle.get_area(), 2, "deliberate error")
AssertionError: 1 != 2 : deliberate error

----------------------------------------------------------------------
Ran 5 tests in 0.001s

FAILED (failures=1)

12.1.3 Via `pytest`

A third party option is the module pytest docs that is often considered easier to handle than unittest. As there is a compatibility between pytest and unittest we can run

pdm run pytest -q

INFO: Inside an active virtualenv 
/home/runner/work/MECH-M-DUAL-1-SWD/MECH-M-DUAL-1-SWD/.venv, reusing it.
Set env var PDM_IGNORE_ACTIVE_VENV to ignore it.

..F..                                                                    [100%]
=================================== FAILURES ===================================
__________________ TestGetAreaRectangle.test_deliberate_error __________________

self = <tests.rectangle.test_rectangle.TestGetAreaRectangle testMethod=test_deliberate_error>

    def test_deliberate_error(self):
        rectangle = Rectangle(1, 1)
>       self.assertEqual(rectangle.get_area(), 2, "deliberate error")
E       AssertionError: 1 != 2 : deliberate error

tests/rectangle/test_rectangle.py:26: AssertionError
=========================== short test summary info ============================
FAILED tests/rectangle/test_rectangle.py::TestGetAreaRectangle::test_deliberate_error - AssertionError: 1 != 2 : deliberate error
1 failed, 4 passed in 0.08s

as it will automatically search in the directory tests for files with the prefix test_.

12.1 Unit Tests

12.1.1 Via assert

12.1.2 Via unittest

12.1.3 Via pytest

12.1.1 Via `assert`

12.1.2 Via `unittest`

12.1.3 Via `pytest`