Handling repetitive tests in Django

When writing tests (unit and/or functional), one of the goal is to cover all edge-cases. DDT is a great way to write such tests; here is how to do so in Python and Django.

DDT introduction

For the definition, let’s quote the author of the Python DDT library.

DDT (Data-Driven Tests) allows you to multiply one test case by running it with different test data, and make it appear as multiple test cases.

Carles Barrobés

Let’s break down that definition with a simplistic example. We are testing aud_to_usd(value) method. That hypothetical method converts Australian dollars to USDs.

1.Multiply one test case by running it with different test data

You would for example have a test like test_aud_to_usd_ok() with various data:

AUD 1 => USD 0.76
AUD 5 => USD 3.78
AUD 0 => USD 0
…

2. Make it appear as multiple test cases

Every single one of the above examples should run independently. In other words, for each case, one unique test should fail or pass.

Issues with using a forloop

The first requirement can easily be achieved via a forloop within a test. The code would look like this:

class ConvertionTestCase(unittest.TestCase):

    def test_aud_to_usd_ok(self):
        for value, result in (
            (1, 0.76),
            (1.00, 0.76),
            (5, 3.78),
            (5.00, 3.78),
            (0, 0),
        ):
            self.assertEquals(aud_to_usd(value), result)

Running it would gives me a single error or a single passing test. In case of failure, I wouldn’t know which test has failed (below, is it 1 or 1.00?) and if the following tests would have passed or not:

nosetests -v convert.py 
test_aud_to_usd_ok (convert.ConvertionTestCase) ... FAIL

======================================================================
FAIL: test_aud_to_usd_ok (convert.ConvertionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../convert.py", line 21, in test_aud_to_usd_ok
    self.assertEquals(aud_to_usd(value), result)
AssertionError: 0 != 0.76

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)

Let’s fix it with DDT

First, we need to install it by running:

pip install ddt

Then, all we need to do is to decorate or class with ddt and move or testing tuple ((value, result),) above the test method under the data decorator - just like this:

import unittest
from ddt import ddt, data, file_data, unpack


@ddt
class ConvertionTestCase(unittest.TestCase):

    @data(
        (1, 0.76),
        (1.00, 0.76),
        (5, 3.78),
        (5.00, 3.78),
        (0, 0),
    )
    @unpack
    def test_aud_to_usd_ok(self, value, result):
        self.assertEquals(aud_to_usd(value), result)

nosetests -v convert.py 
test_aud_to_usd_ok_1__1__0_76_ (convert.ConvertionTestCase) ... FAIL
test_aud_to_usd_ok_2__1_0__0_76_ (convert.ConvertionTestCase) ... FAIL
test_aud_to_usd_ok_3__5__3_78_ (convert.ConvertionTestCase) ... FAIL
test_aud_to_usd_ok_4__5_0__3_78_ (convert.ConvertionTestCase) ... FAIL
test_aud_to_usd_ok_5__0__0_ (convert.ConvertionTestCase) ... ok

FAIL: test_aud_to_usd_ok_1__1__0_76_ (convert.ConvertionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "...python2.7/site-packages/ddt.py", line 144, in wrapper
    return func(self, *args, **kwargs)
  File ".../convert.py", line 22, in test_aud_to_usd_ok
    self.assertEquals(aud_to_usd(value), result)
AssertionError: 0 != 0.76

....

As you can see, the DDT library has generated tests for me such as test_aud_to_usd_ok_2__1_0__0_76_ and I know exactly which ones have failed or passed. I can now easily fix my code. With the following code, I can satisfy my requirements:

def aud_to_usd(value):
    return round(value * 0.75698, 2)

nosetests -v convert.py 
test_aud_to_usd_ok_1__1__0_76_ (convert.ConvertionTestCase) ... ok
test_aud_to_usd_ok_2__1_0__0_76_ (convert.ConvertionTestCase) ... ok
test_aud_to_usd_ok_3__5__3_78_ (convert.ConvertionTestCase) ... ok
test_aud_to_usd_ok_4__5_0__3_78_ (convert.ConvertionTestCase) ... ok
test_aud_to_usd_ok_5__0__0_ (convert.ConvertionTestCase) ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.001s

A concrete example built-in Django

Let’s pretend that you’re building a contact us form with an “enquiry type” dropdown. Depending what the message is about, it must go to a different service. Instead of writing a functional test per service, you can use DDT.

@ddt
class ContactUsTestCase(

    @override_settings(FROM_EMAIL="Eat Some Code <no-reply@esc.com>")
    @data(
        (MESSAGE_OTHER, "info@esc.com"),
        (MESSAGE_JOB, "jobs@esc.com")
    )
    @unpack
    def test_contact_ok__email_sent(self, selected_service, email_to):
        response = self.client.get(path="/contact-us/")

        form = response.forms['contact_form']
        form['name'] = "Robert"
        form['email'] = "robert@gmail.com"
        form['message'] = "Blablabla"
        form['enquiry_type'] = selected_service    # selected service as per above data
        response = form.submit().follow()

        self.assertContains(response, "Thank you")

        # check the email is well sent
        self.assertEquals(len(mail.outbox), 1)
        email = mail.outbox[0]
        self.assertEquals(email.from_email, 'ESC <no-reply@esc.com>')
        self.assertEquals(email.to, [email_to])    # email_to as per above data
        self.assertEquals(email.cc, [])
        self.assertIn(u"Robert", email.body)
        self.assertIn(u"robert@gmail.com", email.body)
        self.assertIn(u"Blablabla", email.body)

The test, test_contact_ok__email_sent will run twice: once with “Other” selected and once with “Job” selected. That way, you’re sure that the email is going to the right people in both cases. If the template is different per service, you’re also making sure that both templates are working.

DDT for Python - is an awesome library. I highly recommend it. If you know good alternatives, please let me know in the comments below.

Handling repetitive tests in Django

June 2018

DDT introduction

Issues with using a forloop

Let’s fix it with DDT

A concrete example built-in Django