=========================== Testing Django applications =========================== Automated testing is an extremely useful bug-killing tool for the modern Web developer. You can use a collection of tests -- a **test suite** -- to solve, or avoid, a number of problems: * When you're writing new code, you can use tests to validate your code works as expected. * When you're refactoring or modifying old code, you can use tests to ensure your changes haven't affected your application's behavior unexpectedly. Testing a Web application is a complex task, because a Web application is made of several layers of logic -- from HTTP-level request handling, to form validation and processing, to template rendering. With Django's test-execution framework and assorted utilities, you can simulate requests, insert test data, inspect your application's output and generally verify your code is doing what it should be doing. The best part is, it's really easy. This document is split into two primary sections. First, we explain how to write tests with Django. Then, we explain how to run them. .. admonition:: Note This testing framework is currently under development. It may change slightly before the next official Django release. (That's *no* excuse not to write tests, though!) Writing tests ============= There are two primary ways to write tests with Django, corresponding to the two test frameworks that ship in the Python standard library. The two frameworks are: * **Doctests** -- tests that are embedded in your functions' docstrings and are written in a way that emulates a session of the Python interactive interpreter. For example:: def my_func(a_list, idx): """ >>> a = ['larry', 'curly', 'moe'] >>> my_func(a, 0) 'larry' >>> my_func(a, 1) 'curly' """ return a_list[idx] * **Unit tests** -- tests that are expressed as methods on a Python class that subclasses ``unittest.TestCase``. For example:: import unittest class MyFuncTestCase(unittest.TestCase) def testBasic(self): a = ['larry', 'curly', 'moe'] self.assertEquals(my_func(a, 0), 'larry') self.assertEquals(my_func(a, 1), 'curly') You can choose the test framework you like, depending on which syntax you prefer, or you can mix and match, using one framework for some of your code and the other framework for other code. You can also use any *other* Python test frameworks, as we'll explain in a bit. Writing doctests ---------------- Doctests use Python's standard doctest_ module, which searches your docstrings for statements that resemble a session of the Python interactive interpreter. A full explanation of how doctest works is out of the scope of this document; read Python's official documentation for the details. .. admonition:: What's a **docstring**? A good explanation of docstrings (and some guidelines for using them effectively) can be found in :PEP:`257`: A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition. Such a docstring becomes the ``__doc__`` special attribute of that object. For example, this function has a docstring that describes what it does:: def add_two(num): "Adds 2 to the given number and returns the result." return num + 2 Because tests often make great documentation, putting tests directly in your docstrings is an effective way to document *and* test your code. For a given Django application, the test runner looks for doctests in two places: * The ``models.py`` file. You can define module-level doctests and/or a doctest for individual models. It's common practice to put application-level doctests in the module docstring and model-level doctests in the model docstrings. * A file called ``tests.py`` in the application directory -- i.e., the directory that holds ``models.py``. This file is a hook for any and all doctests you want to write that aren't necessarily related to models. Here is an example model doctest:: # models.py from django.db import models class Animal(models.Model): """ An animal that knows how to make noise # Create some animals >>> lion = Animal.objects.create(name="lion", sound="roar") >>> cat = Animal.objects.create(name="cat", sound="meow") # Make 'em speak >>> lion.speak() 'The lion says "roar"' >>> cat.speak() 'The cat says "meow"' """ name = models.CharField(max_length=20) sound = models.CharField(max_length=20) def speak(self): return 'The %s says "%s"' % (self.name, self.sound) When you `run your tests`_, the test runner will find this docstring, notice that portions of it look like an interactive Python session, and execute those lines while checking that the results match. In the case of model tests, note that the test runner takes care of creating its own test database. That is, any test that accesses a database -- by creating and saving model instances, for example -- will not affect your production database. Each doctest begins with a "blank slate" -- a fresh database containing an empty table for each model. (See the section on fixtures, below, for more on this.) Note that to use this feature, the database user Django is connecting as must have ``CREATE DATABASE`` rights. For more details about how doctest works, see the `standard library documentation for doctest`_ .. _doctest: http://docs.python.org/lib/module-doctest.html .. _standard library documentation for doctest: doctest_ Writing unit tests ------------------ Like doctests, Django's unit tests use a standard library module: unittest_. This module uses a different way of defining tests, taking a class-based approach. As with doctests, for a given Django application, the test runner looks for unit tests in two places: * The ``models.py`` file. The test runner looks for any subclass of ``unittest.TestCase`` in this module. * A file called ``tests.py`` in the application directory -- i.e., the directory that holds ``models.py``. Again, the test runner looks for any subclass of ``unittest.TestCase`` in this module. This example ``unittest.TestCase`` subclass is equivalent to the example given in the doctest section above:: import unittest from myapp.models import Animal class AnimalTestCase(unittest.TestCase): def setUp(self): self.lion = Animal.objects.create(name="lion", sound="roar") self.cat = Animal.objects.create(name="cat", sound="meow") def testSpeaking(self): self.assertEquals(self.lion.speak(), 'The lion says "roar"') self.assertEquals(self.cat.speak(), 'The cat says "meow"') When you `run your tests`_, the default behavior of the test utility is to find all the test cases (that is, subclasses of ``unittest.TestCase``) in ``models.py`` and ``tests.py``, automatically build a test suite out of those test cases, and run that suite. In the Django development version, there is a second way to define the test suite for a module: if you define a function called ``suite()`` in either ``models.py`` or ``tests.py``, the Django test runner will use that function to construct the test suite for that module. This follows the `suggested organization`_ for unit tests. See the Python documentation for more details on how to construct a complex test suite. For more details about ``unittest``, see the `standard library unittest documentation`_. .. _unittest: http://docs.python.org/lib/module-unittest.html .. _standard library unittest documentation: unittest_ .. _run your tests: `Running tests`_ .. _suggested organization: http://docs.python.org/lib/organizing-tests.html Which should I use? ------------------- Because Django supports both of the standard Python test frameworks, it's up to you and your tastes to decide which one to use. You can even decide to use *both*. For developers new to testing, however, this choice can seem confusing. Here, then, are a few key differences to help you decide which approach is right for you: * If you've been using Python for a while, ``doctest`` will probably feel more "pythonic". It's designed to make writing tests as easy as possible, so it requires no overhead of writing classes or methods. You simply put tests in docstrings. This has the added advantage of serving as documentation (and correct documentation, at that!). If you're just getting started with testing, using doctests will probably get you started faster. * The ``unittest`` framework will probably feel very familiar to developers coming from Java. ``unittest`` is inspired by Java's JUnit, so you'll feel at home with this method if you've used JUnit or any test framework inspired by JUnit. * If you need to write a bunch of tests that share similar code, then you'll appreciate the ``unittest`` framework's organization around classes and methods. This makes it easy to abstract common tasks into common methods. The framework also supports explicit setup and/or cleanup routines, which give you a high level of control over the environment in which your test cases are run. Again, remember that you can use both systems side-by-side (even in the same app). In the end, most projects will eventually end up using both. Each shines in different circumstances. Running tests ============= Once you've written tests, run them using your project's ``manage.py`` utility:: $ ./manage.py test By default, this will run every test in every application in ``INSTALLED_APPS``. If you only want to run tests for a particular application, add the application name to the command line. For example, if your ``INSTALLED_APPS`` contains ``'myproject.polls'`` and ``'myproject.animals'``, you can run the ``myproject.animals`` unit tests alone with this command:: # ./manage.py test animals Note that we used ``animals``, not ``myproject.animals``. **New in Django development version:** If you use unit tests, as opposed to doctests, you can be even *more* specific in choosing which tests to execute. To run a single test case in an application (for example, the ``AnimalTestCase`` described in the "Writing unit tests" section), add the name of the test case to the label on the command line:: $ ./manage.py test animals.AnimalTestCase And it gets even more granular than that! To run a *single* test method inside a test case, add the name of the test method to the label:: $ ./manage.py test animals.AnimalTestCase.testFluffyAnimals The test database ----------------- Tests that require a database (namely, model tests) will not use your "real" (production) database. A separate, blank database is created for the tests. Regardless of whether the tests pass or fail, the test database is destroyed when all the tests have been executed. By default this test database gets its name by prepending ``test_`` to the value of the ``DATABASE_NAME`` setting. When using the SQLite database engine the tests will by default use an in-memory database (i.e., the database will be created in memory, bypassing the filesystem entirely!). If you want to use a different database name, specify the ``TEST_DATABASE_NAME`` setting. Aside from using a separate database, the test runner will otherwise use all of the same database settings you have in your settings file: ``DATABASE_ENGINE``, ``DATABASE_USER``, ``DATABASE_HOST``, etc. The test database is created by the user specified by ``DATABASE_USER``, so you'll need to make sure that the given user account has sufficient privileges to create a new database on the system. **New in Django development version:** For fine-grained control over the character encoding of your test database, use the ``TEST_DATABASE_CHARSET`` setting. If you're using MySQL, you can also use the ``TEST_DATABASE_COLLATION`` setting to control the particular collation used by the test database. See the settings_ documentation for details of these advanced settings. .. _settings: ../settings/ Understanding the test output ----------------------------- When you run your tests, you'll see a number of messages as the test runner prepares itself. You can control the level of detail of these messages with the ``verbosity`` option on the command line:: Creating test database... Creating table myapp_animal Creating table myapp_mineral Loading 'initial_data' fixtures... No fixtures found. This tells you that the test runner is creating a test database, as described in the previous section. Once the test database has been created, Django will run your tests. If everything goes well, you'll see something like this:: ---------------------------------------------------------------------- Ran 22 tests in 0.221s OK If there are test failures, however, you'll see full details about which tests failed:: ====================================================================== FAIL: Doctest: ellington.core.throttle.models ---------------------------------------------------------------------- Traceback (most recent call last): File "/dev/django/test/doctest.py", line 2153, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for myapp.models File "/dev/myapp/models.py", line 0, in models ---------------------------------------------------------------------- File "/dev/myapp/models.py", line 14, in myapp.models Failed example: throttle.check("actor A", "action one", limit=2, hours=1) Expected: True Got: False ---------------------------------------------------------------------- Ran 2 tests in 0.048s FAILED (failures=1) A full explanation of this error output is beyond the scope of this document, but it's pretty intuitive. You can consult the documentation of Python's ``unittest`` library for details. Note that the return code for the test-runner script is the total number of failed and erroneous tests. If all the tests pass, the return code is 0. This feature is useful if you're using the test-runner script in a shell script and need to test for success or failure at that level. Testing tools ============= Django provides a small set of tools that come in handy when writing tests. The test client --------------- The test client is a Python class that acts as a dummy Web browser, allowing you to test your views and interact with your Django-powered application programatically. Some of the things you can do with the test client are: * Simulate GET and POST requests on a URL and observe the response -- everything from low-level HTTP (result headers and status codes) to page content. * Test that the correct view is executed for a given URL. * Test that a given request is rendered by a given Django template, with a template context that contains certain values. Note that the test client is not intended to be a replacement for Twill_, Selenium_, or other "in-browser" frameworks. Django's test client has a different focus. In short: * Use Django's test client to establish that the correct view is being called and that the view is collecting the correct context data. * Use in-browser frameworks such as Twill and Selenium to test *rendered* HTML and the *behavior* of Web pages, namely JavaScript functionality. A comprehensive test suite should use a combination of both test types. .. _Twill: http://twill.idyll.org/ .. _Selenium: http://www.openqa.org/selenium/ Overview and a quick example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use the test client, instantiate ``django.test.client.Client`` and retrieve Web pages:: >>> from django.test.client import Client >>> c = Client() >>> response = c.post('/login/', {'username': 'john', 'password': 'smith'}) >>> response.status_code 200 >>> response = c.get('/customer/details/') >>> response.content '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 ...' As this example suggests, you can instantiate ``Client`` from within a session of the Python interactive interpreter. Note a few important things about how the test client works: * The test client does *not* require the Web server to be running. In fact, it will run just fine with no Web server running at all! That's because it avoids the overhead of HTTP and deals directly with the Django framework. This helps make the unit tests run quickly. * When retrieving pages, remember to specify the *path* of the URL, not the whole domain. For example, this is correct:: >>> c.get('/login/') This is incorrect:: >>> c.get('http://www.example.com/login/') The test client is not capable of retrieving Web pages that are not powered by your Django project. If you need to retrieve other Web pages, use a Python standard library module such as urllib_ or urllib2_. * To resolve URLs, the test client uses whatever URLconf is pointed-to by your ``ROOT_URLCONF`` setting. * Although the above example would work in the Python interactive interpreter, some of the test client's functionality, notably the template-related functionality, is only available *while tests are running*. The reason for this is that Django's test runner performs a bit of black magic in order to determine which template was loaded by a given view. This black magic (essentially a patching of Django's template system in memory) only happens during test running. .. _urllib: http://docs.python.org/lib/module-urllib.html .. _urllib2: http://docs.python.org/lib/module-urllib2.html Making requests ~~~~~~~~~~~~~~~ Use the ``django.test.client.Client`` class to make requests. It requires no arguments at time of construction:: >>> c = Client() Once you have a ``Client`` instance, you can call any of the following methods: ``get(path, data={})`` Makes a GET request on the provided ``path`` and returns a ``Response`` object, which is documented below. The key-value pairs in the ``data`` dictionary are used to create a GET data payload. For example:: >>> c = Client() >>> c.get('/customers/details/', {'name': 'fred', 'age': 7}) ...will result in the evaluation of a GET request equivalent to:: /customers/details/?name=fred&age=7 ``post(path, data={}, content_type=MULTIPART_CONTENT)`` Makes a POST request on the provided ``path`` and returns a ``Response`` object, which is documented below. The key-value pairs in the ``data`` dictionary are used to submit POST data. For example:: >>> c = Client() >>> c.post('/login/', {'name': 'fred', 'passwd': 'secret'}) ...will result in the evaluation of a POST request to this URL:: /login/ ...with this POST data:: name=fred&passwd=secret If you provide ``content_type`` (e.g., ``text/xml`` for an XML payload), the contents of ``data`` will be sent as-is in the POST request, using ``content_type`` in the HTTP ``Content-Type`` header. If you don't provide a value for ``content_type``, the values in ``data`` will be transmitted with a content type of ``multipart/form-data``. In this case, the key-value pairs in ``data`` will be encoded as a multipart message and used to create the POST data payload. To submit multiple values for a given key -- for example, to specify the selections for a ``<select multiple>`` -- provide the values as a list or tuple for the required key. For example, this value of ``data`` would submit three selected values for the field named ``choices``:: {'choices': ('a', 'b', 'd')} Submitting files is a special case. To POST a file, you need only provide the file field name as a key, and a file handle to the file you wish to upload as a value. For example:: >>> c = Client() >>> f = open('wishlist.doc') >>> c.post('/customers/wishes/', {'name': 'fred', 'attachment': f}) >>> f.close() (The name ``attachment`` here is not relevant; use whatever name your file-processing code expects.) Note that you should manually close the file after it has been provided to ``post()``. ``login(**credentials)`` **New in Django development version** If your site uses Django's `authentication system`_ and you deal with logging in users, you can use the test client's ``login()`` method to simulate the effect of a user logging into the site. After you call this method, the test client will have all the cookies and session data required to pass any login-based tests that may form part of a view. The format of the ``credentials`` argument depends on which `authentication backend`_ you're using (which is configured by your ``AUTHENTICATION_BACKENDS`` setting). If you're using the standard authentication backend provided by Django (``ModelBackend``), ``credentials`` should be the user's username and password, provided as keyword arguments:: >>> c = Client() >>> c.login(username='fred', password='secret') >>> # Now you can access a view that's only available to logged-in users. If you're using a different authentication backend, this method may require different credentials. It requires whichever credentials are required by your backend's ``authenticate()`` method. ``login()`` returns ``True`` if it the credentials were accepted and login was successful. Finally, you'll need to remember to create user accounts before you can use this method. As we explained above, the test runner is executed using a test database, which contains no users by default. As a result, user accounts that are valid on your production site will not work under test conditions. You'll need to create users as part of the test suite -- either manually (using the Django model API) or with a test fixture. ``logout()`` **New in Django development version** If your site uses Django's `authentication system`_, the ``logout()`` method can be used to simulate the effect of a user logging out of your site. After you call this method, the test client will have all the cookies and session data cleared to defaults. Subsequent requests will appear to come from an AnonymousUser. .. _authentication system: ../authentication/ .. _authentication backend: ../authentication/#other-authentication-sources Testing responses ~~~~~~~~~~~~~~~~~ The ``get()`` and ``post()`` methods both return a ``Response`` object. This ``Response`` object is *not* the same as the ``HttpResponse`` object returned Django views; the test response object has some additional data useful for test code to verify. Specifically, a ``Response`` object has the following attributes: =============== ========================================================== Attribute Description =============== ========================================================== ``client`` The test client that was used to make the request that resulted in the response. ``content`` The body of the response, as a string. This is the final page content as rendered by the view, or any error message. ``context`` The template ``Context`` instance that was used to render the template that produced the response content. If the rendered page used multiple templates, then ``context`` will be a list of ``Context`` objects, in the order in which they were rendered. ``headers`` The HTTP headers of the response. This is a dictionary. ``request`` The request data that stimulated the response. ``status_code`` The HTTP status of the response, as an integer. See RFC2616_ for a full list of HTTP status codes. ``template`` The ``Template`` instance that was used to render the final content. Use ``template.name`` to get the template's file name, if the template was loaded from a file. (The name is a string such as ``'admin/index.html'``.) If the rendered page used multiple templates -- e.g., using `template inheritance`_ -- then ``template`` will be a list of ``Template`` instances, in the order in which they were rendered. =============== ========================================================== .. _RFC2616: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html .. _template inheritance: ../templates/#template-inheritance Exceptions ~~~~~~~~~~ If you point the test client at a view that raises an exception, that exception will be visible in the test case. You can then use a standard ``try...catch`` block or ``unittest.TestCase.assertRaises()`` to test for exceptions. The only exceptions that are not visible to the test client are ``Http404``, ``PermissionDenied`` and ``SystemExit``. Django catches these exceptions internally and converts them into the appropriate HTTP response codes. In these cases, you can check ``response.status_code`` in your test. Persistent state ~~~~~~~~~~~~~~~~ The test client is stateful. If a response returns a cookie, then that cookie will be stored in the test client and sent with all subsequent ``get()`` and ``post()`` requests. Expiration policies for these cookies are not followed. If you want a cookie to expire, either delete it manually or create a new ``Client`` instance (which will effectively delete all cookies). A test client has two attributes that store persistent state information. You can access these properties as part of a test condition. =============== ========================================================== Attribute Description =============== ========================================================== ``cookies`` A Python ``SimpleCookie`` object, containing the current values of all the client cookies. See the `Cookie module documentation`_ for more. ``session`` A dictionary-like object containing session information. See the `session documentation`_ for full details. =============== ========================================================== .. _Cookie module documentation: http://docs.python.org/lib/module-Cookie.html .. _session documentation: ../sessions/ Example ~~~~~~~ The following is a simple unit test using the test client:: import unittest from django.test.client import Client class SimpleTest(unittest.TestCase): def setUp(self): # Every test needs a client. self.client = Client() def test_details(self): # Issue a GET request. response = self.client.get('/customer/details/') # Check that the respose is 200 OK. self.failUnlessEqual(response.status_code, 200) # Check that the rendered context contains 5 customers. self.failUnlessEqual(len(response.context['customers']), 5) TestCase -------- Normal Python unit test classes extend a base class of ``unittest.TestCase``. Django provides an extension of this base class -- ``django.test.TestCase`` -- that provides some additional capabilities that can be useful for testing Web sites. Converting a normal ``unittest.TestCase`` to a Django ``TestCase`` is easy: just change the base class of your test from ``unittest.TestCase`` to ``django.test.TestCase``. All of the standard Python unit test functionality will continue to be available, but it will be augmented with some useful additions. Default test client ~~~~~~~~~~~~~~~~~~~ **New in Django development version** Every test case in a ``django.test.TestCase`` instance has access to an instance of a Django test client. This client can be accessed as ``self.client``. This client is recreated for each test, so you don't have to worry about state (such as cookies) carrying over from one test to another. This means, instead of instantiating a ``Client`` in each test:: import unittest from django.test.client import Client class SimpleTest(unittest.TestCase): def test_details(self): client = Client() response = client.get('/customer/details/') self.failUnlessEqual(response.status_code, 200) def test_index(self): client = Client() response = client.get('/customer/index/') self.failUnlessEqual(response.status_code, 200) ...you can just refer to ``self.client``, like so:: from django.test import TestCase class SimpleTest(TestCase): def test_details(self): response = self.client.get('/customer/details/') self.failUnlessEqual(response.status_code, 200) def test_index(self): response = self.client.get('/customer/index/') self.failUnlessEqual(response.status_code, 200) Fixture loading ~~~~~~~~~~~~~~~ A test case for a database-backed Web site isn't much use if there isn't any data in the database. To make it easy to put test data into the database, Django's custom ``TestCase`` class provides a way of loading **fixtures**. A fixture is a collection of data that Django knows how to import into a database. For example, if your site has user accounts, you might set up a fixture of fake user accounts in order to populate your database during tests. The most straightforward way of creating a fixture is to use the ``manage.py dumpdata`` command. This assumes you already have some data in your database. See the `dumpdata documentation`_ for more details. .. note:: If you've ever run ``manage.py syncdb``, you've already used a fixture without even knowing it! When you call ``syncdb`` in the database for the first time, Django installs a fixture called ``initial_data``. This gives you a way of populating a new database with any initial data, such as a default set of categories. Fixtures with other names can always be installed manually using the ``manage.py loaddata`` command. Once you've created a fixture and placed it somewhere in your Django project, you can use it in your unit tests by specifying a ``fixtures`` class attribute on your ``django.test.TestCase`` subclass:: from django.test import TestCase from myapp.models import Animal class AnimalTestCase(TestCase): fixtures = ['mammals.json', 'birds'] def setUp(self): # Test definitions as before. def testFluffyAnimals(self): # A test that uses the fixtures. Here's specifically what will happen: * At the start of each test case, before ``setUp()`` is run, Django will flush the database, returning the database to the state it was in directly after ``syncdb`` was called. * Then, all the named fixtures are installed. In this example, Django will install any JSON fixture named ``mammals``, followed by any fixture named ``birds``. See the `loaddata documentation`_ for more details on defining and installing fixtures. This flush/load procedure is repeated for each test in the test case, so you can be certain that the outcome of a test will not be affected by another test, or by the order of test execution. .. _dumpdata documentation: ../django-admin/#dumpdata-appname-appname .. _loaddata documentation: ../django-admin/#loaddata-fixture-fixture Emptying the test outbox ~~~~~~~~~~~~~~~~~~~~~~~~ **New in Django development version** If you use Django's custom ``TestCase`` class, the test runner will clear the contents of the test e-mail outbox at the start of each test case. For more detail on e-mail services during tests, see `E-mail services`_. Assertions ~~~~~~~~~~ **New in Django development version** As Python's normal ``unittest.TestCase`` class implements assertion methods such as ``assertTrue`` and ``assertEquals``, Django's custom ``TestCase`` class provides a number of custom assertion methods that are useful for testing Web applications: ``assertContains(response, text, count=None, status_code=200)`` Asserts that a ``Response`` instance produced the given ``status_code`` and that ``text`` appears in the content of the response. If ``count`` is provided, ``text`` must occur exactly ``count`` times in the response. ``assertFormError(response, form, field, errors)`` Asserts that a field on a form raises the provided list of errors when rendered on the form. ``form`` is the name the ``Form`` instance was given in the template context. Note that this works only for ``newforms.Form`` instances, not ``oldforms.Form`` instances. ``field`` is the name of the field on the form to check. If ``field`` has a value of ``None``, non-field errors (errors you can access via ``form.non_field_errors()``) will be checked. ``errors`` is an error string, or a list of error strings, that are expected as a result of form validation. ``assertTemplateNotUsed(response, template_name)`` Asserts that the template with the given name was *not* used in rendering the response. ``assertRedirects(response, expected_url, status_code=302, target_status_code=200)`` Asserts that the response return a ``status_code`` redirect status, it redirected to ``expected_url`` (including any GET data), and the subsequent page was received with ``target_status_code``. ``assertTemplateUsed(response, template_name)`` Asserts that the template with the given name was used in rendering the response. The name is a string such as ``'admin/index.html'``. E-mail services --------------- **New in Django development version** If any of your Django views send e-mail using `Django's e-mail functionality`_, you probably don't want to send e-mail each time you run a test using that view. For this reason, Django's test runner automatically redirects all Django-sent e-mail to a dummy outbox. This lets you test every aspect of sending e-mail -- from the number of messages sent to the contents of each message -- without actually sending the messages. The test runner accomplishes this by transparently replacing the normal `SMTPConnection`_ class with a different version. (Don't worry -- this has no effect on any other e-mail senders outside of Django, such as your machine's mail server, if you're running one.) During test running, each outgoing e-mail is saved in ``django.core.mail.outbox``. This is a simple list of all `EmailMessage`_ instances that have been sent. It does not exist under normal execution conditions, i.e., when you're not running unit tests. The outbox is created during test setup, along with the dummy `SMTPConnection`_. When the test framework is torn down, the standard `SMTPConnection`_ class is restored, and the test outbox is destroyed. Here's an example test that examines ``django.core.mail.outbox`` for length and contents:: from django.core import mail from django.test import TestCase class EmailTest(TestCase): def test_send_email(self): # Send message. mail.send_mail('Subject here', 'Here is the message.', 'from@example.com', ['to@example.com'], fail_silently=False) # Test that one message has been sent. self.assertEqual(len(mail.outbox), 1) # Verify that the subject of the first message is correct. self.assertEqual(mail.outbox[0].subject, 'Subject here') As noted `previously`_, the test outbox is emptied at the start of every test in a Django ``TestCase``. To empty the outbox manually, assign the empty list to ``mail.outbox``:: from django.core import mail # Empty the test outbox mail.outbox = [] .. _`Django's e-mail functionality`: ../email/ .. _`SMTPConnection`: ../email/#the-emailmessage-and-smtpconnection-classes .. _`EmailMessage`: ../email/#the-emailmessage-and-smtpconnection-classes .. _`previously`: #emptying-the-test-outbox Using different testing frameworks ================================== Clearly, ``doctest`` and ``unittest`` are not the only Python testing frameworks. While Django doesn't provide explicit support for alternative frameworks, it does provide a way to invoke tests constructed for an alternative framework as if they were normal Django tests. When you run ``./manage.py test``, Django looks at the ``TEST_RUNNER`` setting to determine what to do. By default, ``TEST_RUNNER`` points to ``'django.test.simple.run_tests'``. This method defines the default Django testing behavior. This behavior involves: #. Performing global pre-test setup. #. Creating the test database. #. Running ``syncdb`` to install models and initial data into the test database. #. Looking for unit tests and doctests in the ``models.py`` and ``tests.py`` files in each installed application. #. Running the unit tests and doctests that are found. #. Destroying the test database. #. Performing global post-test teardown. If you define your own test runner method and point ``TEST_RUNNER`` at that method, Django will execute your test runner whenever you run ``./manage.py test``. In this way, it is possible to use any test framework that can be executed from Python code. Defining a test runner ---------------------- **New in Django development version** By convention, a test runner should be called ``run_tests``. The only strict requirement is that it has the same arguments as the Django test runner: ``run_tests(test_labels, verbosity=1, interactive=True, extra_tests=[])`` ``test_labels`` is a list of strings describing the tests to be run. A test label can take one of three forms: * ``app.TestCase.test_method`` -- Run a single test method in a test case. * ``app.TestCase`` -- Run all the test methods in a test case. * ``app`` -- Search for and run all tests in the named application. If ``test_labels`` has a value of ``None``, the test runner should run search for tests in all the applications in ``INSTALLED_APPS``. ``verbosity`` determines the amount of notification and debug information that will be printed to the console; ``0`` is no output, ``1`` is normal output, and ``2`` is verbose output. If ``interactive`` is ``True``, the test suite has permission to ask the user for instructions when the test suite is executed. An example of this behavior would be asking for permission to delete an existing test database. If ``interactive`` is ``False``, the test suite must be able to run without any manual intervention. ``extra_tests`` is a list of extra ``TestCase`` instances to add to the suite that is executed by the test runner. These extra tests are run in addition to those discovered in the modules listed in ``module_list``. This method should return the number of tests that failed. Testing utilities ----------------- To assist in the creation of your own test runner, Django provides a number of utility methods in the ``django.test.utils`` module. ``setup_test_environment()`` Performs any global pre-test setup, such as the installing the instrumentation of the template rendering system and setting up the dummy ``SMTPConnection``. ``teardown_test_environment()`` Performs any global post-test teardown, such as removing the black magic hooks into the template system and restoring normal e-mail services. ``create_test_db(verbosity=1, autoclobber=False)`` Creates a new test database and runs ``syncdb`` against it. ``verbosity`` has the same behavior as in ``run_tests()``. ``autoclobber`` describes the behavior that will occur if a database with the same name as the test database is discovered: * If ``autoclobber`` is ``False``, the user will be asked to approve destroying the existing database. ``sys.exit`` is called if the user does not approve. * If autoclobber is ``True``, the database will be destroyed without consulting the user. ``create_test_db()`` has the side effect of modifying ``settings.DATABASE_NAME`` to match the name of the test database. New in the Django development version, this function returns the name of the test database that it created. ``destroy_test_db(old_database_name, verbosity=1)`` Destroys the database whose name is in the ``DATABASE_NAME`` setting and restores the value of ``DATABASE_NAME`` to the provided name. ``verbosity`` has the same behavior as in ``run_tests()``.