===================== Database transactions ===================== .. module:: django.db.transaction Django gives you a few ways to control how database transactions are managed. Managing database transactions ============================== Django's default transaction behavior ------------------------------------- Django's default behavior is to run in autocommit mode. Each query is immediately committed to the database. :ref:`See below for details `. .. Django uses transactions or savepoints automatically to guarantee the integrity of ORM operations that require multiple queries, especially :ref:`delete() ` and :ref:`update() ` queries. .. versionchanged:: 1.6 Previous version of Django featured :ref:`a more complicated default behavior `. Tying transactions to HTTP requests ----------------------------------- The recommended way to handle transactions in Web requests is to tie them to the request and response phases via Django's ``TransactionMiddleware``. It works like this. When a request starts, Django starts a transaction. If the response is produced without problems, Django commits any pending transactions. If the view function produces an exception, Django rolls back any pending transactions. To activate this feature, just add the ``TransactionMiddleware`` middleware to your :setting:`MIDDLEWARE_CLASSES` setting:: MIDDLEWARE_CLASSES = ( 'django.middleware.cache.UpdateCacheMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.transaction.TransactionMiddleware', 'django.middleware.cache.FetchFromCacheMiddleware', ) The order is quite important. The transaction middleware applies not only to view functions, but also for all middleware modules that come after it. So if you use the session middleware after the transaction middleware, session creation will be part of the transaction. The various cache middlewares are an exception: ``CacheMiddleware``, :class:`~django.middleware.cache.UpdateCacheMiddleware`, and :class:`~django.middleware.cache.FetchFromCacheMiddleware` are never affected. Even when using database caching, Django's cache backend uses its own database connection internally. .. note:: The ``TransactionMiddleware`` only affects the database aliased as "default" within your :setting:`DATABASES` setting. If you are using multiple databases and want transaction control over databases other than "default", you will need to write your own transaction middleware. Controlling transactions explicitly ----------------------------------- .. versionadded:: 1.6 Django provides a single API to control database transactions. .. function:: atomic(using=None) This function creates an atomic block for writes to the database. (Atomicity is the defining property of database transactions.) When the block completes successfully, the changes are committed to the database. When it raises an exception, the changes are rolled back. ``atomic`` can be nested. In this case, when an inner block completes successfully, its effects can still be rolled back if an exception is raised in the outer block at a later point. ``atomic`` takes a ``using`` argument which should be the name of a database. If this argument isn't provided, Django uses the ``"default"`` database. ``atomic`` is usable both as a decorator:: from django.db import transaction @transaction.atomic def viewfunc(request): # This code executes inside a transaction. do_stuff() and as a context manager:: from django.db import transaction def viewfunc(request): # This code executes in autocommit mode (Django's default). do_stuff() with transaction.atomic(): # This code executes inside a transaction. do_more_stuff() Wrapping ``atomic`` in a try/except block allows for natural handling of integrity errors:: from django.db import IntegrityError, transaction @transaction.atomic def viewfunc(request): do_stuff() try: with transaction.atomic(): do_stuff_that_could_fail() except IntegrityError: handle_exception() do_more_stuff() In this example, even if ``do_stuff_that_could_fail()`` causes a database error by breaking an integrity constraint, you can execute queries in ``do_more_stuff()``, and the changes from ``do_stuff()`` are still there. In order to guarantee atomicity, ``atomic`` disables some APIs. Attempting to commit, roll back, or change the autocommit state of the database connection within an ``atomic`` block will raise an exception. ``atomic`` can only be used in autocommit mode. It will raise an exception if autocommit is turned off. Under the hood, Django's transaction management code: - opens a transaction when entering the outermost ``atomic`` block; - creates a savepoint when entering an inner ``atomic`` block; - releases or rolls back to the savepoint when exiting an inner block; - commits or rolls back the transaction when exiting the outermost block. .. _transaction-management-functions: Controlling transaction management in views =========================================== For most people, implicit request-based transactions work wonderfully. However, if you need more fine-grained control over how transactions are managed, you can use a set of functions in ``django.db.transaction`` to control transactions on a per-function or per-code-block basis. These functions, described in detail below, can be used in two different ways: * As a decorator_ on a particular function. For example:: from django.db import transaction @transaction.commit_on_success def viewfunc(request): # ... # this code executes inside a transaction # ... * As a `context manager`_ around a particular block of code:: from django.db import transaction def viewfunc(request): # ... # this code executes using default transaction management # ... with transaction.commit_on_success(): # ... # this code executes inside a transaction # ... Both techniques work with all supported version of Python. .. _decorator: http://docs.python.org/glossary.html#term-decorator .. _context manager: http://docs.python.org/glossary.html#term-context-manager For maximum compatibility, all of the examples below show transactions using the decorator syntax, but all of the follow functions may be used as context managers, too. .. note:: Although the examples below use view functions as examples, these decorators and context managers can be used anywhere in your code that you need to deal with transactions. .. _topics-db-transactions-autocommit: .. function:: autocommit Use the ``autocommit`` decorator to switch a view function to Django's default commit behavior. Example:: from django.db import transaction @transaction.autocommit def viewfunc(request): .... @transaction.autocommit(using="my_other_database") def viewfunc2(request): .... Within ``viewfunc()``, transactions will be committed as soon as you call ``model.save()``, ``model.delete()``, or any other function that writes to the database. ``viewfunc2()`` will have this same behavior, but for the ``"my_other_database"`` connection. .. function:: commit_on_success Use the ``commit_on_success`` decorator to use a single transaction for all the work done in a function:: from django.db import transaction @transaction.commit_on_success def viewfunc(request): .... @transaction.commit_on_success(using="my_other_database") def viewfunc2(request): .... If the function returns successfully, then Django will commit all work done within the function at that point. If the function raises an exception, though, Django will roll back the transaction. .. function:: commit_manually Use the ``commit_manually`` decorator if you need full control over transactions. It tells Django you'll be managing the transaction on your own. Whether you are writing or simply reading from the database, you must ``commit()`` or ``rollback()`` explicitly or Django will raise a :exc:`TransactionManagementError` exception. This is required when reading from the database because ``SELECT`` statements may call functions which modify tables, and thus it is impossible to know if any data has been modified. Manual transaction management looks like this:: from django.db import transaction @transaction.commit_manually def viewfunc(request): ... # You can commit/rollback however and whenever you want transaction.commit() ... # But you've got to remember to do it yourself! try: ... except: transaction.rollback() else: transaction.commit() @transaction.commit_manually(using="my_other_database") def viewfunc2(request): .... .. _topics-db-transactions-requirements: Requirements for transaction handling ===================================== Django requires that every transaction that is opened is closed before the completion of a request. If you are using :func:`autocommit` (the default commit mode) or :func:`commit_on_success`, this will be done for you automatically. However, if you are manually managing transactions (using the :func:`commit_manually` decorator), you must ensure that the transaction is either committed or rolled back before a request is completed. This applies to all database operations, not just write operations. Even if your transaction only reads from the database, the transaction must be committed or rolled back before you complete a request. .. _managing-autocommit: Managing autocommit =================== .. versionadded:: 1.6 Django provides a straightforward API to manage the autocommit state of each database connection, if you need to. .. function:: get_autocommit(using=None) .. function:: set_autocommit(using=None, autocommit=True) These functions take a ``using`` argument which should be the name of a database. If it isn't provided, Django uses the ``"default"`` database. .. _deactivate-transaction-management: How to globally deactivate transaction management ================================================= Control freaks can totally disable all transaction management by setting :setting:`TRANSACTIONS_MANAGED` to ``True`` in the Django settings file. If you do this, Django won't enable autocommit. You'll get the regular behavior of the underlying database library. This requires you to commit explicitly every transaction, even those started by Django or by third-party libraries. Thus, this is best used in situations where you want to run your own transaction-controlling middleware or do something really strange. In almost all situations, you'll be better off using the default behavior, or the transaction middleware, and only modify selected functions as needed. .. _topics-db-transactions-savepoints: Savepoints ========== A savepoint is a marker within a transaction that enables you to roll back part of a transaction, rather than the full transaction. Savepoints are available with the SQLite (≥ 3.6.8), PostgreSQL, Oracle and MySQL (when using the InnoDB storage engine) backends. Other backends provide the savepoint functions, but they're empty operations -- they don't actually do anything. Savepoints aren't especially useful if you are using the default ``autocommit`` behavior of Django. However, if you are using ``commit_on_success`` or ``commit_manually``, each open transaction will build up a series of database operations, awaiting a commit or rollback. If you issue a rollback, the entire transaction is rolled back. Savepoints provide the ability to perform a fine-grained rollback, rather than the full rollback that would be performed by ``transaction.rollback()``. Each of these functions takes a ``using`` argument which should be the name of a database for which the behavior applies. If no ``using`` argument is provided then the ``"default"`` database is used. Savepoints are controlled by three methods on the transaction object: .. method:: transaction.savepoint(using=None) Creates a new savepoint. This marks a point in the transaction that is known to be in a "good" state. Returns the savepoint ID (sid). .. method:: transaction.savepoint_commit(sid, using=None) Updates the savepoint to include any operations that have been performed since the savepoint was created, or since the last commit. .. method:: transaction.savepoint_rollback(sid, using=None) Rolls the transaction back to the last point at which the savepoint was committed. The following example demonstrates the use of savepoints:: from django.db import transaction @transaction.commit_manually def viewfunc(request): a.save() # open transaction now contains a.save() sid = transaction.savepoint() b.save() # open transaction now contains a.save() and b.save() if want_to_keep_b: transaction.savepoint_commit(sid) # open transaction still contains a.save() and b.save() else: transaction.savepoint_rollback(sid) # open transaction now contains only a.save() transaction.commit() Database-specific notes ======================= Savepoints in SQLite -------------------- While SQLite ≥ 3.6.8 supports savepoints, a flaw in the design of the :mod:`sqlite3` makes them hardly usable. When autocommit is enabled, savepoints don't make sense. When it's disabled, :mod:`sqlite3` commits implicitly before savepoint-related statement. (It commits before any statement other than ``SELECT``, ``INSERT``, ``UPDATE``, ``DELETE`` and ``REPLACE``.) As a consequence, savepoints are only usable inside a transaction ie. inside an :func:`atomic` block. Transactions in MySQL --------------------- If you're using MySQL, your tables may or may not support transactions; it depends on your MySQL version and the table types you're using. (By "table types," we mean something like "InnoDB" or "MyISAM".) MySQL transaction peculiarities are outside the scope of this article, but the MySQL site has `information on MySQL transactions`_. If your MySQL setup does *not* support transactions, then Django will function in autocommit mode: Statements will be executed and committed as soon as they're called. If your MySQL setup *does* support transactions, Django will handle transactions as explained in this document. .. _information on MySQL transactions: http://dev.mysql.com/doc/refman/5.0/en/sql-syntax-transactions.html Handling exceptions within PostgreSQL transactions -------------------------------------------------- When a call to a PostgreSQL cursor raises an exception (typically ``IntegrityError``), all subsequent SQL in the same transaction will fail with the error "current transaction is aborted, queries ignored until end of transaction block". Whilst simple use of ``save()`` is unlikely to raise an exception in PostgreSQL, there are more advanced usage patterns which might, such as saving objects with unique fields, saving using the force_insert/force_update flag, or invoking custom SQL. There are several ways to recover from this sort of error. Transaction rollback ~~~~~~~~~~~~~~~~~~~~ The first option is to roll back the entire transaction. For example:: a.save() # Succeeds, but may be undone by transaction rollback try: b.save() # Could throw exception except IntegrityError: transaction.rollback() c.save() # Succeeds, but a.save() may have been undone Calling ``transaction.rollback()`` rolls back the entire transaction. Any uncommitted database operations will be lost. In this example, the changes made by ``a.save()`` would be lost, even though that operation raised no error itself. Savepoint rollback ~~~~~~~~~~~~~~~~~~ You can use :ref:`savepoints ` to control the extent of a rollback. Before performing a database operation that could fail, you can set or update the savepoint; that way, if the operation fails, you can roll back the single offending operation, rather than the entire transaction. For example:: a.save() # Succeeds, and never undone by savepoint rollback try: sid = transaction.savepoint() b.save() # Could throw exception transaction.savepoint_commit(sid) except IntegrityError: transaction.savepoint_rollback(sid) c.save() # Succeeds, and a.save() is never undone In this example, ``a.save()`` will not be undone in the case where ``b.save()`` raises an exception. Under the hood ============== .. _autocommit-details: Details on autocommit --------------------- In the SQL standards, each SQL query starts a transaction, unless one is already in progress. Such transactions must then be committed or rolled back. This isn't always convenient for application developers. To alleviate this problem, most databases provide an autocommit mode. When autocommit is turned on, each SQL query is wrapped in its own transaction. In other words, the transaction is not only automatically started, but also automatically committed. :pep:`249`, the Python Database API Specification v2.0, requires autocommit to be initially turned off. Django overrides this default and turns autocommit on. To avoid this, you can :ref:`deactivate the transaction management `, but it isn't recommended. .. versionchanged:: 1.6 Before Django 1.6, autocommit was turned off, and it was emulated by forcing a commit after write operations in the ORM. .. warning:: If you're using the database API directly — for instance, you're running SQL queries with ``cursor.execute()`` — be aware that autocommit is on, and consider wrapping your operations in a transaction to ensure consistency. .. _transaction-states: Transaction management states ----------------------------- At any time, each database connection is in one of these two states: - **auto mode**: autocommit is enabled; - **managed mode**: autocommit is disabled. Django starts in auto mode. ``TransactionMiddleware``, :func:`commit_on_success` and :func:`commit_manually` activate managed mode; :func:`autocommit` activates auto mode. Internally, Django keeps a stack of states. Activations and deactivations must be balanced. For example, at the beginning of each HTTP request, ``TransactionMiddleware`` switches to managed mode; at the end of the request, it commits or rollbacks, and switches back to auto mode. .. admonition:: Nesting decorators / context managers :func:`commit_on_success` has two effects: it changes the transaction state, and defines an atomic transaction block. Nesting with :func:`autocommit` and :func:`commit_manually` will give the expected results in terms of transaction state, but not in terms of transaction semantics. Most often, the inner block will commit, breaking the atomicity of the outer block. Django currently doesn't provide any APIs to create transactions in auto mode. .. _transactions-changes-from-1.5: Changes from Django 1.5 and earlier =================================== Since version 1.6, Django uses database-level autocommit in auto mode. Previously, it implemented application-level autocommit by triggering a commit after each ORM write. As a consequence, each database query (for instance, an ORM read) started a transaction that lasted until the next ORM write. Such "automatic transactions" no longer exist in Django 1.6. There are four known scenarios where this is backwards-incompatible. Note that managed mode isn't affected at all. This section assumes auto mode. See the :ref:`description of modes ` above. Sequences of custom SQL queries ------------------------------- If you're executing several :ref:`custom SQL queries ` in a row, each one now runs in its own transaction, instead of sharing the same "automatic transaction". If you need to enforce atomicity, you must wrap the sequence of queries in :func:`commit_on_success`. To check for this problem, look for calls to ``cursor.execute()``. They're usually followed by a call to ``transaction.commit_unless_managed``, which isn't necessary any more and should be removed. Select for update ----------------- If you were relying on "automatic transactions" to provide locking between :meth:`~django.db.models.query.QuerySet.select_for_update` and a subsequent write operation — an extremely fragile design, but nonetheless possible — you must wrap the relevant code in :func:`commit_on_success`. Using a high isolation level ---------------------------- If you were using the "repeatable read" isolation level or higher, and if you relied on "automatic transactions" to guarantee consistency between successive reads, the new behavior is backwards-incompatible. To maintain consistency, you must wrap such sequences in :func:`commit_on_success`. MySQL defaults to "repeatable read" and SQLite to "serializable"; they may be affected by this problem. At the "read committed" isolation level or lower, "automatic transactions" have no effect on the semantics of any sequence of ORM operations. PostgreSQL and Oracle default to "read committed" and aren't affected, unless you changed the isolation level. Using unsupported database features ----------------------------------- With triggers, views, or functions, it's possible to make ORM reads result in database modifications. Django 1.5 and earlier doesn't deal with this case and it's theoretically possible to observe a different behavior after upgrading to Django 1.6 or later. In doubt, use :func:`commit_on_success` to enforce integrity.