mirror of
https://github.com/django/django.git
synced 2024-12-24 10:05:46 +00:00
872 lines
33 KiB
Plaintext
872 lines
33 KiB
Plaintext
=========
|
|
Databases
|
|
=========
|
|
|
|
Django attempts to support as many features as possible on all database
|
|
backends. However, not all database backends are alike, and we've had to make
|
|
design decisions on which features to support and which assumptions we can make
|
|
safely.
|
|
|
|
This file describes some of the features that might be relevant to Django
|
|
usage. Of course, it is not intended as a replacement for server-specific
|
|
documentation or reference manuals.
|
|
|
|
General notes
|
|
=============
|
|
|
|
.. _persistent-database-connections:
|
|
|
|
Persistent connections
|
|
----------------------
|
|
|
|
.. versionadded:: 1.6
|
|
|
|
Persistent connections avoid the overhead of re-establishing a connection to
|
|
the database in each request. They're controlled by the
|
|
:setting:`CONN_MAX_AGE` parameter which defines the maximum lifetime of a
|
|
connection. It can be set independently for each database.
|
|
|
|
The default value is ``0``, preserving the historical behavior of closing the
|
|
database connection at the end of each request. To enable persistent
|
|
connections, set :setting:`CONN_MAX_AGE` to a positive number of seconds. For
|
|
unlimited persistent connections, set it to ``None``.
|
|
|
|
Connection management
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Django opens a connection to the database when it first makes a database
|
|
query. It keeps this connection open and reuses it in subsequent requests.
|
|
Django closes the connection once it exceeds the maximum age defined by
|
|
:setting:`CONN_MAX_AGE` or when it isn't usable any longer.
|
|
|
|
In detail, Django automatically opens a connection to the database whenever it
|
|
needs one and doesn't have one already — either because this is the first
|
|
connection, or because the previous connection was closed.
|
|
|
|
At the beginning of each request, Django closes the connection if it has
|
|
reached its maximum age. If your database terminates idle connections after
|
|
some time, you should set :setting:`CONN_MAX_AGE` to a lower value, so that
|
|
Django doesn't attempt to use a connection that has been terminated by the
|
|
database server. (This problem may only affect very low traffic sites.)
|
|
|
|
At the end of each request, Django closes the connection if it has reached its
|
|
maximum age or if it is in an unrecoverable error state. If any database
|
|
errors have occurred while processing the requests, Django checks whether the
|
|
connection still works, and closes it if it doesn't. Thus, database errors
|
|
affect at most one request; if the connection becomes unusable, the next
|
|
request gets a fresh connection.
|
|
|
|
Caveats
|
|
~~~~~~~
|
|
|
|
Since each thread maintains its own connection, your database must support at
|
|
least as many simultaneous connections as you have worker threads.
|
|
|
|
Sometimes a database won't be accessed by the majority of your views, for
|
|
example because it's the database of an external system, or thanks to caching.
|
|
In such cases, you should set :setting:`CONN_MAX_AGE` to a low value or even
|
|
``0``, because it doesn't make sense to maintain a connection that's unlikely
|
|
to be reused. This will help keep the number of simultaneous connections to
|
|
this database small.
|
|
|
|
The development server creates a new thread for each request it handles,
|
|
negating the effect of persistent connections. Don't enable them during
|
|
development.
|
|
|
|
When Django establishes a connection to the database, it sets up appropriate
|
|
parameters, depending on the backend being used. If you enable persistent
|
|
connections, this setup is no longer repeated every request. If you modify
|
|
parameters such as the connection's isolation level or time zone, you should
|
|
either restore Django's defaults at the end of each request, force an
|
|
appropriate value at the beginning of each request, or disable persistent
|
|
connections.
|
|
|
|
.. _postgresql-notes:
|
|
|
|
PostgreSQL notes
|
|
================
|
|
|
|
Django supports PostgreSQL 8.4 and higher.
|
|
|
|
PostgreSQL connection settings
|
|
-------------------------------
|
|
|
|
See :setting:`HOST` for details.
|
|
|
|
Optimizing PostgreSQL's configuration
|
|
-------------------------------------
|
|
|
|
Django needs the following parameters for its database connections:
|
|
|
|
- ``client_encoding``: ``'UTF8'``,
|
|
- ``default_transaction_isolation``: ``'read committed'`` by default,
|
|
or the value set in the connection options (see below),
|
|
- ``timezone``: ``'UTC'`` when :setting:`USE_TZ` is ``True``, value of
|
|
:setting:`TIME_ZONE` otherwise.
|
|
|
|
If these parameters already have the correct values, Django won't set them for
|
|
every new connection, which improves performance slightly. You can configure
|
|
them directly in :file:`postgresql.conf` or more conveniently per database
|
|
user with `ALTER ROLE`_.
|
|
|
|
Django will work just fine without this optimization, but each new connection
|
|
will do some additional queries to set these parameters.
|
|
|
|
.. _ALTER ROLE: http://www.postgresql.org/docs/current/interactive/sql-alterrole.html
|
|
|
|
.. _postgresql-autocommit-mode:
|
|
|
|
Autocommit mode
|
|
---------------
|
|
|
|
.. versionchanged:: 1.6
|
|
|
|
In previous versions of Django, database-level autocommit could be enabled by
|
|
setting the ``autocommit`` key in the :setting:`OPTIONS` part of your database
|
|
configuration in :setting:`DATABASES`::
|
|
|
|
DATABASES = {
|
|
# ...
|
|
'OPTIONS': {
|
|
'autocommit': True,
|
|
},
|
|
}
|
|
|
|
Since Django 1.6, autocommit is turned on by default. This configuration is
|
|
ignored and can be safely removed.
|
|
|
|
.. _database-isolation-level:
|
|
|
|
Isolation level
|
|
---------------
|
|
|
|
.. versionadded:: 1.6
|
|
|
|
Like PostgreSQL itself, Django defaults to the ``READ COMMITTED`` `isolation
|
|
level`_. If you need a higher isolation level such as ``REPEATABLE READ`` or
|
|
``SERIALIZABLE``, set it in the :setting:`OPTIONS` part of your database
|
|
configuration in :setting:`DATABASES`::
|
|
|
|
import psycopg2.extensions
|
|
|
|
DATABASES = {
|
|
# ...
|
|
'OPTIONS': {
|
|
'isolation_level': psycopg2.extensions.ISOLATION_LEVEL_SERIALIZABLE,
|
|
},
|
|
}
|
|
|
|
.. note::
|
|
|
|
Under higher isolation levels, your application should be prepared to
|
|
handle exceptions raised on serialization failures. This option is
|
|
designed for advanced uses.
|
|
|
|
.. _isolation level: http://www.postgresql.org/docs/current/static/transaction-iso.html
|
|
|
|
Indexes for ``varchar`` and ``text`` columns
|
|
--------------------------------------------
|
|
|
|
When specifying ``db_index=True`` on your model fields, Django typically
|
|
outputs a single ``CREATE INDEX`` statement. However, if the database type
|
|
for the field is either ``varchar`` or ``text`` (e.g., used by ``CharField``,
|
|
``FileField``, and ``TextField``), then Django will create
|
|
an additional index that uses an appropriate `PostgreSQL operator class`_
|
|
for the column. The extra index is necessary to correctly perform
|
|
lookups that use the ``LIKE`` operator in their SQL, as is done with the
|
|
``contains`` and ``startswith`` lookup types.
|
|
|
|
.. _PostgreSQL operator class: http://www.postgresql.org/docs/current/static/indexes-opclass.html
|
|
|
|
.. _mysql-notes:
|
|
|
|
MySQL notes
|
|
===========
|
|
|
|
Version support
|
|
---------------
|
|
|
|
Django supports MySQL 5.0.3 and higher.
|
|
|
|
`MySQL 5.0`_ adds the ``information_schema`` database, which contains detailed
|
|
data on all database schema. Django's ``inspectdb`` feature uses it.
|
|
|
|
Django expects the database to support Unicode (UTF-8 encoding) and delegates to
|
|
it the task of enforcing transactions and referential integrity. It is important
|
|
to be aware of the fact that the two latter ones aren't actually enforced by
|
|
MySQL when using the MyISAM storage engine, see the next section.
|
|
|
|
.. _MySQL: http://www.mysql.com/
|
|
.. _MySQL 5.0: http://dev.mysql.com/doc/refman/5.0/en/index.html
|
|
|
|
.. _mysql-storage-engines:
|
|
|
|
Storage engines
|
|
---------------
|
|
|
|
MySQL has several `storage engines`_. You can change the default storage engine
|
|
in the server configuration.
|
|
|
|
Until MySQL 5.5.4, the default engine was MyISAM_ [#]_. The main drawbacks of
|
|
MyISAM are that it doesn't support transactions or enforce foreign-key
|
|
constraints. On the plus side, it was the only engine that supported full-text
|
|
indexing and searching until MySQL 5.6.4.
|
|
|
|
Since MySQL 5.5.5, the default storage engine is InnoDB_. This engine is fully
|
|
transactional and supports foreign key references. It's probably the best
|
|
choice at this point. However, note that the InnoDB autoincrement counter
|
|
is lost on a MySQL restart because it does not remember the
|
|
``AUTO_INCREMENT`` value, instead recreating it as "max(id)+1". This may
|
|
result in an inadvertent reuse of :class:`~django.db.models.AutoField` values.
|
|
|
|
If you upgrade an existing project to MySQL 5.5.5 and subsequently add some
|
|
tables, ensure that your tables are using the same storage engine (i.e. MyISAM
|
|
vs. InnoDB). Specifically, if tables that have a ``ForeignKey`` between them
|
|
use different storage engines, you may see an error like the following when
|
|
running ``migrate``::
|
|
|
|
_mysql_exceptions.OperationalError: (
|
|
1005, "Can't create table '\\db_name\\.#sql-4a8_ab' (errno: 150)"
|
|
)
|
|
|
|
.. _storage engines: http://dev.mysql.com/doc/refman/5.5/en/storage-engines.html
|
|
.. _MyISAM: http://dev.mysql.com/doc/refman/5.5/en/myisam-storage-engine.html
|
|
.. _InnoDB: http://dev.mysql.com/doc/refman/5.5/en/innodb-storage-engine.html
|
|
|
|
.. [#] Unless this was changed by the packager of your MySQL package. We've
|
|
had reports that the Windows Community Server installer sets up InnoDB as
|
|
the default storage engine, for example.
|
|
|
|
.. _mysql-db-api-drivers:
|
|
|
|
MySQL DB API Drivers
|
|
--------------------
|
|
|
|
The Python Database API is described in `PEP 249`_. MySQL has two prominent
|
|
drivers that implement this API:
|
|
|
|
.. _PEP 249: http://www.python.org/dev/peps/pep-0249/
|
|
|
|
- `MySQLdb`_ is a native driver that has been developed and supported for over
|
|
a decade by Andy Dustman.
|
|
- `MySQL Connector/Python`_ is a pure Python driver from Oracle that does not
|
|
require the MySQL client library or any Python modules outside the standard
|
|
library.
|
|
|
|
.. _MySQLdb: https://pypi.python.org/pypi/MySQL-python/1.2.4
|
|
.. _MySQL Connector/Python: http://dev.mysql.com/downloads/connector/python
|
|
|
|
Both drivers are thread-safe and both provide connection pooling. The major
|
|
difference is that MySQL Connector/Python supports Python 3.
|
|
|
|
In addition to a DB API driver, Django needs an adaptor to access the database
|
|
drivers from its ORM. Django provides an adaptor for MySQLdb while MySQL
|
|
Connector/Python includes `its own`_.
|
|
|
|
.. _its own: http://dev.mysql.com/doc/refman/5.6/en/connector-python-django-backend.html
|
|
|
|
MySQLdb
|
|
~~~~~~~
|
|
|
|
Django requires MySQLdb version 1.2.1p2 or later.
|
|
|
|
.. note::
|
|
If you see ``ImportError: cannot import name ImmutableSet`` when trying to
|
|
use Django, your MySQLdb installation may contain an outdated ``sets.py``
|
|
file that conflicts with the built-in module of the same name from Python
|
|
2.4 and later. To fix this, verify that you have installed MySQLdb version
|
|
1.2.1p2 or newer, then delete the ``sets.py`` file in the MySQLdb
|
|
directory that was left by an earlier version.
|
|
|
|
.. note::
|
|
There are known issues with the way MySQLdb converts date strings into
|
|
datetime objects. Specifically, date strings with value 0000-00-00 are
|
|
valid for MySQL but will be converted into None by MySQLdb.
|
|
|
|
This means you should be careful while using loaddata/dumpdata with rows
|
|
that may have 0000-00-00 values, as they will be converted to None.
|
|
|
|
.. note::
|
|
At the time of writing, the latest release of MySQLdb (1.2.4) doesn't
|
|
support Python 3. In order to use MySQLdb under Python 3, you'll have to
|
|
install an unofficial fork, such as `MySQL-for-Python-3`_.
|
|
|
|
This port is still in alpha. In particular, it doesn't support binary
|
|
data, making it impossible to use :class:`django.db.models.BinaryField`.
|
|
|
|
.. _MySQL-for-Python-3: https://github.com/clelland/MySQL-for-Python-3
|
|
|
|
MySQL Connector/Python
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
MySQL Connector/Python is available through two `release branches`_: Generally
|
|
Available (GA, currently 1.0.x) and Development (Dev, currently 1.1.x beta).
|
|
The Django adapter is currently only available in the Dev branch.
|
|
|
|
.. _release branches: http://dev.mysql.com/downloads/connector/python/
|
|
|
|
Creating your database
|
|
----------------------
|
|
|
|
You can `create your database`_ using the command-line tools and this SQL::
|
|
|
|
CREATE DATABASE <dbname> CHARACTER SET utf8;
|
|
|
|
This ensures all tables and columns will use UTF-8 by default.
|
|
|
|
.. _create your database: http://dev.mysql.com/doc/refman/5.0/en/create-database.html
|
|
|
|
.. _mysql-collation:
|
|
|
|
Collation settings
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
The collation setting for a column controls the order in which data is sorted
|
|
as well as what strings compare as equal. It can be set on a database-wide
|
|
level and also per-table and per-column. This is `documented thoroughly`_ in
|
|
the MySQL documentation. In all cases, you set the collation by directly
|
|
manipulating the database tables; Django doesn't provide a way to set this on
|
|
the model definition.
|
|
|
|
.. _documented thoroughly: http://dev.mysql.com/doc/refman/5.0/en/charset.html
|
|
|
|
By default, with a UTF-8 database, MySQL will use the
|
|
``utf8_general_ci`` collation. This results in all string equality
|
|
comparisons being done in a *case-insensitive* manner. That is, ``"Fred"`` and
|
|
``"freD"`` are considered equal at the database level. If you have a unique
|
|
constraint on a field, it would be illegal to try to insert both ``"aa"`` and
|
|
``"AA"`` into the same column, since they compare as equal (and, hence,
|
|
non-unique) with the default collation.
|
|
|
|
In many cases, this default will not be a problem. However, if you really want
|
|
case-sensitive comparisons on a particular column or table, you would change
|
|
the column or table to use the ``utf8_bin`` collation. The main thing to be
|
|
aware of in this case is that if you are using MySQLdb 1.2.2, the database
|
|
backend in Django will then return bytestrings (instead of unicode strings) for
|
|
any character fields it receive from the database. This is a strong variation
|
|
from Django's normal practice of *always* returning unicode strings. It is up
|
|
to you, the developer, to handle the fact that you will receive bytestrings if
|
|
you configure your table(s) to use ``utf8_bin`` collation. Django itself should
|
|
mostly work smoothly with such columns (except for the ``contrib.sessions``
|
|
``Session`` and ``contrib.admin`` ``LogEntry`` tables described below), but
|
|
your code must be prepared to call ``django.utils.encoding.smart_text()`` at
|
|
times if it really wants to work with consistent data -- Django will not do
|
|
this for you (the database backend layer and the model population layer are
|
|
separated internally so the database layer doesn't know it needs to make this
|
|
conversion in this one particular case).
|
|
|
|
If you're using MySQLdb 1.2.1p2, Django's standard
|
|
:class:`~django.db.models.CharField` class will return unicode strings even
|
|
with ``utf8_bin`` collation. However, :class:`~django.db.models.TextField`
|
|
fields will be returned as an ``array.array`` instance (from Python's standard
|
|
``array`` module). There isn't a lot Django can do about that, since, again,
|
|
the information needed to make the necessary conversions isn't available when
|
|
the data is read in from the database. This problem was `fixed in MySQLdb
|
|
1.2.2`_, so if you want to use :class:`~django.db.models.TextField` with
|
|
``utf8_bin`` collation, upgrading to version 1.2.2 and then dealing with the
|
|
bytestrings (which shouldn't be too difficult) as described above is the
|
|
recommended solution.
|
|
|
|
Should you decide to use ``utf8_bin`` collation for some of your tables with
|
|
MySQLdb 1.2.1p2 or 1.2.2, you should still use ``utf8_general_ci``
|
|
(the default) collation for the ``django.contrib.sessions.models.Session``
|
|
table (usually called ``django_session``) and the
|
|
``django.contrib.admin.models.LogEntry`` table (usually called
|
|
``django_admin_log``). Those are the two standard tables that use
|
|
:class:`~django.db.models.TextField` internally.
|
|
|
|
.. _fixed in MySQLdb 1.2.2: http://sourceforge.net/tracker/index.php?func=detail&aid=1495765&group_id=22307&atid=374932
|
|
|
|
Connecting to the database
|
|
--------------------------
|
|
|
|
Refer to the :doc:`settings documentation </ref/settings>`.
|
|
|
|
Connection settings are used in this order:
|
|
|
|
1. :setting:`OPTIONS`.
|
|
2. :setting:`NAME`, :setting:`USER`, :setting:`PASSWORD`,
|
|
:setting:`HOST`, :setting:`PORT`
|
|
3. MySQL option files.
|
|
|
|
In other words, if you set the name of the database in :setting:`OPTIONS`,
|
|
this will take precedence over :setting:`NAME`, which would override
|
|
anything in a `MySQL option file`_.
|
|
|
|
Here's a sample configuration which uses a MySQL option file::
|
|
|
|
# settings.py
|
|
DATABASES = {
|
|
'default': {
|
|
'ENGINE': 'django.db.backends.mysql',
|
|
'OPTIONS': {
|
|
'read_default_file': '/path/to/my.cnf',
|
|
},
|
|
}
|
|
}
|
|
|
|
|
|
# my.cnf
|
|
[client]
|
|
database = NAME
|
|
user = USER
|
|
password = PASSWORD
|
|
default-character-set = utf8
|
|
|
|
Several other MySQLdb connection options may be useful, such as ``ssl``,
|
|
``init_command``, and ``sql_mode``. Consult the `MySQLdb documentation`_ for
|
|
more details.
|
|
|
|
.. _MySQL option file: http://dev.mysql.com/doc/refman/5.0/en/option-files.html
|
|
.. _MySQLdb documentation: http://mysql-python.sourceforge.net/
|
|
|
|
Creating your tables
|
|
--------------------
|
|
|
|
When Django generates the schema, it doesn't specify a storage engine, so
|
|
tables will be created with whatever default storage engine your database
|
|
server is configured for. The easiest solution is to set your database server's
|
|
default storage engine to the desired engine.
|
|
|
|
If you're using a hosting service and can't change your server's default
|
|
storage engine, you have a couple of options.
|
|
|
|
* After the tables are created, execute an ``ALTER TABLE`` statement to
|
|
convert a table to a new storage engine (such as InnoDB)::
|
|
|
|
ALTER TABLE <tablename> ENGINE=INNODB;
|
|
|
|
This can be tedious if you have a lot of tables.
|
|
|
|
* Another option is to use the ``init_command`` option for MySQLdb prior to
|
|
creating your tables::
|
|
|
|
'OPTIONS': {
|
|
'init_command': 'SET storage_engine=INNODB',
|
|
}
|
|
|
|
This sets the default storage engine upon connecting to the database.
|
|
After your tables have been created, you should remove this option as it
|
|
adds a query that is only needed during table creation to each database
|
|
connection.
|
|
|
|
* Another method for changing the storage engine is described in
|
|
AlterModelOnSyncDB_.
|
|
|
|
.. _AlterModelOnSyncDB: https://code.djangoproject.com/wiki/AlterModelOnSyncDB
|
|
|
|
Table names
|
|
-----------
|
|
|
|
There are `known issues`_ in even the latest versions of MySQL that can cause the
|
|
case of a table name to be altered when certain SQL statements are executed
|
|
under certain conditions. It is recommended that you use lowercase table
|
|
names, if possible, to avoid any problems that might arise from this behavior.
|
|
Django uses lowercase table names when it auto-generates table names from
|
|
models, so this is mainly a consideration if you are overriding the table name
|
|
via the :class:`~django.db.models.Options.db_table` parameter.
|
|
|
|
.. _known issues: http://bugs.mysql.com/bug.php?id=48875
|
|
|
|
Savepoints
|
|
----------
|
|
|
|
Both the Django ORM and MySQL (when using the InnoDB :ref:`storage engine
|
|
<mysql-storage-engines>`) support database :ref:`savepoints
|
|
<topics-db-transactions-savepoints>`.
|
|
|
|
If you use the MyISAM storage engine please be aware of the fact that you will
|
|
receive database-generated errors if you try to use the :ref:`savepoint-related
|
|
methods of the transactions API <topics-db-transactions-savepoints>`. The reason
|
|
for this is that detecting the storage engine of a MySQL database/table is an
|
|
expensive operation so it was decided it isn't worth to dynamically convert
|
|
these methods in no-op's based in the results of such detection.
|
|
|
|
Notes on specific fields
|
|
------------------------
|
|
|
|
Character fields
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
Any fields that are stored with ``VARCHAR`` column types have their
|
|
``max_length`` restricted to 255 characters if you are using ``unique=True``
|
|
for the field. This affects :class:`~django.db.models.CharField`,
|
|
:class:`~django.db.models.SlugField` and
|
|
:class:`~django.db.models.CommaSeparatedIntegerField`.
|
|
|
|
DateTime fields
|
|
~~~~~~~~~~~~~~~
|
|
|
|
MySQL does not have a timezone-aware column type. If an attempt is made to
|
|
store a timezone-aware ``time`` or ``datetime`` to a
|
|
:class:`~django.db.models.TimeField` or :class:`~django.db.models.DateTimeField`
|
|
respectively, a ``ValueError`` is raised rather than truncating data.
|
|
|
|
MySQL does not store fractions of seconds. Fractions of seconds are truncated
|
|
to zero when the time is stored.
|
|
|
|
Row locking with ``QuerySet.select_for_update()``
|
|
-------------------------------------------------
|
|
|
|
MySQL does not support the ``NOWAIT`` option to the ``SELECT ... FOR UPDATE``
|
|
statement. If ``select_for_update()`` is used with ``nowait=True`` then a
|
|
``DatabaseError`` will be raised.
|
|
|
|
.. _sqlite-notes:
|
|
|
|
SQLite notes
|
|
============
|
|
|
|
SQLite_ provides an excellent development alternative for applications that
|
|
are predominantly read-only or require a smaller installation footprint. As
|
|
with all database servers, though, there are some differences that are
|
|
specific to SQLite that you should be aware of.
|
|
|
|
.. _SQLite: http://www.sqlite.org/
|
|
|
|
.. _sqlite-string-matching:
|
|
|
|
Substring matching and case sensitivity
|
|
-----------------------------------------
|
|
|
|
For all SQLite versions, there is some slightly counter-intuitive behavior when
|
|
attempting to match some types of strings. These are triggered when using the
|
|
:lookup:`iexact` or :lookup:`contains` filters in Querysets. The behavior
|
|
splits into two cases:
|
|
|
|
1. For substring matching, all matches are done case-insensitively. That is a
|
|
filter such as ``filter(name__contains="aa")`` will match a name of ``"Aabb"``.
|
|
|
|
2. For strings containing characters outside the ASCII range, all exact string
|
|
matches are performed case-sensitively, even when the case-insensitive options
|
|
are passed into the query. So the :lookup:`iexact` filter will behave exactly
|
|
the same as the :lookup:`exact` filter in these cases.
|
|
|
|
Some possible workarounds for this are `documented at sqlite.org`_, but they
|
|
aren't utilised by the default SQLite backend in Django, as incorporating them
|
|
would be fairly difficult to do robustly. Thus, Django exposes the default
|
|
SQLite behavior and you should be aware of this when doing case-insensitive or
|
|
substring filtering.
|
|
|
|
.. _documented at sqlite.org: http://www.sqlite.org/faq.html#q18
|
|
|
|
SQLite 3.3.6 or newer strongly recommended
|
|
------------------------------------------
|
|
|
|
Versions of SQLite 3.3.5 and older contains the following bugs:
|
|
|
|
* A bug when `handling`_ ``ORDER BY`` parameters. This can cause problems when
|
|
you use the ``select`` parameter for the ``extra()`` QuerySet method. The bug
|
|
can be identified by the error message ``OperationalError: ORDER BY terms
|
|
must not be non-integer constants``.
|
|
|
|
* A bug when handling `aggregation`_ together with DateFields and
|
|
DecimalFields.
|
|
|
|
.. _handling: http://www.sqlite.org/cvstrac/tktview?tn=1768
|
|
.. _aggregation: https://code.djangoproject.com/ticket/10031
|
|
|
|
SQLite 3.3.6 was released in April 2006, so most current binary distributions
|
|
for different platforms include newer version of SQLite usable from Python
|
|
through either the ``pysqlite2`` or the ``sqlite3`` modules.
|
|
|
|
Version 3.5.9
|
|
-------------
|
|
|
|
The Ubuntu "Intrepid Ibex" (8.10) SQLite 3.5.9-3 package contains a bug that
|
|
causes problems with the evaluation of query expressions. If you are using
|
|
Ubuntu "Intrepid Ibex", you will need to update the package to version
|
|
3.5.9-3ubuntu1 or newer (recommended) or find an alternate source for SQLite
|
|
packages, or install SQLite from source.
|
|
|
|
At one time, Debian Lenny shipped with the same malfunctioning SQLite 3.5.9-3
|
|
package. However the Debian project has subsequently issued updated versions
|
|
of the SQLite package that correct these bugs. If you find you are getting
|
|
unexpected results under Debian, ensure you have updated your SQLite package
|
|
to 3.5.9-5 or later.
|
|
|
|
The problem does not appear to exist with other versions of SQLite packaged
|
|
with other operating systems.
|
|
|
|
Version 3.6.2
|
|
--------------
|
|
|
|
SQLite version 3.6.2 (released August 30, 2008) introduced a bug into ``SELECT
|
|
DISTINCT`` handling that is triggered by, amongst other things, Django's
|
|
``DateQuerySet`` (returned by the ``dates()`` method on a queryset).
|
|
|
|
You should avoid using this version of SQLite with Django. Either upgrade to
|
|
3.6.3 (released September 22, 2008) or later, or downgrade to an earlier
|
|
version of SQLite.
|
|
|
|
.. _using-newer-versions-of-pysqlite:
|
|
|
|
Using newer versions of the SQLite DB-API 2.0 driver
|
|
----------------------------------------------------
|
|
|
|
For versions of Python 2.5 or newer that include ``sqlite3`` in the standard
|
|
library Django will now use a ``pysqlite2`` interface in preference to
|
|
``sqlite3`` if it finds one is available.
|
|
|
|
This provides the ability to upgrade both the DB-API 2.0 interface or SQLite 3
|
|
itself to versions newer than the ones included with your particular Python
|
|
binary distribution, if needed.
|
|
|
|
"Database is locked" errors
|
|
---------------------------
|
|
|
|
SQLite is meant to be a lightweight database, and thus can't support a high
|
|
level of concurrency. ``OperationalError: database is locked`` errors indicate
|
|
that your application is experiencing more concurrency than ``sqlite`` can
|
|
handle in default configuration. This error means that one thread or process has
|
|
an exclusive lock on the database connection and another thread timed out
|
|
waiting for the lock the be released.
|
|
|
|
Python's SQLite wrapper has
|
|
a default timeout value that determines how long the second thread is allowed to
|
|
wait on the lock before it times out and raises the ``OperationalError: database
|
|
is locked`` error.
|
|
|
|
If you're getting this error, you can solve it by:
|
|
|
|
* Switching to another database backend. At a certain point SQLite becomes
|
|
too "lite" for real-world applications, and these sorts of concurrency
|
|
errors indicate you've reached that point.
|
|
|
|
* Rewriting your code to reduce concurrency and ensure that database
|
|
transactions are short-lived.
|
|
|
|
* Increase the default timeout value by setting the ``timeout`` database
|
|
option option::
|
|
|
|
'OPTIONS': {
|
|
# ...
|
|
'timeout': 20,
|
|
# ...
|
|
}
|
|
|
|
This will simply make SQLite wait a bit longer before throwing "database
|
|
is locked" errors; it won't really do anything to solve them.
|
|
|
|
``QuerySet.select_for_update()`` not supported
|
|
----------------------------------------------
|
|
|
|
SQLite does not support the ``SELECT ... FOR UPDATE`` syntax. Calling it will
|
|
have no effect.
|
|
|
|
"pyformat" parameter style in raw queries not supported
|
|
-------------------------------------------------------
|
|
|
|
For most backends, raw queries (``Manager.raw()`` or ``cursor.execute()``)
|
|
can use the "pyformat" parameter style, where placeholders in the query
|
|
are given as ``'%(name)s'`` and the parameters are passed as a dictionary
|
|
rather than a list. SQLite does not support this.
|
|
|
|
.. _sqlite-connection-queries:
|
|
|
|
Parameters not quoted in ``connection.queries``
|
|
-----------------------------------------------
|
|
|
|
``sqlite3`` does not provide a way to retrieve the SQL after quoting and
|
|
substituting the parameters. Instead, the SQL in ``connection.queries`` is
|
|
rebuilt with a simple string interpolation. It may be incorrect. Make sure
|
|
you add quotes where necessary before copying a query into an SQLite shell.
|
|
|
|
.. _oracle-notes:
|
|
|
|
Oracle notes
|
|
============
|
|
|
|
Django supports `Oracle Database Server`_ versions 9i and
|
|
higher. Oracle version 10g or later is required to use Django's
|
|
``regex`` and ``iregex`` query operators. You will also need at least
|
|
version 4.3.1 of the `cx_Oracle`_ Python driver.
|
|
|
|
Note that due to a Unicode-corruption bug in ``cx_Oracle`` 5.0, that
|
|
version of the driver should **not** be used with Django;
|
|
``cx_Oracle`` 5.0.1 resolved this issue, so if you'd like to use a
|
|
more recent ``cx_Oracle``, use version 5.0.1.
|
|
|
|
``cx_Oracle`` 5.0.1 or greater can optionally be compiled with the
|
|
``WITH_UNICODE`` environment variable. This is recommended but not
|
|
required.
|
|
|
|
.. _`Oracle Database Server`: http://www.oracle.com/
|
|
.. _`cx_Oracle`: http://cx-oracle.sourceforge.net/
|
|
|
|
In order for the ``python manage.py migrate`` command to work, your Oracle
|
|
database user must have privileges to run the following commands:
|
|
|
|
* CREATE TABLE
|
|
* CREATE SEQUENCE
|
|
* CREATE PROCEDURE
|
|
* CREATE TRIGGER
|
|
|
|
To run Django's test suite, the user needs these *additional* privileges:
|
|
|
|
* CREATE USER
|
|
* DROP USER
|
|
* CREATE TABLESPACE
|
|
* DROP TABLESPACE
|
|
* CONNECT WITH ADMIN OPTION
|
|
* RESOURCE WITH ADMIN OPTION
|
|
|
|
The Oracle database backend uses the ``SYS.DBMS_LOB`` package, so your user
|
|
will require execute permissions on it. It's normally accessible to all users
|
|
by default, but in case it is not, you'll need to grant permissions like so:
|
|
|
|
.. code-block:: sql
|
|
|
|
GRANT EXECUTE ON SYS.DBMS_LOB TO user;
|
|
|
|
Connecting to the database
|
|
--------------------------
|
|
|
|
Your Django settings.py file should look something like this for Oracle::
|
|
|
|
DATABASES = {
|
|
'default': {
|
|
'ENGINE': 'django.db.backends.oracle',
|
|
'NAME': 'xe',
|
|
'USER': 'a_user',
|
|
'PASSWORD': 'a_password',
|
|
'HOST': '',
|
|
'PORT': '',
|
|
}
|
|
}
|
|
|
|
|
|
If you don't use a ``tnsnames.ora`` file or a similar naming method that
|
|
recognizes the SID ("xe" in this example), then fill in both
|
|
:setting:`HOST` and :setting:`PORT` like so::
|
|
|
|
DATABASES = {
|
|
'default': {
|
|
'ENGINE': 'django.db.backends.oracle',
|
|
'NAME': 'xe',
|
|
'USER': 'a_user',
|
|
'PASSWORD': 'a_password',
|
|
'HOST': 'dbprod01ned.mycompany.com',
|
|
'PORT': '1540',
|
|
}
|
|
}
|
|
|
|
You should supply both :setting:`HOST` and :setting:`PORT`, or leave both
|
|
as empty strings.
|
|
|
|
Threaded option
|
|
----------------
|
|
|
|
If you plan to run Django in a multithreaded environment (e.g. Apache using the
|
|
the default MPM module on any modern operating system), then you **must** set
|
|
the ``threaded`` option of your Oracle database configuration to True::
|
|
|
|
'OPTIONS': {
|
|
'threaded': True,
|
|
},
|
|
|
|
Failure to do this may result in crashes and other odd behavior.
|
|
|
|
INSERT ... RETURNING INTO
|
|
-------------------------
|
|
|
|
By default, the Oracle backend uses a ``RETURNING INTO`` clause to efficiently
|
|
retrieve the value of an ``AutoField`` when inserting new rows. This behavior
|
|
may result in a ``DatabaseError`` in certain unusual setups, such as when
|
|
inserting into a remote table, or into a view with an ``INSTEAD OF`` trigger.
|
|
The ``RETURNING INTO`` clause can be disabled by setting the
|
|
``use_returning_into`` option of the database configuration to False::
|
|
|
|
'OPTIONS': {
|
|
'use_returning_into': False,
|
|
},
|
|
|
|
In this case, the Oracle backend will use a separate ``SELECT`` query to
|
|
retrieve AutoField values.
|
|
|
|
Naming issues
|
|
-------------
|
|
|
|
Oracle imposes a name length limit of 30 characters. To accommodate this, the
|
|
backend truncates database identifiers to fit, replacing the final four
|
|
characters of the truncated name with a repeatable MD5 hash value.
|
|
Additionally, the backend turns database identifiers to all-uppercase.
|
|
|
|
To prevent these transformations (this is usually required only when dealing
|
|
with legacy databases or accessing tables which belong to other users), use
|
|
a quoted name as the value for ``db_table``::
|
|
|
|
class LegacyModel(models.Model):
|
|
class Meta:
|
|
db_table = '"name_left_in_lowercase"'
|
|
|
|
class ForeignModel(models.Model):
|
|
class Meta:
|
|
db_table = '"OTHER_USER"."NAME_ONLY_SEEMS_OVER_30"'
|
|
|
|
Quoted names can also be used with Django's other supported database
|
|
backends; except for Oracle, however, the quotes have no effect.
|
|
|
|
When running ``migrate``, an ``ORA-06552`` error may be encountered if
|
|
certain Oracle keywords are used as the name of a model field or the
|
|
value of a ``db_column`` option. Django quotes all identifiers used
|
|
in queries to prevent most such problems, but this error can still
|
|
occur when an Oracle datatype is used as a column name. In
|
|
particular, take care to avoid using the names ``date``,
|
|
``timestamp``, ``number`` or ``float`` as a field name.
|
|
|
|
NULL and empty strings
|
|
----------------------
|
|
|
|
Django generally prefers to use the empty string ('') rather than
|
|
NULL, but Oracle treats both identically. To get around this, the
|
|
Oracle backend ignores an explicit ``null`` option on fields that
|
|
have the empty string as a possible value and generates DDL as if
|
|
``null=True``. When fetching from the database, it is assumed that
|
|
a ``NULL`` value in one of these fields really means the empty
|
|
string, and the data is silently converted to reflect this assumption.
|
|
|
|
``TextField`` limitations
|
|
-------------------------
|
|
|
|
The Oracle backend stores ``TextFields`` as ``NCLOB`` columns. Oracle imposes
|
|
some limitations on the usage of such LOB columns in general:
|
|
|
|
* LOB columns may not be used as primary keys.
|
|
|
|
* LOB columns may not be used in indexes.
|
|
|
|
* LOB columns may not be used in a ``SELECT DISTINCT`` list. This means that
|
|
attempting to use the ``QuerySet.distinct`` method on a model that
|
|
includes ``TextField`` columns will result in an error when run against
|
|
Oracle. As a workaround, use the ``QuerySet.defer`` method in conjunction
|
|
with ``distinct()`` to prevent ``TextField`` columns from being included in
|
|
the ``SELECT DISTINCT`` list.
|
|
|
|
.. _third-party-notes:
|
|
|
|
Using a 3rd-party database backend
|
|
==================================
|
|
|
|
In addition to the officially supported databases, there are backends provided
|
|
by 3rd parties that allow you to use other databases with Django:
|
|
|
|
* `Sybase SQL Anywhere`_
|
|
* `IBM DB2`_
|
|
* `Microsoft SQL Server 2005`_
|
|
* Firebird_
|
|
* ODBC_
|
|
* ADSDB_
|
|
|
|
The Django versions and ORM features supported by these unofficial backends
|
|
vary considerably. Queries regarding the specific capabilities of these
|
|
unofficial backends, along with any support queries, should be directed to
|
|
the support channels provided by each 3rd party project.
|
|
|
|
.. _Sybase SQL Anywhere: http://code.google.com/p/sqlany-django/
|
|
.. _IBM DB2: http://code.google.com/p/ibm-db/
|
|
.. _Microsoft SQL Server 2005: http://code.google.com/p/django-mssql/
|
|
.. _Firebird: http://code.google.com/p/django-firebird/
|
|
.. _ODBC: https://github.com/aurorasoftware/django-pyodbc/
|
|
.. _ADSDB: http://code.google.com/p/adsdb-django/
|