============================================
Django 3.0 release notes - UNDER DEVELOPMENT
============================================

*Expected December 2019*

Welcome to Django 3.0!

These release notes cover the :ref:`new features <whats-new-3.0>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-3.0>` you'll
want to be aware of when upgrading from Django 2.2 or earlier. We've
:ref:`dropped some features<removed-features-3.0>` that have reached the end of
their deprecation cycle, and we've :ref:`begun the deprecation process for
some features <deprecated-features-3.0>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 3.0 supports Python 3.6, 3.7, and 3.8. We **highly recommend** and only
officially support the latest release of each series.

The Django 2.2.x series is the last to support Python 3.5.

Third-party library support for older version of Django
=======================================================

Following the release of Django 3.0, we suggest that third-party app authors
drop support for all versions of Django prior to 2.2. At that time, you should
be able to run your package's tests using ``python -Wd`` so that deprecation
warnings appear. After making the deprecation warning fixes, your app should be
compatible with Django 3.0.

.. _whats-new-3.0:

What's new in Django 3.0
========================

MariaDB support
---------------

Django now officially supports `MariaDB <https://mariadb.org/>`_ 10.1 and
higher. See :ref:`MariaDB notes <mariadb-notes>` for more details.

ASGI support
------------

Django 3.0 begins our journey to making Django fully async-capable by providing
support for running as an `ASGI <https://asgi.readthedocs.io/>`_ application.

This is in addition to our existing WSGI support. Django intends to support
both for the foreseeable future. Async features will only be available to
applications that run under ASGI, however.

There is no need to switch your applications over unless you want to start
experimenting with asynchronous code, but we have
:doc:`documentation on deploying with ASGI </howto/deployment/asgi/index>` if
you want to learn more.

Note that as a side-effect of this change, Django is now aware of asynchronous
event loops and will block you calling code marked as "async unsafe" - such as
ORM operations - from an asynchronous context. If you were using Django from
async code before, this may trigger if you were doing it incorrectly. If you
see a ``SynchronousOnlyOperation`` error, then closely examine your code and
move any database operations to be in a synchronous child thread.

Exclusion constraints on PostgreSQL
-----------------------------------

The new :class:`~django.contrib.postgres.constraints.ExclusionConstraint` class
enable adding exclusion constraints on PostgreSQL. Constraints are added to
models using the
:attr:`Meta.constraints <django.db.models.Options.constraints>` option.

Filter expressions
------------------

Expressions that output :class:`~django.db.models.BooleanField` may now be
used directly in ``QuerySet`` filters, without having to first annotate and
then filter against the annotation.

Enumerations for model field choices
------------------------------------

Custom enumeration types ``TextChoices``, ``IntegerChoices``, and ``Choices``
are now available as a way to define :attr:`.Field.choices`. ``TextChoices``
and ``IntegerChoices`` types are provided for text and integer fields. The
``Choices`` class allows defining a compatible enumeration for other concrete
data types. These custom enumeration types support human-readable labels that
can be translated and accessed via a property on the enumeration or its
members. See :ref:`Enumeration types <field-choices-enum-types>` for more
details and examples.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added support for the ``admin_order_field`` attribute on properties in
  :attr:`.ModelAdmin.list_display`.

* The new :meth:`ModelAdmin.get_inlines()
  <django.contrib.admin.ModelAdmin.get_inlines>` method allows specifying the
  inlines based on the request or model instance.

* Select2 library is upgraded from version 4.0.3 to 4.0.7.

* jQuery is upgraded from version 3.3.1 to 3.4.1.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``reset_url_token`` attribute in
  :class:`~django.contrib.auth.views.PasswordResetConfirmView` allows
  specifying a token parameter displayed as a component of password reset
  URLs.

* Added :class:`~django.contrib.auth.backends.BaseBackend` class to ease
  customization of authentication backends.

* Added :meth:`~django.contrib.auth.models.User.get_user_permissions()` method
  to mirror the existing
  :meth:`~django.contrib.auth.models.User.get_group_permissions()` method.

* Added HTML ``autocomplete`` attribute to widgets of username, email, and
  password fields in :mod:`django.contrib.auth.forms` for better interaction
  with browser password managers.

* :djadmin:`createsuperuser` now falls back to environment variables for
  password and required fields, when a corresponding command line argument
  isn't provided in non-interactive mode.

* :attr:`~django.contrib.auth.models.CustomUser.REQUIRED_FIELDS` now supports
  :class:`~django.db.models.ManyToManyField`\s.

* The new :meth:`.UserManager.with_perm` method returns users that have the
  specified permission.

* The default iteration count for the PBKDF2 password hasher is increased from
  150,000 to 180,000.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* Allowed MySQL spatial lookup functions to operate on real geometries.
  Previous support was limited to bounding boxes.

* Added the :class:`~django.contrib.gis.db.models.functions.GeometryDistance`
  function, supported on PostGIS.

* Added support for the ``furlong`` unit in
  :class:`~django.contrib.gis.measure.Distance`.

* The :setting:`GEOIP_PATH` setting now supports :class:`pathlib.Path`.

* The :class:`~django.contrib.gis.geoip2.GeoIP2` class now accepts
  :class:`pathlib.Path` ``path``.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :class:`~django.contrib.postgres.fields.RangeOperators` helps to
  avoid typos in SQL operators that can be used together with
  :class:`~django.contrib.postgres.fields.RangeField`.

* The new :class:`~django.contrib.postgres.fields.RangeBoundary` expression
  represents the range boundaries.

* The new :class:`~django.contrib.postgres.operations.AddIndexConcurrently`
  and :class:`~django.contrib.postgres.operations.RemoveIndexConcurrently`
  classes allow creating and dropping indexes ``CONCURRENTLY`` on PostgreSQL.

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new
  :meth:`~django.contrib.sessions.backends.base.SessionBase.get_session_cookie_age()`
  method allows dynamically specifying the session cookie age.

:mod:`django.contrib.syndication`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added the ``language`` class attribute to the
  :class:`django.contrib.syndication.views.Feed` to customize a feed language.
  The default value is :func:`~django.utils.translation.get_language()` instead
  of :setting:`LANGUAGE_CODE`.

Cache
~~~~~

* :func:`~django.utils.cache.add_never_cache_headers` and
  :func:`~django.views.decorators.cache.never_cache` now add the ``private``
  directive to ``Cache-Control`` headers.

File Storage
~~~~~~~~~~~~

* The new :meth:`.Storage.get_alternative_name` method allows customizing the
  algorithm for generating filenames if a file with the uploaded name already
  exists.

Forms
~~~~~

* Formsets may control the widget used when ordering forms via
  :attr:`~django.forms.formsets.BaseFormSet.can_order` by setting the
  :attr:`~django.forms.formsets.BaseFormSet.ordering_widget` attribute or
  overriding :attr:`~django.forms.formsets.BaseFormSet.get_ordering_widget()`.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* Added the :setting:`LANGUAGE_COOKIE_HTTPONLY`,
  :setting:`LANGUAGE_COOKIE_SAMESITE`, and :setting:`LANGUAGE_COOKIE_SECURE`
  settings to set the ``HttpOnly``, ``SameSite``, and ``Secure`` flags on
  language cookies. The default values of these settings preserve the previous
  behavior.

* Added support and translations for the Uzbek language.

Logging
~~~~~~~

* The new ``reporter_class`` parameter of
  :class:`~django.utils.log.AdminEmailHandler` allows providing an
  ``django.views.debug.ExceptionReporter`` subclass to customize the traceback
  text sent to site :setting:`ADMINS` when :setting:`DEBUG` is ``False``.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :option:`compilemessages --ignore` option allows ignoring specific
  directories when searching for ``.po`` files to compile.

* :option:`showmigrations --list` now shows the applied datetimes when
  ``--verbosity`` is 2 and above.

* On PostgreSQL, :djadmin:`dbshell` now supports client-side TLS certificates.

* :djadmin:`inspectdb` now introspects :class:`~django.db.models.OneToOneField`
  when a foreign key has a unique or primary key constraint.

* The new :option:`--skip-checks` option skips running system checks prior to
  running the command.

* The :option:`startapp --template` and :option:`startproject --template`
  options now support templates stored in XZ archives (``.tar.xz``, ``.txz``)
  and LZMA archives (``.tar.lzma``, ``.tlz``).

Models
~~~~~~

* Added hash database functions :class:`~django.db.models.functions.MD5`,
  :class:`~django.db.models.functions.SHA1`,
  :class:`~django.db.models.functions.SHA224`,
  :class:`~django.db.models.functions.SHA256`,
  :class:`~django.db.models.functions.SHA384`, and
  :class:`~django.db.models.functions.SHA512`.

* Added the :class:`~django.db.models.functions.Sign` database function.

* The new ``is_dst``  parameter of the
  :class:`~django.db.models.functions.Trunc` database functions determines the
  treatment of nonexistent and ambiguous datetimes.

* ``connection.queries`` now shows ``COPY … TO`` statements on PostgreSQL.

* :class:`~django.db.models.FilePathField` now accepts a callable for ``path``.

* Allowed symmetrical intermediate table for self-referential
  :class:`~django.db.models.ManyToManyField`.

* The ``name`` attributes of :class:`~django.db.models.CheckConstraint`,
  :class:`~django.db.models.UniqueConstraint`, and
  :class:`~django.db.models.Index` now support app label and class
  interpolation using the ``'%(app_label)s'`` and ``'%(class)s'`` placeholders.

* The new :attr:`.Field.descriptor_class` attribute allows model fields to
  customize the get and set behavior by overriding their
  :py:ref:`descriptors <descriptors>`.

* :class:`~django.db.models.Avg` and :class:`~django.db.models.Sum` now support
  the ``distinct`` argument.

* Added :class:`~django.db.models.SmallAutoField` which acts much like an
  :class:`~django.db.models.AutoField` except that it only allows values under
  a certain (database-dependent) limit. Values from ``1`` to ``32767`` are safe
  in all databases supported by Django.

* :class:`~django.db.models.AutoField`,
  :class:`~django.db.models.BigAutoField`, and
  :class:`~django.db.models.SmallAutoField` now inherit from
  ``IntegerField``, ``BigIntegerField`` and ``SmallIntegerField`` respectively.
  System checks and validators are now also properly inherited.

* :attr:`.FileField.upload_to` now supports :class:`pathlib.Path`.

* :class:`~django.db.models.CheckConstraint` is now supported on MySQL 8.0.16+.

* The new ``allows_group_by_selected_pks_on_model()`` method of
  ``django.db.backends.base.BaseDatabaseFeatures`` allows optimization of
  ``GROUP BY`` clauses to require only the selected models' primary keys. By
  default, it's supported only for managed models on PostgreSQL.

  To enable the ``GROUP BY`` primary key-only optimization for unmanaged
  models, you have to subclass the PostgreSQL database engine, overriding the
  features class ``allows_group_by_selected_pks_on_model()`` method as you
  require. See :ref:`Subclassing the built-in database backends
  <subclassing-database-backends>` for an example.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Allowed :class:`~django.http.HttpResponse` to be initialized with
  :class:`memoryview` content.

* For use in, for example, Django templates, :attr:`.HttpRequest.headers` now
  allows look ups using underscores (e.g. ``user_agent``) in place of hyphens.

.. _whats-new-security-3.0:

Security
~~~~~~~~

* :setting:`X_FRAME_OPTIONS` now defaults to ``'DENY'``. In older versions, the
  :setting:`X_FRAME_OPTIONS` setting defaults to ``'SAMEORIGIN'``. If your site
  uses frames of itself, you will need to explicitly set ``X_FRAME_ORIGINS =
  'SAMEORIGIN'`` for them to continue working.

* :setting:`SECURE_CONTENT_TYPE_NOSNIFF` setting now defaults to ``True``. With
  the enabled :setting:`SECURE_CONTENT_TYPE_NOSNIFF`, the
  :class:`~django.middleware.security.SecurityMiddleware` sets the
  :ref:`x-content-type-options` header on all responses that do not already
  have it.

* :class:`~django.middleware.security.SecurityMiddleware` can now send the
  :ref:`Referrer-Policy <referrer-policy>` header.

Tests
~~~~~

* The new test :class:`~django.test.Client` argument
  ``raise_request_exception`` allows controlling whether or not exceptions
  raised during the request should also be raised in the test. The value
  defaults to ``True`` for backwards compatibility. If it is ``False`` and an
  exception occurs, the test client will return a 500 response with the
  attribute :attr:`~django.test.Response.exc_info`, a tuple providing
  information of the exception that occurred.

* Tests and test cases to run can be selected by test name pattern using the
  new :option:`test -k` option.

* HTML comparison, as used by
  :meth:`~django.test.SimpleTestCase.assertHTMLEqual`, now treats text, character
  references, and entity references that refer to the same character as
  equivalent.

* Django test runner now supports headless mode for selenium tests on supported
  browsers. Add the ``--headless`` option to enable this mode.

* Django test runner now supports ``--start-at`` and ``--start-after`` options
  to run tests starting from a specific top-level module.

* Django test runner now supports a ``--pdb`` option to spawn a debugger at
  each error or failure.

.. _backwards-incompatible-3.0:

Backwards incompatible changes in 3.0
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* The second argument of ``DatabaseIntrospection.get_geometry_type()`` is now
  the row description instead of the column name.

* ``DatabaseIntrospection.get_field_type()`` may no longer return tuples.

* If the database can create foreign keys in the same SQL statement that adds a
  field, add ``SchemaEditor.sql_create_column_inline_fk`` with the appropriate
  SQL; otherwise, set ``DatabaseFeatures.can_create_inline_fk = False``.

* ``DatabaseFeatures.can_return_id_from_insert`` and
  ``can_return_ids_from_bulk_insert`` are renamed to
  ``can_return_columns_from_insert`` and ``can_return_rows_from_bulk_insert``.

* Database functions now handle :class:`datetime.timezone` formats when created
  using :class:`datetime.timedelta` instances (e.g.
  ``timezone(timedelta(hours=5))``, which would output ``'UTC+05:00'``).
  Third-party backends should handle this format when preparing
  :class:`~django.db.models.DateTimeField` in ``datetime_cast_date_sql()``,
  ``datetime_extract_sql()``, etc.

* Entries for ``AutoField``, ``BigAutoField``, and ``SmallAutoField`` are added
  to  ``DatabaseOperations.integer_field_ranges`` to support the integer range
  validators on these field types. Third-party backends may need to customize
  the default entries.

* ``DatabaseOperations.fetch_returned_insert_id()`` is replaced by
  ``fetch_returned_insert_columns()`` which returns a list of values returned
  by the ``INSERT … RETURNING`` statement, instead of a single value.

* ``DatabaseOperations.return_insert_id()`` is replaced by
  ``return_insert_columns()`` that accepts a ``fields``
  argument, which is an iterable of fields to be returned after insert. Usually
  this is only the auto-generated primary key.

:mod:`django.contrib.admin`
---------------------------

* Admin's model history change messages now prefers more readable field labels
  instead of field names.

:mod:`django.contrib.gis`
-------------------------

* Support for PostGIS 2.1 is removed.

* Support for SpatiaLite 4.1 and 4.2 is removed.

* Support for GDAL 1.11 and GEOS 3.4 is removed.

Dropped support for PostgreSQL 9.4
----------------------------------

Upstream support for PostgreSQL 9.4 ends in December 2019. Django 3.0 supports
PostgreSQL 9.5 and higher.

Dropped support for Oracle 12.1
-------------------------------

Upstream support for Oracle 12.1 ends in July 2021. Django 2.2 will be
supported until April 2022. Django 3.0 officially supports Oracle 12.2 and 18c.

Removed private Python 2 compatibility APIs
-------------------------------------------

While Python 2 support was removed in Django 2.0, some private APIs weren't
removed from Django so that third party apps could continue using them until
the Python 2 end-of-life.

Since we expect apps to drop Python 2 compatibility when adding support for
Django 3.0, we're removing these APIs at this time.

* ``django.test.utils.str_prefix()`` - Strings don't have 'u' prefixes in
  Python 3.

* ``django.test.utils.patch_logger()`` - Use
  :meth:`unittest.TestCase.assertLogs` instead.

* ``django.utils.lru_cache.lru_cache()`` - Alias of
  :func:`functools.lru_cache`.

* ``django.utils.decorators.available_attrs()`` - This function returns
  ``functools.WRAPPER_ASSIGNMENTS``.

* ``django.utils.decorators.ContextDecorator`` - Alias of
  :class:`contextlib.ContextDecorator`.

* ``django.utils._os.abspathu()`` - Alias of :func:`os.path.abspath`.

* ``django.utils._os.upath()`` and ``npath()`` - These functions do nothing on
  Python 3.

* ``django.utils.six`` - Remove usage of this vendored library or switch to
  `six <https://pypi.org/project/six/>`_.

* ``django.utils.encoding.python_2_unicode_compatible()`` - Alias of
  ``six.python_2_unicode_compatible()``.

* ``django.utils.functional.curry()`` - Use :func:`functools.partial` or
  :class:`functools.partialmethod`. See :commit:`5b1c389603a353625ae1603`.

* ``django.utils.safestring.SafeBytes`` - Unused since Django 2.0.

New default value for the ``FILE_UPLOAD_PERMISSIONS`` setting
-------------------------------------------------------------

In older versions, the :setting:`FILE_UPLOAD_PERMISSIONS` setting defaults to
``None``. With the default :setting:`FILE_UPLOAD_HANDLERS`, this results in
uploaded files having different permissions depending on their size and which
upload handler is used.

``FILE_UPLOAD_PERMISSION`` now defaults to ``0o644`` to avoid this
inconsistency.

New default values for security settings
----------------------------------------

To make Django projects more secure by default, some security settings now have
more secure default values:

* :setting:`X_FRAME_OPTIONS` now defaults to ``'DENY'``.

* :setting:`SECURE_CONTENT_TYPE_NOSNIFF` now defaults to ``True``.

See the *What's New* :ref:`Security section <whats-new-security-3.0>` above for
more details on these changes.

Miscellaneous
-------------

* ``ContentType.__str__()`` now includes the model's ``app_label`` to
  disambiguate model's with the same name in different apps.

* Because accessing the language in the session rather than in the cookie is
  deprecated, ``LocaleMiddleware`` no longer looks for the user's language in
  the session and :func:`django.contrib.auth.logout` no longer preserves the
  session's language after logout.

* :func:`django.utils.html.escape` now uses :func:`html.escape` to escape HTML.
  This converts ``'`` to ``&#x27;`` instead of the previous equivalent decimal
  code ``&#39;``.

* The ``django-admin test -k`` option now works as the :option:`unittest
  -k<unittest.-k>` option rather than as a shortcut for ``--keepdb``.

* Support for ``pywatchman`` < 1.2.0 is removed.

* HTML rendered by :class:`~django.forms.SelectDateWidget` for required fields
  now have the ``placeholder`` attribute, which, mainly, may require some
  adjustments in tests that compare HTML.

* :func:`~django.utils.http.urlencode` now encodes iterable values as they are
  when ``doseq=False``, rather than iterating them, bringing it into line with
  the standard library :func:`urllib.parse.urlencode` function.

* ``intword`` template filter now translates ``1.0`` as a singular phrase and
  all other numeric values as plural. This may be incorrect for some languages.

* Assigning a value to a model's :class:`~django.db.models.ForeignKey` or
  :class:`~django.db.models.OneToOneField` ``'_id'`` attribute now unsets the
  corresponding field. Accessing the field afterwards will result in a query.

* :func:`~django.utils.cache.patch_vary_headers` now handles an asterisk
  ``'*'`` according to :rfc:`7231#section-7.1.4`, i.e. if a list of header
  field names contains an asterisk, then the ``Vary`` header will consist of a
  single asterisk ``'*'``.

* On MySQL 8.0.16+, ``PositiveIntegerField`` and ``PositiveSmallIntegerField``
  now include a check constraint to prevent negative values in the database.

* ``alias=None`` is added to the signature of
  :meth:`.Expression.get_group_by_cols`.

.. _deprecated-features-3.0:

Features deprecated in 3.0
==========================

``django.utils.encoding.force_text()`` and ``smart_text()``
-----------------------------------------------------------

The ``smart_text()`` and ``force_text()`` aliases (since Django 2.0) of
``smart_str()`` and ``force_str()`` are deprecated. Ignore this deprecation if
your code supports Python 2 as the behavior of ``smart_str()`` and
``force_str()`` is different there.

Miscellaneous
-------------

* ``django.utils.http.urlquote()``, ``urlquote_plus()``, ``urlunquote()``, and
  ``urlunquote_plus()`` are deprecated in favor of the functions that they're
  aliases for: :func:`urllib.parse.quote`, :func:`~urllib.parse.quote_plus`,
  :func:`~urllib.parse.unquote`, and :func:`~urllib.parse.unquote_plus`.

* ``django.utils.translation.ugettext()``, ``ugettext_lazy()``,
  ``ugettext_noop()``, ``ungettext()``, and ``ungettext_lazy()`` are deprecated
  in favor of the functions that they're aliases for:
  :func:`django.utils.translation.gettext`,
  :func:`~django.utils.translation.gettext_lazy`,
  :func:`~django.utils.translation.gettext_noop`,
  :func:`~django.utils.translation.ngettext`, and
  :func:`~django.utils.translation.ngettext_lazy`.

* To limit creation of sessions and hence favor some caching strategies,
  :func:`django.views.i18n.set_language` will stop setting the user's language
  in the session in Django 4.0. Since Django 2.1, the language is always stored
  in the :setting:`LANGUAGE_COOKIE_NAME` cookie.

* ``django.utils.text.unescape_entities()`` is deprecated in favor of
  :func:`html.unescape`. Note that unlike ``unescape_entities()``,
  ``html.unescape()`` evaluates lazy strings immediately.

* To avoid possible confusion as to effective scope, the private internal
  utility ``is_safe_url()`` is renamed to
  ``url_has_allowed_host_and_scheme()``. That a URL has an allowed host and
  scheme doesn't in general imply that it's "safe". It may still be quoted
  incorrectly, for example. Ensure to also use
  :func:`~django.utils.encoding.iri_to_uri` on the path component of untrusted
  URLs.

.. _removed-features-3.0:

Features removed in 3.0
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 3.0.

See :ref:`deprecated-features-2.0` for details on these changes, including how
to remove usage of these features.

* The ``django.db.backends.postgresql_psycopg2`` module is removed.

* ``django.shortcuts.render_to_response()`` is removed.

* The ``DEFAULT_CONTENT_TYPE`` setting is removed.

* ``HttpRequest.xreadlines()`` is removed.

* Support for the ``context`` argument of ``Field.from_db_value()`` and
  ``Expression.convert_value()`` is removed.

* The ``field_name`` keyword argument of ``QuerySet.earliest()`` and
  ``latest()`` is removed.

See :ref:`deprecated-features-2.1` for details on these changes, including how
to remove usage of these features.

* The ``ForceRHR`` GIS function is removed.

* ``django.utils.http.cookie_date()`` is removed.

* The ``staticfiles`` and ``admin_static`` template tag libraries are removed.

* ``django.contrib.staticfiles.templatetags.staticfiles.static()`` is removed.