Add the starts of some documentation

Still lots more to write

docs/ref/index.txt

@@ -25,6 +25,7 @@ API Reference
    schema-editor
    settings
    signals
+   tasks
    templates/index
    template-response
    unicode

docs/ref/settings.txt

@@ -2660,6 +2660,68 @@ backend definition in :setting:`STORAGES`.
     Defining this setting overrides the default value and is *not* merged with
     it.
 
+.. setting:: TASKS
+
+``TASKS``
+----------
+
+Default::
+
+    {
+        "default": {
+            "BACKEND": "django.tasks.backends.immediate.ImmediateBackend",
+        }
+    }
+
+A dictionary containing the settings for all task backends to be used with
+Django. It is a nested dictionary whose contents maps backend aliases to a
+dictionary containing the options for each backend.
+
+The :setting:`TASKS` setting must configure a ``default`` backend; any number
+of additional backends may also be specified. Depending on which backend is used,
+other options may be required. The following options are available as standard:
+
+.. setting:: TASKS-BACKEND
+
+``BACKEND``
+~~~~~~~~~~~
+
+Default: ``''`` (Empty string)
+
+The task backend to use. The built-in backends are:
+
+* ``'django.tasks.backends.dummy.DummyBackend'``
+* ``'django.tasks.backends.immediate.ImmediateBackend'``
+
+You can use a backend that doesn't ship with Django by setting
+:setting:`BACKEND <TASKS-BACKEND>` to a fully-qualified path of a backend
+class (i.e. ``mypackage.backends.whatever.WhateverBackend``).
+
+.. setting:: TASKS-ENQUEUE_ON_COMMIT
+
+``ENQUEUE_ON_COMMIT``
+~~~~~~~~~~~~~~~~~~~~~
+
+Default: ``True``
+
+Whether a task should be enqueued at the end of the current transaction (if there
+is one) commits successfully, rather than enqueueing immediately.
+
+This can also be configured on a per-task basis.
+
+.. setting:: TASKS-QUEUES
+
+``QUEUES``
+~~~~~~~~~~~~~~~~~~~~~
+
+Default: ``["default"]``
+
+Specify the queue names supported by the backend. This can be used to ensure
+tasks aren't enqueued to queues which don't exist.
+
+To disable queue name validation, set :setting:`QUEUES <TASKS-QUEUES>` to an
+empty list (``[]``).
+
 .. setting:: TEMPLATES
 
 ``TEMPLATES``

docs/ref/signals.txt

@@ -703,3 +703,34 @@ Arguments sent with this signal:
     The database connection that was opened. This can be used in a
     multiple-database configuration to differentiate connection signals
     from different databases.
+
+Tasks signals
+=============
+
+Signals sent by the :doc:`tasks </ref/tasks>` framework.
+
+.. data:: django.tasks.signals.task_enqueued
+
+Sent once a task has been enqueued. If
+:attr:`django.tasks.task.Task.enqueue_on_commit` is set, the signal is only sent
+once the transaction commits successfully.
+
+Arguments that are sent with this signal:
+
+``sender``
+    The backend class which the task was enqueued on to.
+
+``task_result``
+    The enqueued :class:`TaskResult <django.tasks.task.TaskResult>`.
+
+.. data:: django.tasks.signals.task_finished
+
+Sent once a task has finished executing, successfully or otherwise.
+
+Arguments that are sent with this signal:
+
+``sender``
+    The backend class which the task was enqueued on to.
+
+``task_result``
+    The finished :class:`TaskResult <django.tasks.task.TaskResult>`.

docs/ref/tasks.txt

@@ -0,0 +1,296 @@
+=====
+Tasks
+=====
+
+.. module:: django.tasks
+   :synopsis: Django's built-in background task system.
+
+Backends
+========
+
+Base backend
+------------
+
+.. module:: django.tasks.backends.base
+
+.. class:: BaseTaskBackend
+
+``BaseTaskBackend`` is the parent class for all task backends.
+
+.. method:: BaseTaskBackend.get_result(result_id)
+
+    Retrieve a result by its id. If the result does not exist,
+    :exc:`ResultDoesNotExist <django.tasks.exceptions.ResultDoesNotExist>`
+    is raised.
+
+    If the backend does not support ``get_result``, :exc:`NotImplementedError`
+    is raised.
+
+.. method:: BaseTaskBackend.aget_result(result_id)
+
+    The ``async`` variant of :meth:`BaseTaskBackend.get_result`.
+
+Introspection
+~~~~~~~~~~~~~
+
+Introspection allows for identifying the supported functionality of a
+backend, and potentially changing behavior accordingly.
+
+.. attribute:: BaseTaskBackend.supports_defer
+
+    Whether the backend supports enqueueing tasks to be
+    executed after a specific time using the ``run_after`` attribute.
+
+.. attribute:: BaseTaskBackend.supports_async_task
+
+    Whether the backend supports enqueueing async functions (coroutines).
+
+.. attribute:: BaseTaskBackend.supports_get_result
+
+    Whether the backend supports retrieving task results from another thread
+    after they have been enqueued.
+
+Available backends
+------------------
+
+Immediate backend
+~~~~~~~~~~~~~~~~~
+
+.. module:: django.tasks.backends.immediate
+
+.. class:: ImmediateBackend
+
+The immediate backend executes tasks immediately, rather than in the background.
+
+Dummy backend
+~~~~~~~~~~~~~
+
+.. module:: django.tasks.backends.dummy
+
+.. class:: DummyBackend
+
+The dummy backend doesn't execute enqueued tasks at all, instead storing results
+for later use.
+
+.. attribute:: DummyBackend.results
+
+    A list of results for the enqueued tasks, in the order they were enqueued.
+
+.. method:: DummyBackend.clear
+
+    Clears the list of stored results.
+
+Task definition
+===============
+
+.. module:: django.tasks.task
+
+.. function:: task
+
+A decorator defining a :class:`Task`::
+
+    from django.tasks import task
+
+
+    @task
+    def calculate_meaning_of_life() -> int:
+        return 42
+
+In the above example, ``calculate_meaning_of_life`` is a :class:`Task` instance.
+
+``task`` accepts a number of arguments, which are used as-is for the ``Task``:
+
+* :attr:`priority <Task.priority>`
+* :attr:`backend <Task.backend>`
+* :attr:`queue_name <Task.queue_name>`
+* :attr:`enqueue_on_commit <Task.enqueue_on_commit>`
+
+If the task is not valid, according to the backend,
+:exc:`django.tasks.exceptions.InvalidTaskError` is raised.
+
+.. class:: Task
+
+An class representing a task to be run in the background. Tasks should be
+defined using the :func:`task` decorator.
+
+Attributes of ``Task`` cannot be modified.
+
+.. attribute:: Task.priority
+
+    The priority of the task. Priorities must be between -100 and 100.
+
+    By default, tasks are enqueued with a priority of 0.
+
+.. attribute:: Task.backend
+
+    The alias of the backend the task should be enqueued to.
+
+.. attribute:: Task.queue_name
+
+    The name of the queue the task will be enqueued on to. Defaults to
+    ``"default"``.
+
+    This must match a queue defined in :setting:`QUEUES <TASKS-QUEUES>`.
+
+.. attribute:: Task.run_after
+
+    The earliest time the task will be executed, or ``None`` to have no time set.
+
+.. attribute:: Task.enqueue_on_commit
+
+    Whether the task should be enqueued when the transaction commits successfully,
+    or immediately.
+
+    By default, the behavior of the backend's
+    :setting:`ENQUEUE_ON_COMMIT <TASKS-ENQUEUE_ON_COMMIT>` is used.
+
+.. attribute:: Task.name
+
+    An identifier for a task. Potentially useful for debugging.
+
+    A task's name is not necessarily unique.
+
+.. method:: Task.using()
+
+    Create a new task with modified defaults. The existing task is left unchanged.
+
+    ``using`` allows modifying the following attributes:
+
+    * :attr:`priority <Task.priority>`
+    * :attr:`backend <Task.backend>`
+    * :attr:`queue_name <Task.queue_name>`
+    * :attr:`run_after <Task.run_after>`
+
+    ``run_after`` may also be provided as a :class:`timedelta <datetime.timedelta>`, which
+    is used relative to the current time (when ``using`` is called).
+
+.. method:: Task.enqueue(*args, **kwargs)
+
+    Enqueue the task for later execution.
+
+    Arguments and keyword arguments are passed to the task's function as-is.
+
+    If the task is not valid, according to the backend,
+    :exc:`django.tasks.exceptions.InvalidTaskError` is raised.
+
+.. method:: Task.aenqueue(*args, **kwargs)
+
+    The ``async`` variant of :meth:`enqueue <Task.enqueue>`.
+
+.. method:: Task.get_result(result_id)
+
+    Retrieve a result by its id. If the result does not exist, or is not the same
+    type as the current task,
+    :exc:`ResultDoesNotExist <django.tasks.exceptions.ResultDoesNotExist>`
+    is raised.
+
+    If the backend does not support ``get_result``, :exc:`NotImplementedError`
+    is raised.
+
+.. method:: Task.aget_result(*args, **kwargs)
+
+    The ``async`` variant of :meth:`get_result <Task.get_result>`.
+
+Task results
+============
+
+.. class:: ResultStatus
+
+    An Enum representing the status of a :class:`TaskResult`.
+
+.. attribute:: ResultStatus.NEW
+
+    The :class:`Task` has not been run.
+
+.. attribute:: ResultStatus.RUNNING
+
+    The :class:`Task` is currently being executed.
+
+.. attribute:: ResultStatus.FAILED
+
+    The :class:`Task` raised an exception during execution, or was unable
+    to start.
+
+.. attribute:: ResultStatus.COMPLETE
+
+    The :class:`Task` has finished executing successfully.
+
+.. class:: TaskResult
+
+The ``TaskResult`` stores the information about a specific execution of a
+:class:`Task`.
+
+Attributes of ``TaskResult`` cannot be modified.
+
+.. attribute:: TaskResult.task
+
+    The :class:`Task` the result was enqueued for.
+
+.. attribute:: TaskResult.id
+
+    A unique identifier for the result, which can be passed to
+    :meth:`Task.get_result`.
+
+    The id must be a UUID4, represented as a dashed string.
+
+.. attribute:: TaskResult.status
+
+    The :class:`status <ResultStatus>` of the result.
+
+.. attribute:: TaskResult.enqueued_at
+
+    The time when the task was enqueued.
+
+    If :attr:`Task.enqueue_on_commit` was set, this is the time the transaction
+    completed.
+
+.. attribute:: TaskResult.started_at
+
+    The time when the task began execution.
+
+.. attribute:: TaskResult.finished_at
+
+    The time when the task finished execution, whether it failed or completed.
+
+.. attribute:: TaskResult.backend
+
+    The backend the result is from.
+
+.. attribute:: TaskResult.exception
+
+    The exception raised when executing the task, or ``None`` if no exception
+    was raised.
+
+.. attribute:: TaskResult.traceback
+
+    The exception traceback from the raised exception when the task failed.
+
+.. attribute:: TaskResult.return_value
+
+    The return value from the task function.
+
+    If the task has not finished yet, or failed, :exc:`ValueError` is raised.
+
+.. method:: TaskResult.refresh
+
+    Refresh the result's attributes from the queue store.
+
+.. method:: TaskResult.arefresh
+
+    The ``async`` variant of :meth:`TaskResult.refresh`.
+
+
+Exceptions
+==========
+
+.. module:: django.tasks.exceptions
+
+.. exception:: ResultDoesNotExist
+
+    Raised by :meth:`get_result <django.tasks.backends.base.BaseTaskBackend.get_result>`
+    when the provided ``result_id`` does not exist.
+
+.. exception:: InvalidTaskError
+
+    Raised when the :class:`Task <django.tasks.task.Task>` attempting to be
+    enqueued is invalid.

docs/topics/index.txt

@@ -32,3 +32,4 @@ Introductions to all the key parts of Django you'll need to know:
    checks
    external-packages
    async
+   tasks

docs/topics/tasks.txt

@@ -0,0 +1,135 @@
+========================
+Django's tasks framework
+========================
+
+For a web application, there's often more than just turning HTTP requests into
+HTTP responses. For some functionality, it may be beneficial to run code outside
+of the request-response cycle.
+
+That's where background tasks come in.
+
+Background tasks can offload complexity outside of the request-response cycle,
+to be run somewhere else, potentially at a later date. This keeps requests fast,
+latency down, and improves the user's experience.
+
+Background task fundamentals
+============================
+
+When work needs to be done in the background, Django creates a Task, which is
+stored in the Queue Store. This task contains all of the metadata needed to
+execute the task, as well as a unique identifier for Django to retrieve the
+result later.
+
+Outside of Django, a Worker looks at the Queue Store for new Tasks to run. When
+a new Task is added, the Worker claims the task, executes it, and saves the
+status and result back to the Queue Store.
+
+.. _configuring-a-task-backend:
+
+Configuring a Task backend
+==========================
+
+Background tasks require some work to set up.
+
+Different task backends have different characteristics and configuration options,
+which may impact the performance and reliability of your application.
+
+Task backends are configured using the :setting:`TASKS` setting in your settings
+file.
+
+.. _immediate-task-backend:
+
+Immediate execution
+-------------------
+
+This is the default backend if another is not specified in your settings file.
+The immediate backend runs enqueued tasks immediately, rather than in the
+background. This allows background task functionality to be slowly added
+to an application, before the required infrastructure is available.
+
+.. warning::
+
+    When :setting:`ENQUEUE_ON_COMMIT <TASKS-ENQUEUE_ON_COMMIT>` is ``False``,
+    the task will be executed within the same transaction it was enqueued in.
+
+    This may lead to unexpected behavior changes when changing backend in future.
+
+To use it, set :setting:`BACKEND <TASKS-BACKEND>` to
+``"django.tasks.backends.immediate.ImmediateBackend"``::
+
+    TASKS = {"default": {"BACKEND": "django.tasks.backends.immediate.ImmediateBackend"}}
+
+.. _dummy-task-backend:
+
+Dummy backend
+-------------
+
+The dummy backend doesn't execute enqueued tasks at all, instead storing results
+for later use.
+
+This backend is not intended for use in production - it is provided as a
+convenience that can be used during development and testing.
+
+To use it, set :setting:`BACKEND <TASKS-BACKEND>` to
+``"django.tasks.backends.dummy.DummyBackend"``::
+
+    TASKS = {"default": {"BACKEND": "django.tasks.backends.dummy.DummyBackend"}}
+
+The results for enqueued tasks can be retrieved from the backend's
+:attr:`results <django.tasks.backends.dummy.DummyBackend.results>` attribute::
+
+    from django.tasks import default_task_backend
+
+    my_task.enqueue()
+
+    assert len(default_task_backend.results) == 1
+
+Stored results can be cleared using the
+:meth:`clear <django.tasks.backends.dummy.DummyBackend.clear>` method::
+
+    default_task_backend.clear()
+
+    assert len(default_task_backend.results) == 0
+
+Using a custom backend
+----------------------
+
+While Django includes support for a number of task backends out-of-the-box,
+sometimes you might want to customize the the task backend. To use an external
+task backend with Django, use the Python import path as the
+:setting:`BACKEND <TASKS-BACKEND>` of the :setting:`TASKS` setting, like so::
+
+    TASKS = {
+        "default": {
+            "BACKEND": "path.to.backend",
+        }
+    }
+
+If you're building your own backend, you can use the standard task backends
+as reference implementations. You'll find the code in the
+:source:`django/tasks/backends/` directory of the Django source.
+
+Backend arguments
+-----------------
+
+Each backend can be given additional arguments to control its behavior.
+These arguments are provided as additional keys in the :setting:`TASKS` setting.
+Valid arguments are as follows:
+
+* :setting:`ENQUEUE_ON_COMMIT <TASKS-ENQUEUE_ON_COMMIT>`: Whether a task should
+  be enqueued at the end of the current transaction (if there
+  is one) commits successfully, rather than enqueueing immediately. This argument
+  defaults to ``True``.
+
+* :setting:`QUEUES <TASKS-QUEUES>`: Restrict the queue names a task may be
+  enqueued to. By default, only the ``"default"`` queue is allowed. Queue name
+  validation can be disabled by setting this to an empty list.
+
+Asynchronous support
+--------------------
+
+Django has developing support for asynchronous task backends.
+
+``django.tasks.backends.base.BaseTaskBackend`` has async variants of all base
+methods. By convention, the asynchronous versions of all methods are prefixed
+with ``a``. The arguments for both variants are the same.