diff --git a/docs/index.txt b/docs/index.txt index cf907df335..03ccb3b493 100644 --- a/docs/index.txt +++ b/docs/index.txt @@ -73,7 +73,9 @@ manipulating the data of your Web application. Learn more about it below: :doc:`Accessing related objects ` * **Migrations:** - :doc:`Introduction to Migrations` + :doc:`Introduction to Migrations` | + :doc:`Operations reference ` | + :doc:`SchemaEditor ` * **Advanced:** :doc:`Managers ` | diff --git a/docs/ref/migration-operations.txt b/docs/ref/migration-operations.txt new file mode 100644 index 0000000000..70406adc22 --- /dev/null +++ b/docs/ref/migration-operations.txt @@ -0,0 +1,320 @@ +==================== +Migration Operations +==================== + +Migration files are composed of one or more Operations, objects that +declaratively record what the migration should do to your database. + +Django also uses these Operation objects to work out what your models +looked like historically, and to calculate what changes you've made to +your models since the last migration so it can automatically write +your migrations; that's why they're declarative, as it means Django can +easily load them all into memory and run through them without touching +the database to work out what your project should look like. + +There are also more specialised Operation objects which are for things like +:ref:`data migrations ` and for advanced manual database +manipulation. You can also write your own Operation classes if you want +to encapsulate a custom change you commonly make. + +If you need an empty migration file to write your own Operation objects +into, just use ``python manage.py makemigrations --empty yourappname``, +but be aware that manually adding schema-altering operations can confuse the +migration autodetector and make resulting runs of ``makemigrations`` output +incorrect code. + +All of the core Django operations are available from the +``django.db.migrations.operations`` module. + + +Schema Operations +================= + +CreateModel +----------- +:: + + CreateModel(name, fields, options=None, bases=None) + +Creates a new model in the project history and a corresponding table in the +database to match it. + +``name`` is the model name, as would be written in the ``models.py`` file. + +``fields`` is a list of 2-tuples of ``(field_name, field_instance)``. +The field instance should be an unbound field (so just ``models.CharField()``, +rather than a field takes from another model). + +``options`` is an optional dictionary of values from the model's ``Meta`` class. + +``bases`` is an optional list of other classes to have this model inheirit from; +it can contain both class objects as well as strings in the format +``"appname.ModelName"`` if you want to depend on another model (so you inherit +from the historical version). If it's not supplied, it defaults to just +inheriting from the standard ``models.Model``. + + +DeleteModel +----------- +:: + + DeleteModel(name) + +Deletes the model from the project history and its table from the database. + + +RenameModel +----------- +:: + + RenameModel(old_name, new_name) + +Renames the model from an old name to a new one. + +You may have to manually add +this if you change the model's name and quite a few of its fields at once; to +the autodetector, this will look like you deleted a model with the old name +and added a new one with a different name, and the migration it creates will +lose any data in the old table. + + +AlterModelTable +--------------- +:: + + AlterModelTable(name, table) + +Changes the model's table name (the ``db_table`` option on the ``Meta`` subclass) + + +AlterUniqueTogether +------------------- +:: + + AlterUniqueTogether(name, unique_together) + +Changes the model's set of unique constraints +(the ``unique_together`` option on the ``Meta`` subclass) + + +AlterIndexTogether +------------------ +:: + + AlterIndexTogether(name, index_together) + +Changes the model's set of custom indexes +(the ``index_together`` option on the ``Meta`` subclass) + + +AddField +-------- +:: + + AddField(model_name, name, field, preserve_default=True) + +Adds a field to a model. ``model_name`` is the model's name, ``name`` is +the field's name, and ``field`` is an unbound Field instance (the thing +you would put in the field declaration in ``models.py`` - for example, +``models.IntegerField(null=True)``. + +The ``preserve_default`` argument indicates whether the field's default +value is permanent and should be baked into the project state (``True``), +or if it is temporary and just for this migration (``False``) - usually +because the migration is adding a non-nullable field to a table and needs +a default value to put into existing rows. It does not effect the behaviour +of setting defaults in the database directly - Django never sets database +defaults, and always applies them in the Django ORM code. + + +RemoveField +----------- +:: + + RemoveField(model_name, name) + +Removes a field from a model. + +Bear in mind that when reversed this is actually adding a field to a model; +if the field is not nullable this may make this operation irreversible (apart +from any data loss, which of course is irreversible). + + +AlterField +---------- +:: + + AlterField(model_name, name, field) + +Alters a field's definition, including changes to its type, ``null``, ``unique``, +``db_column`` and other field attributes. + +Note that not all changes are possible on all databases - for example, you +cannot change a text-type field like ``models.TextField()`` into a number-type +field like ``models.IntegerField()`` on most databases. + + +RenameField +----------- +:: + + RenameField(model_name, old_name, new_name) + +Changes a field's name (and, unless ``db_column`` is set, its column name). + + + +Special Operations +================== + +RunSQL +------ + +:: + + RunSQL(sql, reverse_sql=None, state_operations=None, multiple=False) + +Allows runnning of arbitrary SQL on the database - useful for more advanced +features of database backends that Django doesn't support directly, like +partial indexes. + +``sql``, and ``reverse_sql`` if provided, should be strings of SQL to run on the +database. They will be passed to the database as a single SQL statement unless +``multiple`` is set to ``True``, in which case they will be split into separate +statements manually by the operation before being passed through. + +In some extreme cases, the built-in statement splitter may not be able to split +correctly, in which case you should manually split the SQL into multiple calls +to ``RunSQL``. + +The ``state_operations`` argument is so you can supply operations that are +equivalent to the SQL in terms of project state; for example, if you are +manually creating a column, you should pass in a list containing an ``AddField`` +operation here so that the autodetector still has an up-to-date state of the +model (otherwise, when you next run ``makemigrations``, it won't see any +operation that adds that field and so will try to run it again). + + +.. _operation-run-python: + +RunPython +--------- + +:: + + RunPython(code, reverse_code=None) + +Runs custom Python code in a historical context. ``code`` (and ``reverse_code`` +if supplied) should be callable objects that accept two arguments; the first is +an instance of ``django.apps.registry.Apps`` containing historical models that +match the operation's place in the project history, and the second is an +instance of SchemaEditor. + +You are advised to write the code as a separate function above the ``Migration`` +class in the migration file, and just pass it to ``RunPython``. + + +SeparateDatabaseAndState +------------------------ + +:: + + SeparateDatabaseAndState(database_operations=None, state_operations=None) + +A highly specalist operation that let you mix and match the database +(schema-changing) and state (autodetector-powering) aspects of operations. + +It accepts two list of operations, and when asked to apply state will use the +state list, and when asked to apply changes to the database will use the database +list. Do not use this operation unless you're very sure you know what you're doing. + + +Writing your own +================ + +Operations have a relatively simple API, and they're designed so that you can +easily write your own to supplement the built-in Django ones. The basic structure +of an Operation looks like this:: + + from django.db.migrations.operations.base import Operation + + class MyCustomOperation(Operation): + + # If this is False, it means that this operation will be ignored by + # sqlmigrate; if true, it will be run and the SQL collected for its output. + reduces_to_sql = False + + # If this is False, Django will refuse to reverse past this operation. + reversible = False + + def __init__(self, arg1, arg2): + # Operations are usually instantiated with arguments in migration + # files. Store the values of them on self for later use. + pass + + def state_forwards(self, app_label, state): + # The Operation should take the 'state' parameter (an instance of + # django.db.migrations.state.ProjectState) and mutate it to match + # any schema changes that have occurred. + pass + + def database_forwards(self, app_label, schema_editor, from_state, to_state): + # The Operation should use schema_editor to apply any changes it + # wants to make to the database. + pass + + def database_backwards(self, app_label, schema_editor, from_state, to_state): + # If reversible is True, this is called when the operation is reversed. + pass + + def describe(self): + # This is used to describe what the operation does in console output. + return "Custom Operation" + +You can take this template and work from it, though we suggest looking at the +built-in Django operations in ``django.db.migrations.operations`` - they're +easy to read and cover a lot of the example usage of semi-internal aspects +of the migration framework like ``ProjectState`` and the patterns used to get +historical models. + +Some things to note: + +* You don't need to learn too much about ProjectState to just write simple + migrations; just know that it has a ``.render()`` method that turns it into + an app registry (which you can then call ``get_model`` on). + +* ``database_forwards`` and ``database_backwards`` both get two states passed + to them; these just represent the difference the ``state_forwards`` method + would have applied, but are given to you for convenience and speed reasons. + +* ``to_state`` in the database_backwards method is the *older* state; that is, + the one that will be the current state once the migration has finished reversing. + +* You might see implementations of ``references_model`` on the built-in + operations; this is part of the autodetection code and does not matter for + custom operations. + +As a simple example, let's make an operation that loads PostgreSQL extensions +(which contain some of PostgreSQL's more exciting features). It's simple enough; +there's no model state changes, and all it does is run one command:: + + from django.db.migrations.operations.base import Operation + + class LoadExtension(Operation): + + reversible = True + + def __init__(self, name): + self.name = name + + def state_forwards(self, app_label, state): + pass + + def database_forwards(self, app_label, schema_editor, from_state, to_state): + schema_editor.execute("CREATE EXTENSION IF NOT EXISTS %s" % self.name) + + def database_backwards(self, app_label, schema_editor, from_state, to_state): + schema_editor.execute("DROP EXTENSION %s" % self.name) + + def describe(self): + return "Creates extension %s" % self.name diff --git a/docs/ref/schema-editor.txt b/docs/ref/schema-editor.txt new file mode 100644 index 0000000000..5cff1de6ac --- /dev/null +++ b/docs/ref/schema-editor.txt @@ -0,0 +1,112 @@ +============ +SchemaEditor +============ + +Django's migration system is split into two parts; the logic for calculating +and storing what operations should be run (``django.db.migrations``), and the +database abstraction layer that turns things like "create a model" or +"delete a field" into SQL - which is the job of the ``SchemaEditor``. + +It's unlikely that you will want to interact directly with ``SchemaEditor`` as +a normal developer using Django, but if you want to write your own migration +system, or have more advanced needs, it's a lot nicer than writing SQL. + +Each database backend in Django supplies its own version of ``SchemaEditor``, +and it's always accessible via the ``connection.schema_editor()`` context +manager:: + + with connection.schema_editor() as schema_editor: + schema_editor.delete_model(MyModel) + +It must be used via the context manager as this allows it to manage things +like transactions and deferred SQL (like creating ``ForeignKey`` constraints). + +It exposes all possible operations as methods, that should be called in +the order you wish changes to be applied. Some possible operations or types +of change are not possible on all databases - for example, MyISAM does not +support foreign key constraints. + +Methods +======= + +execute +------- + +:: + + execute(sql, params=[]) + +Executes the SQL statement passed in, with parameters if supplied. This +is a simple wrapper around the normal database cursors that allows +capture of the SQL to a ``.sql`` file if the user wishes. + +create_model +------------ + +:: + + create_model(model) + + +delete_model +------------ + +:: + + delete_model(model) + + +alter_unique_together +--------------------- + +:: + + alter_unique_together(model, old_unique_together, new_unique_together) + + +alter_index_together +-------------------- + +:: + + alter_index_together(model, old_index_together, new_index_together) + + +alter_db_table +-------------- + +:: + + alter_db_table(model, old_db_table, new_db_table) + + +alter_db_tablespace +------------------- + +:: + + alter_db_tablespace(model, old_db_tablespace, new_db_tablespace) + + +add_field +--------- + +:: + + add_field(model, field) + + +remove_field +------------ + +:: + + remove_field(model, field) + + +alter_field +------------ + +:: + + alter_field(model, old_field, new_field, strict=False) diff --git a/docs/topics/migrations.txt b/docs/topics/migrations.txt index f8500f2f2d..45fdc0bef0 100644 --- a/docs/topics/migrations.txt +++ b/docs/topics/migrations.txt @@ -60,7 +60,7 @@ Backend Support Migrations are supported on all backends that Django ships with, as well as any third-party backends if they have programmed in support for schema -alteration (done via the ``SchemaEditor`` class). +alteration (done via the :doc:`SchemaEditor ` class). However, some databases are more capable than others when it comes to schema migrations; some of the caveats are covered below. @@ -311,6 +311,84 @@ from these base classes inherit normally, so if you absolutely need access to these you can opt to move them into a superclass. +.. _data-migrations: + +Data Migrations +--------------- + +As well as changing the database schema, you can also use migrations to change +the data in the database itself, in conjunction with the schema if you want. + +Migrations that alter data are usually called "data migrations"; they're best +written as separate migrations, sitting alongside your schema migrations. + +Django can't automatically generate data migrations for you, as it does with +schema migrations, but it's not very hard to write them. Migration files in +Django are made up of :doc:`Operations `, and +the main operation you use for data migrations is +:ref:`RunPython `. + +To start, make an empty migration file you can work from (Django will put +the file in the right place, suggest a name, and add dependencies for you):: + + python manage.py makemigrations --empty yourappname + +Then, open up the file; it should look something like this:: + + # encoding: utf8 + from django.db import models, migrations + + class Migration(migrations.Migration): + + dependencies = [ + ('yourappname', '0001_initial'), + ] + + operations = [ + ] + +Now, all you need to do is create a new function and have RunPython use it. +RunPython expects a callable as its argument which takes two arguments - the +first is an :doc:`app registry ` that has the historical +versions of all your models loaded into it to match where in your history the +migration sits, and the second is a :doc:`SchemaEditor `, +which you can use to manually effect database schema changes (but beware, +doing this can confuse the migration autodetector!) + +Let's write a simple migration that populates our new ``name`` field with the +combined values of ``first_name`` and ``last_name`` (we've come to our senses +and realised that not everyone has first and last names). All we +need to do is use the historical model and iterate over the rows:: + + # encoding: utf8 + from django.db import models, migrations + + def combine_names(apps, schema_editor): + # We can't import the Person model directly as it may be a newer + # version than this migration expects. We use the historical version. + Person = apps.get_model("yourappname", "Person") + for person in Person.objects.all(): + person.name = "%s %s" % (person.first_name, person.last_name) + person.save() + + class Migration(migrations.Migration): + + dependencies = [ + ('yourappname', '0001_initial'), + ] + + operations = [ + migrations.RunPython(combine_names), + ] + +Once that's done, we can just run ``python manage.py migrate`` as normal and +the data migration will run in place alongside other migrations. + +If you're interested in the more advanced migration operations, or want +to be able to write your own, see our +:doc:`migration operations reference `. + + .. _migration-serializing: Serializing values