mirror of
https://github.com/django/django.git
synced 2024-12-27 19:46:22 +00:00
4a954cfd11
This patch does not remove all occurrences of the words in question. Rather, I went through all of the occurrences of the words listed below, and judged if they a) suggested the reader had some kind of knowledge/experience, and b) if they added anything of value (including tone of voice, etc). I left most of the words alone. I looked at the following words: - simply/simple - easy/easier/easiest - obvious - just - merely - straightforward - ridiculous Thanks to Carlton Gibson for guidance on how to approach this issue, and to Tim Bell for providing the idea. But the enormous lion's share of thanks go to Adam Johnson for his patient and helpful review.
219 lines
9.8 KiB
Plaintext
219 lines
9.8 KiB
Plaintext
====================================
|
|
``LayerMapping`` data import utility
|
|
====================================
|
|
|
|
.. module:: django.contrib.gis.utils.layermapping
|
|
:synopsis: Spatial data import utility for GeoDjango models.
|
|
|
|
.. currentmodule:: django.contrib.gis.utils
|
|
|
|
The :class:`LayerMapping` class provides a way to map the contents of
|
|
vector spatial data files (e.g. shapefiles) into GeoDjango models.
|
|
|
|
This utility grew out of the author's personal needs to eliminate
|
|
the code repetition that went into pulling geometries and fields out of
|
|
a vector layer, converting to another coordinate system (e.g. WGS84), and
|
|
then inserting into a GeoDjango model.
|
|
|
|
.. note::
|
|
|
|
Use of :class:`LayerMapping` requires GDAL.
|
|
|
|
.. warning ::
|
|
|
|
GIS data sources, like shapefiles, may be very large. If you find
|
|
that :class:`LayerMapping` is using too much memory, set
|
|
:setting:`DEBUG` to ``False`` in your settings. When :setting:`DEBUG`
|
|
is set to ``True``, Django :ref:`automatically logs <faq-see-raw-sql-queries>`
|
|
*every* SQL query -- and when SQL statements contain geometries, this may
|
|
consume more memory than is typical.
|
|
|
|
Example
|
|
=======
|
|
|
|
#. You need a GDAL-supported data source, like a shapefile (here we're using
|
|
a simple polygon shapefile, ``test_poly.shp``, with three features)::
|
|
|
|
>>> from django.contrib.gis.gdal import DataSource
|
|
>>> ds = DataSource('test_poly.shp')
|
|
>>> layer = ds[0]
|
|
>>> print(layer.fields) # Exploring the fields in the layer, we only want the 'str' field.
|
|
['float', 'int', 'str']
|
|
>>> print(len(layer)) # getting the number of features in the layer (should be 3)
|
|
3
|
|
>>> print(layer.geom_type) # Should be 'Polygon'
|
|
Polygon
|
|
>>> print(layer.srs) # WGS84 in WKT
|
|
GEOGCS["GCS_WGS_1984",
|
|
DATUM["WGS_1984",
|
|
SPHEROID["WGS_1984",6378137,298.257223563]],
|
|
PRIMEM["Greenwich",0],
|
|
UNIT["Degree",0.017453292519943295]]
|
|
|
|
#. Now we define our corresponding Django model (make sure to use :djadmin:`migrate`)::
|
|
|
|
from django.contrib.gis.db import models
|
|
|
|
class TestGeo(models.Model):
|
|
name = models.CharField(max_length=25) # corresponds to the 'str' field
|
|
poly = models.PolygonField(srid=4269) # we want our model in a different SRID
|
|
|
|
def __str__(self):
|
|
return 'Name: %s' % self.name
|
|
|
|
#. Use :class:`LayerMapping` to extract all the features and place them in the
|
|
database::
|
|
|
|
>>> from django.contrib.gis.utils import LayerMapping
|
|
>>> from geoapp.models import TestGeo
|
|
>>> mapping = {'name' : 'str', # The 'name' model field maps to the 'str' layer field.
|
|
'poly' : 'POLYGON', # For geometry fields use OGC name.
|
|
} # The mapping is a dictionary
|
|
>>> lm = LayerMapping(TestGeo, 'test_poly.shp', mapping)
|
|
>>> lm.save(verbose=True) # Save the layermap, imports the data.
|
|
Saved: Name: 1
|
|
Saved: Name: 2
|
|
Saved: Name: 3
|
|
|
|
Here, :class:`LayerMapping` transformed the three geometries from the shapefile
|
|
in their original spatial reference system (WGS84) to the spatial reference
|
|
system of the GeoDjango model (NAD83). If no spatial reference system is
|
|
defined for the layer, use the ``source_srs`` keyword with a
|
|
:class:`~django.contrib.gis.gdal.SpatialReference` object to specify one.
|
|
|
|
``LayerMapping`` API
|
|
====================
|
|
|
|
.. class:: LayerMapping(model, data_source, mapping, layer=0, source_srs=None, encoding=None, transaction_mode='commit_on_success', transform=True, unique=True, using='default')
|
|
|
|
The following are the arguments and keywords that may be used during
|
|
instantiation of ``LayerMapping`` objects.
|
|
|
|
================= =========================================================
|
|
Argument Description
|
|
================= =========================================================
|
|
``model`` The geographic model, *not* an instance.
|
|
|
|
``data_source`` The path to the OGR-supported data source file
|
|
(e.g., a shapefile). Also accepts
|
|
:class:`django.contrib.gis.gdal.DataSource` instances.
|
|
|
|
``mapping`` A dictionary: keys are strings corresponding to
|
|
the model field, and values correspond to
|
|
string field names for the OGR feature, or if the
|
|
model field is a geographic then it should
|
|
correspond to the OGR geometry type,
|
|
e.g., ``'POINT'``, ``'LINESTRING'``, ``'POLYGON'``.
|
|
================= =========================================================
|
|
|
|
===================== =====================================================
|
|
Keyword Arguments
|
|
===================== =====================================================
|
|
``layer`` The index of the layer to use from the Data Source
|
|
(defaults to 0)
|
|
|
|
``source_srs`` Use this to specify the source SRS manually (for
|
|
example, some shapefiles don't come with a ``'.prj'``
|
|
file). An integer SRID, WKT or PROJ.4 strings, and
|
|
:class:`django.contrib.gis.gdal.SpatialReference`
|
|
objects are accepted.
|
|
|
|
``encoding`` Specifies the character set encoding of the strings
|
|
in the OGR data source. For example, ``'latin-1'``,
|
|
``'utf-8'``, and ``'cp437'`` are all valid encoding
|
|
parameters.
|
|
|
|
``transaction_mode`` May be ``'commit_on_success'`` (default) or
|
|
``'autocommit'``.
|
|
|
|
``transform`` Setting this to False will disable coordinate
|
|
transformations. In other words, geometries will
|
|
be inserted into the database unmodified from their
|
|
original state in the data source.
|
|
|
|
``unique`` Setting this to the name, or a tuple of names,
|
|
from the given model will create models unique
|
|
only to the given name(s). Geometries from
|
|
each feature will be added into the collection
|
|
associated with the unique model. Forces
|
|
the transaction mode to be ``'autocommit'``.
|
|
|
|
``using`` Sets the database to use when importing spatial data.
|
|
Default is ``'default'``.
|
|
===================== =====================================================
|
|
|
|
``save()`` Keyword Arguments
|
|
----------------------------
|
|
|
|
.. method:: LayerMapping.save(verbose=False, fid_range=False, step=False, progress=False, silent=False, stream=sys.stdout, strict=False)
|
|
|
|
The ``save()`` method also accepts keywords. These keywords are
|
|
used for controlling output logging, error handling, and for importing
|
|
specific feature ranges.
|
|
|
|
=========================== =================================================
|
|
Save Keyword Arguments Description
|
|
=========================== =================================================
|
|
``fid_range`` May be set with a slice or tuple of
|
|
(begin, end) feature ID's to map from
|
|
the data source. In other words, this
|
|
keyword enables the user to selectively
|
|
import a subset range of features in the
|
|
geographic data source.
|
|
|
|
``progress`` When this keyword is set, status information
|
|
will be printed giving the number of features
|
|
processed and successfully saved. By default,
|
|
progress information will be printed every 1000
|
|
features processed, however, this default may
|
|
be overridden by setting this keyword with an
|
|
integer for the desired interval.
|
|
|
|
``silent`` By default, non-fatal error notifications are
|
|
printed to ``sys.stdout``, but this keyword may
|
|
be set to disable these notifications.
|
|
|
|
``step`` If set with an integer, transactions will
|
|
occur at every step interval. For example, if
|
|
``step=1000``, a commit would occur after the
|
|
1,000th feature, the 2,000th feature etc.
|
|
|
|
|
|
``stream`` Status information will be written to this file
|
|
handle. Defaults to using ``sys.stdout``, but
|
|
any object with a ``write`` method is supported.
|
|
|
|
``strict`` Execution of the model mapping will cease upon
|
|
the first error encountered. The default value
|
|
(``False``)
|
|
behavior is to attempt to continue.
|
|
|
|
``verbose`` If set, information will be printed
|
|
subsequent to each model save
|
|
executed on the database.
|
|
=========================== =================================================
|
|
|
|
Troubleshooting
|
|
===============
|
|
|
|
Running out of memory
|
|
---------------------
|
|
|
|
As noted in the warning at the top of this section, Django stores all SQL
|
|
queries when ``DEBUG=True``. Set ``DEBUG=False`` in your settings, and this
|
|
should stop excessive memory use when running ``LayerMapping`` scripts.
|
|
|
|
MySQL: ``max_allowed_packet`` error
|
|
-----------------------------------
|
|
|
|
If you encounter the following error when using ``LayerMapping`` and MySQL::
|
|
|
|
OperationalError: (1153, "Got a packet bigger than 'max_allowed_packet' bytes")
|
|
|
|
Then the solution is to increase the value of the ``max_allowed_packet``
|
|
setting in your MySQL configuration. For example, the default value may
|
|
be something low like one megabyte -- the setting may be modified in MySQL's
|
|
configuration file (``my.cnf``) in the ``[mysqld]`` section::
|
|
|
|
max_allowed_packet = 10M
|