python-policy.sgml

<!doctype debiandoc system>

<debiandoc>
  <book>
    <titlepag>
      <title>Debian Python Policy</title>
      <author>
	<name>Neil Schemenauer</name>
	<email>nas@debian.org</email>
      </author>
      <author>
	<name>Matthias Klose</name>
	<email>doko@debian.org</email>
      </author>
      <author>
	<name>Gregor Hoffleit</name>
	<email>flight@debian.org</email>
      </author>
      <author>
        <name>Josselin Mouette</name>
	<email>joss@debian.org</email>
      </author>
      <author>
        <name>Joe Wreschnig</name>
	<email>piman@debian.org</email>
      </author>
      <version>version 0.4.1.0</version>

      <abstract>
	This document describes the packaging of Python within the
	Debian GNU/Linux distribution and the policy requirements for
	packaged Python programs and modules.
      </abstract>

      <copyright>
	<copyrightsummary>
	  Copyright &copy; 1999, 2001, 2003, 2006 Software in the
	  Public Interest
	</copyrightsummary>
	<p>
	  This manual is free software; you can redistribute it and/or
	  modify it under the terms of the GNU General Public License
	  as published by the Free Software Foundation; either version
	  2 of the License, or (at your option) any later version.
	</p>
	<p>
	  This is distributed in the hope that it will be useful, but
	  WITHOUT ANY WARRANTY; without even the implied warranty of
	  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
	  the GNU General Public License for more details.
	</p>
	<p>
	  A copy of the GNU General Public License is available as
	  <tt>/usr/share/common-licences/GPL</tt> in the Debian GNU/Linux
	  distribution or on the World Wide Web at
	  <url id="http://www.gnu.org/copyleft/gpl.html"
	  name="The GNU Public Licence">.
	</p>
	<p>
	  You can also obtain it by writing to the
	  Free Software Foundation, Inc., 59 Temple Place - Suite 330,
	  Boston, MA 02111-1307, USA.
	</p>
      </copyright>
    </titlepag>

    <toc detail="sect1">

    <chapt id="python">
      <heading>Python Packaging</heading>
      <sect id="versions">
	<heading>Versions</heading>
	<p>
	  At any given time, the package <package>python</package> will
	  represent the current default Debian Python version.
	</p>
	<p>
	  The default Debian Python version should alway be the latest stable
	  upstream release that can be integrated in the distribution.
	</p>
	<p>
	  Apart from the default version, legacy versions of Python
	  or beta versions of future releases
	  may be included as well in the distribution, as long as they
	  are needed by other packages, or as long as it seems
	  reasonable to provide them.  (Note: For the scope of this
	  document, Python versions are synonymous to feature
	  releases, i.e. Python 2.0 and 2.0.1 are subminor versions of
	  the same Python version 2.0, but Python 2.1 and 2.2 are
	  indeed different versions.)
	</p>
	<p>
	  For any version, the main package must be called
	  <package>python<var>X</var>.<var>Y</var></package>.
	</p>

	<p>
	  The set of currently supported python versions can be found
	  in <file>/usr/share/python/debian_defaults</file>.
	</p>

      </sect>

      <sect id="base">
	<heading>Main package</heading>
	<p>
	  For every Python version provided in the distribution, the
	  package <package>python<var>X</var>.<var>Y</var></package>
	  shall comprise a complete distribution for
	  <em>deployment</em> of Python scripts and applications. The
	  package includes the binary
	  <file>/usr/bin/python<var>X</var>.<var>Y</var></file> and
	  all modules of the upstream Python distribution.
	</p>
	<p>
	  Excluded are any modules that depend on
	  non-<em>required</em> packages, they will be provided in
	  separate packages.  Some tools and files for the
	  <em>development</em> of Python modules are split off in a
	  separate package
	  <package>python<var>X</var>.<var>Y</var>-dev</package>.
	  Documentation will be provided separately as well.
	</p>
	<p>
	  At any time, the <package>python</package> package must contain
	  a symlink <file>/usr/bin/python</file> to the the appropriate binary
	  <file>/usr/bin/python<var>X</var>.<var>Y</var></file>. The
	  <package>python</package> package must also depend on the
	  appropriate <package>python<var>X</var>.<var>Y</var></package>
	  to ensure this binary is installed. The version of the
	  <package>python</package> package must be greater than or equal to
	  <var>X</var>.<var>Y</var> and smaller than <var>X</var>.<var>Y+1</var>.
	</p>
      </sect>

      <sect id="interpreter">
        <heading>Python Interpreter</heading>
        <sect1 id="interpreter_name">
          <heading>Interpreter Name</heading>
          <p>
	    Python scripts depending on the default Python version (see <ref
	    id="base">) or not depending on a specific Python version should
	    use <file>python</file> (unversioned) as the interpreter name.
	    </p>
          <p>
	    Python scripts that only work with a specific Python version must
	    explicitly use the versioned interpreter name
	    (<file>python<var>X</var>.<var>Y</var></file>).
          </p>
        </sect1>
        <sect1 id="interpreter_loc">
          <heading>Interpreter Location</heading>
          <p>
	    The preferred specification for the Python interpreter is
            <file>/usr/bin/python</file> or
            <file>/usr/bin/python<var>X</var>.<var>Y</var></file>.
	    This ensures that a Debian installation of python is used
	    and all dependencies on additional python modules are met.
          </p>
          <p>
	    If a maintainer would like to provide the user with the
	    possibility to override the Debian Python interpreter, he
	    may want to use <file>/usr/bin/env python</file> or
	    <file>/usr/bin/env python<var>X</var>.<var>Y</var></file>.
	    However this is not advisable as it bypasses Debian's dependency
	    checking and makes the package vulnerable to incomplete local
	    installations of python.
	  </p>
        </sect1>
      </sect>

      <sect id="paths">
	<heading>Module Path</heading>
	<p>
	  The module search path for Debian has been amended to
	  include a directory tree in /usr/local at the beginning of
	  the path. By default, sys.path is searched in the following
	  order:
	  <example>
/usr/lib/python<var>XY</var>.zip
/usr/lib/python<var>X</var>.<var>Y</var>
/usr/lib/python<var>X</var>.<var>Y</var>/plat-linux2
/usr/lib/python<var>X</var>.<var>Y</var>/lib-tk
/usr/lib/python<var>X</var>.<var>Y</var>/lib-dynload
/usr/local/lib/python<var>X</var>.<var>Y</var>/site-packages
/usr/lib/python<var>X</var>.<var>Y</var>/site-packages
/var/lib/python-support/python<var>X</var>.<var>Y</var>
/usr/lib/python<var>X</var>.<var>Y</var>/site-packages/<var>module-dir</var>
/usr/lib/site-python
	  </example>
	</p>
	<p>
	  The use of the <file>/usr/lib/site-python</file> directory
	  is deprecated. The directory may be dropped from the path in
	  a future version.  The /usr/lib/python<var>XY</var>.zip
	  archive appeared in python2.3; it is not currently used in
	  Debian.  Modules should not install directly to the
	  <file>/var/lib/python-support</file> directory; it is for
	  use by <ref id="pysupport">.
	</p>
      </sect>

      <sect id="docs">
	<heading>Documentation</heading>
	<p>
	  Python documentation is split out in separate packages
	  <package>python<var>X</var>.<var>Y</var>-doc</package>. The package
	  <package>python-doc</package> will always provide the documentation
	  for the default Debian Python version.
	</p>
	<p>
	  TODO: Policy for documentation of third party packages.
	</p>
      </sect>
    </chapt>

    <chapt id="module_packages">
      <heading>Packaged Modules</heading>
      <p>
	The goal of these policies is to reduce the work necessary for
	Python transitions. Python modules are internally very
	dependent on a specific Python version. However, we want to
	automate recompiling modules when possible, either during the
	upgrade itself (re-bytecompiling pyc and pyo files) or shortly
	thereafter with automated rebuilds (to handle C
	extensions). These policies encourage automated dependency
	generation and loose version bounds whenever possible.
      <sect>
	<heading>Types of Python Modules</heading>
	<p>
	  There are two kinds of Python modules, "pure" Python
	  modules, and extension modules. Pure Python modules are
	  Python source code that works across many versions of
	  Python. Extensions are C code compiled and linked against a
	  specific version of the libpython library, and so can only
	  be used by one version of Python.
	</p>
	<p>
          Python packages are directories containing at least a
          <file>__init__.py</file>, other modules, extensions and
          packages (A package in the Python sense is unrelated to a
          Debian package). Python packages must be packaged into the
          same directory (as done by upstream). Splitting components
          of a package across directories changes the import order and
          may confuse documentation tools and IDEs.
	</p>
	<p>
	  There are two ways to distribute Python modules. Public
	  modules are installed in one of the directories listed
	  in <ref id="paths">. They are accessible to any
	  program. Private modules are installed in a directory such
	  as <file>/usr/share/<var>packagename</var></file>
	  or <file>/usr/lib/<var>packagename</var></file>. They are
	  generally only accessible to a specific program or suite of
	  programs included in the same package.
	</p>
      </sect>
      <sect id="package_names">
	<heading>Module Package Names</heading>
	<p>
	  Public modules should be packaged with a name
	  of <package>python-<var>foo</var></package>,
	  where <var>foo</var> is the name of the module. Such a
	  package should support the current Debian Python version,
	  and more if possible (there are several tools to help
	  implement this, see <ref id="packaging_tools">). For
	  example, if Python 2.3, 2.4, and 2.5 are supported, the
	  Python command
	  <example>
import foo
	  </example>
	  should import the module when the user is running any
	  of <prgn>/usr/bin/python2.3</prgn>, <prgn>/usr/bin/python2.4</prgn>,
	  and <prgn>/usr/bin/python2.5</prgn>. This requirement also
	  applies to extension modules; binaries for all the supported
	  Python versions should be included in a single package.
	</p>
      </sect>
      <sect id="specifying_versions">
	<heading>Specifying Supported Versions</heading>
	<p>
	  The <tt>XS-Python-Version</tt> field
	  in <file>debian/control</file> specifies the versions of
	  Python supported by the package. This is used to track
	  packages during Python transitions, and is also used by some
	  packaging scripts to automatically generate appropriate
	  Depends and Provides lines. The format of the field may be
	  one of the following:
	  <example>
XS-Python-Version: all
XS-Python-Version: current
XS-Python-Version: current, >= X.Y
XS-Python-Version: >= X.Y
XS-Python-Version: >= A.B, << X.Y
XS-Python-Version: A.B, X.Y
	  </example>
	  Where "all" means the package supports any Python version
	  available, and "current" means it supports Debian's current
	  Python version. Explicit Versions or version ranges can also
	  be used.
	</p>
	<p>
	  Your control file should also have a line:
	  <example>
XB-Python-Version: ${python:Versions}
	  </example>
	  The python:Versions is substituted by the supported Python
	  versions of the binary package, based on
	  <tt>XS-Python-Version</tt>. (If you are not using
	  <prgn>dh_python</prgn> you will need to handle this
	  substitution yourself.) The format of the field
	  <tt>XB-Python-Version</tt> is the same as the
	  <tt>XS-Python-Version</tt> field for packages not containing
	  extensions. Packages with extensions must list the versions
	  explicitely.
	</p>
	<p>
	  If your package is used by another module or application
	  that requires a specific Python version, it should also
	  <tt>Provide: python<var>X</var>.<var>Y</var>-foo</tt> for
	  each version it supports.
	</p>
      </sect>

      <sect id="dependencies">
	<heading>Dependencies</heading>
	<p>
	  Packaged modules available for the default Python version
	  (or many versions including the default) as described
	  in <ref id="package_names"> must depend on "<package>python
	  (&gt;=&nbsp;<var>X</var>.<var>Y</var></package>)". If they
	  require other modules to work, they must depend on the
	  corresponding <package>python-foo</package>. They must not
	  depend on
	  any <package>python<var>X</var>.<var>Y</var>-foo</package>.
	</p>
	<p>
	  Packaged modules available for one particular version of Python must
	  depend on the corresponding
	  <package>python<var>X</var>.<var>Y</var></package> package instead.
	  If they need other modules, they must depend on the corresponding
	  <package>python<var>X</var>.<var>Y</var>-foo</package> packages, and
	  must not depend on any <package>python-foo</package>.
	</p>
      </sect>

      <sect id="provides">
	<heading>Provides</heading>
	<p>
          Provides in packages of the form
          <package>python-<var>foo</var></package> must be specified,
          if the package contains an extension for more than one
          python version. Provides should also be added on request of
          maintainers who depend on a non-default python version.
	</p>
	<p>
	  Packaged modules available for one particular version of Python must
	  depend on the corresponding
	  <package>python<var>X</var>.<var>Y</var></package> package instead.
	  If they need other modules, they must depend on the corresponding
	  <package>python<var>X</var>.<var>Y</var>-foo</package> packages, and
	  must not depend on any <package>python-foo</package>.
	</p>
      </sect>

      <sect id="bytecompilation">
        <heading>Modules Bytecompilation</heading>
	<p>
	  If a package provides any binary-independent modules
	  (<file>foo.py</file> files), the corresponding bytecompiled
	  modules (<file>foo.pyc</file> files) and optimized modules
	  (<file>foo.pyo</file> files) must not ship in the
	  package. Instead, they should be generated in the package's
	  postinst, and removed in the package's prerm. The package's
	  prerm has to make sure that both <file>foo.pyc</file> and
	  <file>foo.pyo</file> are removed.
	</p>
	<p>
          A package should only byte-compile the files which belong to
          the package.
	</p>
	<p>
          The file <file>/etc/python/debian_config</file> allows
          configuration how modules should be byte-compiled. The
          postinst scripts should respect these settings.
	</p>
	<p>
          Modules in private installation directories and in
          <file>/usr/lib/site-python</file> should be byte-compiled,
          when the default python version changes.
	</p>
      </sect>
    </chapt>

    <chapt id="programs">
      <heading>Python Programs</heading>

      <sect id="version_indep_progs">
	<heading>Programs using the default python</heading>
	<p>
	  Programs that can run with any version of Python must
	  begin with <tt>#!/usr/bin/python</tt> or <tt>#!/usr/bin/env
	  python</tt> (the former is preferred). They must also
	  specify a dependency on <package>python</package>, with a
	  versioned dependency if necessary.
	</p>
	<p>
	  If the program needs the python module <tt>foo</tt>,
	  it must depend on <package>python-foo</package>.
	</p>

        <sect1 id="current_version_progs">
          <heading>Programs Shipping Private Modules</heading>
	  <p>
	    A program using <file>/usr/bin/python</file> as
	    interpreter can come up with private Python modules. These
	    modules should be installed in
	    <tt>/usr/share/<var>module</var></tt>, or
	    <tt>/usr/lib/<var>module</var></tt> if the modules are
	    architecture-dependent (e.g. extensions).
	  </p>
	  <p>
	    <file>/usr/lib/site-python</file> is deprecated and should
	    no longer be used for this purpose.
	  </p>
	  <p>
	    The rules explained in <ref id="bytecompilation"> apply to
	    those private modules: the bytecompiled modules must not
	    be shipped with the package, they should be generated in
	    the package's postinst, using the current default Python
	    version, and removed in the prerm. Modules should be
	    bytecompiled using the current default Python version.
	  </p>
	  <p>
	    Programs that have private compiled extensions must either
	    handle multiple version support themselves, or declare a
	    tight dependency on the current Python version
	    (e.g. <tt>Depends: python (>= 2.4), python (<= 2.5)</tt>. No 
	    tools currently exist to alleviate this situation.
	  </p>
	</sect1>
      </sect>

      <sect id="version_dep_progs">
	<heading>Programs Using a Particular Python Version</heading>
	<p>
	  A program which requires a specific version of Python must
	  begin with
	  <tt>#!/usr/bin/python<var>X</var>.<var>Y</var></tt> (or
	  <tt>#!/usr/bin/env python<var>X</var>.<var>Y</var></tt>). It
	  must also specify a dependency on
	  <package>python<var>X</var>.<var>Y</var></package> and on
	  any <package>python<var>X</var>.<var>Y</var>-foo</package>
	  package providing necessary modules. It should not depend on
	  any <package>python-foo</package> package, unless it
	  requires a specific version of the package (since virtual
	  packages cannot be versioned). If this is the case, it
	  should depend on both the virtual package and the main
	  package (e.g. <tt>Depends: python2.4-foo, python-foo (>=
	  1.0)</tt>).
	</p>
	<p>
	  The notes on installation directories and bytecompilation
	  for programs that support any version of Python also apply
	  to programs supporting only a single Python version. Modules
	  to be bytecompiled should use the same Python version as the
	  package itself.
	</p>
      </sect>
    </chapt>

    <chapt id="embed">
      <heading>Programs Embedding Python</heading>

      <sect id="build_embedded">
	<heading>Building Embedded Programs</heading>
	<p>
	  Programs which embed a Python interpreter must declare a
	  <tt>Build-Depends</tt> on
	  <package>python<var>X</var>.<var>Y</var>-dev</package>, where
	  python<var>X</var>.<var>Y</var> is the python version the program
	  builds against. It should be the current default python version
	  unless the program doesn't work correctly with this version.
	</p>
      </sect>

      <sect id="embedded_deps">
	<heading>Embedded Python Dependencies</heading>
	<p>
	  Dependencies for programs linking against the shared Python
	  library will be automatically created by
	  <prgn>dpkg-shlibdeps</prgn>. The
	  <tt>libpython<var>X</var>.<var>Y</var>.so.<var>Z</var></tt> library
	  the program is built against is provided by the
	  <package>python<var>X</var>.<var>Y</var></package> package.
	</p>
      </sect>
    </chapt>

    <chapt id="other">
      <heading>Interaction with Locally Installed Python Versions</heading>
      <p>
	As long as you don't install other versions of Python in your
	path, Debian's Python versions won't be affected by a new
	version.
      </p>
      <p>
	If you install a different subrelease of the version of python
	you've got installed, you'll need to be careful to install all
	the modules you use for that version of python too.
      </p>

    </chapt>

    <appendix id="build_dependencies">
      <heading>Build Dependencies</heading>
      <p>
	Build dependencies for Python dependent packages must be
	declared for every Python version that the package is built
	for. The <package>python-all-dev</package> should be used when
	building modules for any or all Python versions. To build for
	a specific version or versions, Build-Depend on
	<package>python<var>X</var>.<var>Y</var>-dev</package>.
      </p>
      <p>
	Some applications and pure Python modules may be able to
	depend only on <package>python</package>
	or <package>python-all</package> and not require the -dev
	packages.
      </p>

      <p>
	Build-Depend on at least:
	<example>
Build-Depends: python2.3 (>= 2.3-1)
Build-Depends: python2.4 (>= 2.4-1)
Build-Depends: python (>= 2.3.5-7)
Build-Depends: python-all

Build-Depends: python2.3-dev (>= 2.3-1)
Build-Depends: python2.4-dev (>= 2.4-1)
Build-Depends: python-dev (>= 2.3.5-7)
Build-Depends: python-all-dev
	</example>
      </p>
      <p>
	If you use either <package>python-support</package> or
	<package>python-central</package> you must additionally
	Build-Depend on those. If you are using <prgn>dh_python</prgn>
	at all, you must Build-Depend on <package>python</package>, as
	<package>debhelper</package> does not depend on it.
      </p>
    </appendix>

    <appendix id="packaging_tools">
      <heading>Packaging Tools</heading>
      <p>
	This section describes the various tools to help package
	Python programs and modules for Debian. Although none of these
	tools are mandatory, their use is strongly encouraged, as the
	above policy has been designed with them in mind (and vice
	versa). This appendix is just an overview. If you use these
	tools, you should read their full documentation.
      </p>
      <sect id="pysupport">
	<heading>python-support</heading>
	<p>
	  The python-support system provides a simple way to
	  bytecompile pure Python modules and manage dependencies. It
	  integrates with <package>debhelper</package>. When using
	  python-support, you should install your modules
	  to <file>/usr/share/python-support/<var>package</var></file>
	  rather than the standard Python directories. python-support
	  will then handle compiling the modules and making
	  appropriate symbolic links for installed Python versions to
	  find them,
	  substitute <tt>${python:Depends}</tt>, <tt>${python:Versions}</tt>,
	  and <tt>${python:Provides}</tt> in your control file, and
	  manage bytecompilation in your postinst/prerm.
	</p>
	<p>
	  To use it, call <prgn>dh_pysupport</prgn>
	  before <prgn>dh_python</prgn>, and make sure you've
	  installed the modules in the right place:
	  <example>
PREFIX := debian/python-package/usr
...
install:
	...
	./setup.py install --no-compile \
		--install-lib=$(PREFIX)/share/python-support/python-package
binary-indep: build install
	...
	dh_pysupport
	dh_python
	...
	  </example>
	</p>
	<p>
	  python-support can also manage private modules. To use this
	  feature, pass a list of directories to be managed by
	  python-support to <prgn>dh_pysupport</prgn>
	  and <prgn>dh_python</prgn>. python-support cannot handle
	  compiled extensions.
	</p>
      </sect>

      <sect id="pycentral">
	<heading>python-central</heading>
	<p>
	  python-central provides another way to manage Python
	  modules. It integrates with <package>debhelper</package>,
	  but can also be used without it. When using python-central,
	  you should install your modules normally. It will then move
	  them to its private directory, and manage the same things
	  python-support does.
	</p>
	<p>
	  To use it, call <prgn>dh_pycentral</prgn>
	  before <prgn>dh_python</prgn>:
	  <example>
install:
	...
	./setup.py install

binary-indep: build install
	...
	dh_pycentral
	dh_python
	...
	  </example>
	</p>
	<p>
	  python-central can handle compiled extensions for multiple
	  Python versions. If you want python-central to handle
	  private modules, you must pass the list of directories
	  containing them to <prgn>dh_python</prgn> (but
	  not <prgn>dh_pycentral</prgn>).
	</p>
	<p>
          If python-central should not move the files to its private
          directory, use<prgn>DH_PYCENTRAL=nomove dh_pycentral</prgn>
          instead.
	</p>
	<p>
          Examples for source packages using python-central are
          pyenchant, python-imaging (modules and extensions),
          pyparallel (modules only).
	</p>
      </sect>

      <sect id="cdbs">
	<heading>CDBS</heading>
	<p>
	  FIXME: Someone familiar with CDBS should write this part.
	</p>
      </sect>

    </appendix>

    <appendix id="upgrade">
      <heading>Upgrade Procedure</heading>
      <p>
	This section describes the procedure for the upgrade when the
	default python version is changed in the <tt>unstable</tt>
	distribution, requiring recompilation of many python-related
	packages.
      </p>
      <p>
	<enumlist>
	  <item>
	    <p>
	      Have a long and heated discussion.
	    </p>
	  </item>
	  <item>
	    <p>
	      The Debian Python maintainer decides for the new default Debian
	      Python version and announces the upgrade.
	    </p>
	  </item>
	  <item>
	    <p>
	      Upload of the python core metapackages <package>python</package>,
	      <package>python-dev</package>, <package>python-doc</package> and
	      several <package>python-<var>module</var></package>, depending on
	      the new <package>python<var>X</var>.<var>Y</var></package>,
	      <package>python<var>X</var>.<var>Y</var>-dev</package> and so on.
	    </p>
	  </item>
	  <item>
	    <p>
	      The release team schedules rebuilds for packages that
	      may need it. Packages that require manual work get
	      updated and uploaded.
	    </p>
	  </item>
	</enumlist>
      </p>
    </appendix>
  </book>
</debiandoc>