<!doctype debiandoc system> <debiandoc> <book> <titlepag> <title>Debian Python Policy</title> <author> <name>Neil Schemenauer</name> <email>nas@debian.org</email> </author> <author> <name>Matthias Klose</name> <email>doko@debian.org</email> </author> <author> <name>Gregor Hoffleit</name> <email>flight@debian.org</email> </author> <author> <name>Josselin Mouette</name> <email>joss@debian.org</email> </author> <version>version 0.3.7.2</version> <abstract> The Python policy is currently in change; a draft for the new policy can be found <url id="http://people.debian.org/~piman/python-policy/" name="here">. This document describes the packaging of Python within the Debian GNU/Linux distribution and the policy requirements for packaged Python programs and modules. </abstract> <copyright> <copyrightsummary> Copyright © 1999, 2001, 2003 Software in the Public Interest </copyrightsummary> <p> This manual is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. </p> <p> This is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. </p> <p> A copy of the GNU General Public License is available as <tt>/usr/share/common-licences/GPL</tt> in the Debian GNU/Linux distribution or on the World Wide Web at <url id="http://www.gnu.org/copyleft/gpl.html" name="The GNU Public Licence">. </p> <p> You can also obtain it by writing to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. </p> </copyright> </titlepag> <toc detail="sect1"> <chapt id="python"> <heading>Python Packaging</heading> <sect id="versions"> <heading>Versions</heading> <p> At any given time, the package <package>python</package> will represent the current default Debian Python version. </p> <p> The default Debian Python version should alway be the latest stable upstream release that can be integrated in the distribution. </p> <p> Apart from the default version, legacy versions of Python or beta versions of future releases may be included as well in the distribution, as long as they are needed by other packages, or as long as it seems reasonable to provide them. (Note: For the scope of this document, Python versions are synonymous to feature releases, i.e. Python 2.0 and 2.0.1 are subminor versions of the same Python version 2.0, but Python 2.1 and 2.2 are indeed different versions.) </p> <p> For any version, the main package must be called <package>python<var>X</var>.<var>Y</var></package>. Names of related packages must include the <package>python<var>X</var>.<var>Y</var></package> part. </p> </sect> <sect id="base"> <heading>Main package</heading> <p> For every Python version provided in the distribution, the package <package>python<var>X</var>.<var>Y</var></package> shall comprise a complete distribution for <em>deployment</em> of Python scripts and applications. The package includes the binary <file>/usr/bin/python<var>X</var>.<var>Y</var></file> and all modules of the upstream Python distribution. </p> <p> Excluded are any modules that depend on non-<em>required</em> packages, they will be provided in separate packages. Some tools and files for the <em>development</em> of Python modules are split off in a separate package <package>python<var>X</var>.<var>Y</var>-dev</package>. Documentation will be provided separately as well. </p> <p> At any time, the <package>python</package> package must contain a symlink <file>/usr/bin/python</file> to the the appropriate binary <file>/usr/bin/python<var>X</var>.<var>Y</var></file>. The <package>python</package> package must also depend on the appropriate <package>python<var>X</var>.<var>Y</var></package> to ensure this binary is installed. The version of the <package>python</package> package must be greater than or equal to <var>X</var>.<var>Y</var> and smaller than <var>X</var>.<var>Y+1</var>. </p> </sect> <sect id="interpreter"> <heading>Python Interpreter</heading> <sect1 id="interpreter_name"> <heading>Interpreter Name</heading> <p> Python scripts depending on the default Python version (see <ref id="base">) or not depending on a specific Python version should use <file>python</file> (unversioned) as the interpreter name. </p> <p> Python scripts that only work with a specific Python version must explicitely use the versioned interpreter name (<file>python<var>X</var>.<var>Y</var></file>). </p> </sect1> <sect1 id="interpreter_loc"> <heading>Interpreter Location</heading> <p> The preferred specification for the Python interpreter is <file>/usr/bin/python</file> or <file>/usr/bin/python<var>X</var>.<var>Y</var></file>. This ensures that a Debian installation of python is used and all dependencies on additional python modules are met. </p> <p> If a maintainer would like to provide the user with the possibility to override the Debian Python interpreter, he may want to use <file>/usr/bin/env python</file> or <file>/usr/bin/env python<var>X</var>.<var>Y</var></file>. However this is not advisable as it bypasses Debian's dependency checking and makes the package vulnerable to incomplete local installations of python. </p> </sect1> </sect> <sect id="paths"> <heading>Module Path</heading> <p> The module search path for Debian has been amended to include a directory tree in /usr/local at the beginning of the path. By default, sys.path is searched in the following order: <example> /usr/lib/python<var>X</var>.<var>Y</var>.zip /usr/lib/python<var>X</var>.<var>Y</var> /usr/lib/python<var>X</var>.<var>Y</var>/plat-linux2 /usr/lib/python<var>X</var>.<var>Y</var>/lib-tk /usr/lib/python<var>X</var>.<var>Y</var>/lib-dynload /usr/local/lib/python<var>X</var>.<var>Y</var>/site-packages /usr/lib/python<var>X</var>.<var>Y</var>/site-packages /usr/lib/python<var>X</var>.<var>Y</var>/site-packages/<var>module-dir</var> /usr/lib/site-python </example> </p> <p> Note that the use of the site-python directories in Python is depreciated. The directories might be dropped from the path in a future version. </p> <p> The /usr/lib/python<var>X</var>.<var>Y</var>.zip archive appeared in python2.3. </p> <p> TODO: in fact this directory was not deprecated at all, and is widely used for shipping modules for the default python version. It has been suggested to automatically bytecompile modules in this directory for the current python version, which is technically feasible and would make it a place of choice for version-independent python modules (e.g. for modules shipped with a single script). </p> <p> TODO: What about <file>/usr/share/python<var>X</var>.<var>Y</var></file>? Wait for upstream ... see <url id="http://python.org/sf/588756" name="http://python.org/sf/588756">. </p> </sect> <sect id="docs"> <heading>Documentation</heading> <p> Python documentation is split out in separate packages <package>python<var>X</var>.<var>Y</var>-doc</package>. The package <package>python-doc</package> will always provide the documentation for the default Debian Python version. </p> <p> TODO: Policy for documentation of third party packages. </p> </sect> </chapt> <chapt id="module_packages"> <heading>Packaged Modules</heading> <sect> <heading>Rationale: A different view</heading> <p> A package with a name <package>python-<var>foo</var></package> will always provide the module <var>foo</var> for the default Debian Python version of the distribution. I.e. the package will extend the function of <file>/usr/bin/python</file> (which is installed by the package <package>python</package>). </p> <p> The system of dependencies of the default packages is robust against upgrades, but introduces a strong dependency: I.e. an upgrade of the <package>python</package> package will be hold back as long as there are still default modules packages left over on the system that can't be upgraded to the new version. </p> <p> The versioned packages (legacy versions) ensure that an upgrade to a new version can take place before upgrading *all* packages with dependencies on Python. </p> </sect> <sect id="variants"> <heading>Packaging Variants</heading> <p> There is more than one way to package a Python module: <enumlist> <item> <p> Support only the default Python version. </p> </item> <item> <p> Support a particular version, or some but not all versions of Python available in Debian. </p> </item> <item> <p> Support all/most versions of python, including the default. This variant is still not completely supported. </p> </item> </enumlist> </p> <sect1 id="default_version"> <heading>Support Only The Default Version</heading> <p> Name your package <package>python-<var>foo</var></package>. This kind of package is called a <em>default module package</em>. Install your modules into <file>/usr/lib/python<var>X</var>.<var>Y</var>/site-packages/</file>. Make your package dependency look like <example> Depends: python (>= X.Y), python (<< X.Y+1) </example> Note that this kind of packaging means that your package will trigger a conflict when the default Debian Python version in the distribution is changed, and that you will have to provide a new version as soon as possible, since the package will block the upgrade of <package>python</package>. </p> <p> You should not make a default, unversioned module package <package>python-<var>foo</var></package> depend on the versioned Python package <package>python<var>X</var>.<var>Y</var></package>! </p> <p> If the build process uses distutils and/or binary modules are built, the source package must declare: <example> Build-Depends: pythonX.Y-dev </example> where <var>X</var>.<var>Y</var> is the version used in the <tt>Depends</tt>. If the packaging process can deal with later versions of python without any changes to the packaging, it can instead declare: <example> Build-Depends: python-dev (>= X.Y) </example> When the default python version is changed, these packages still need to be rebuilt, if modules are installed in a module path specific to the python version. </p> <p> Any scripts and examples provided in the package should use <file>/usr/bin/python</file> as interpreter. </p> <p> TODO: Should a <package>python-foo</package> provide <package>python<var>X</var>.<var>Y</var>-foo</package>, provided that the Debian policy allows us to create such a mass of virtual packages? </p> </sect1> <sect1 id="particular_version"> <heading>Support one or several Particular Version(s)</heading> <p> For each python<var>X</var>.<var>Y</var> version you want to support, name the package <package>python<var>X</var>.<var>Y</var>-<var>foo</var></package> (a <em>versioned module package</em>). Make the dependency look like <example> Depends: python<var>X</var>.<var>Y</var> </example> Each of them should install modules somewhere inside <file>/usr/lib/python<var>X</var>.<var>Y</var>/site-packages/</file>. Any included scripts and examples should use <file>/usr/bin/python<var>X</var>.<var>Y</var></file> as interpreter. </p> <p> If necessary, the packaged source must declare <example> Build-Depends: pythonX.Y-dev, ... </example> including all <var>X</var>.<var>Y</var> supported versions. </p> </sect1> <sect1 id="all_versions"> <heading>Support All/Most Versions (Including Default)</heading> <p> This option is recommended for most modules packages. There are two different cases: <enumlist> <item> <p>Multiple versioned packages</p> <p> You have binary extensions that must be compiled against particular versions of Python. Create multiple <package>python<var>X</var>.<var>Y</var>-<var>foo</var></package> packages as in <ref id="particular_version">. Also create an empty default package <package>python-<var>foo</var></package> with <example> Depends: python (>= X.Y), python (<< X.Y+1), pythonX.Y-foo </example> Note that this kind of packaging means that the default package will trigger a conflict when the default Debian Python version in the distribution is changed, and that you will have to provide a new version of your package as soon as possible, since the package will block the upgrade of <package>python</package>. </p> <p> The packaged sources <tt>Build-Depends</tt> must contain all <package>python<var>X</var>.<var>Y</var>-dev</package> packages that the module is built for. </p> </item> <item>A single package for all versions (NOT YET SUPPORTED!) <p> You have a version independent Python module. Create a single package <package>python-<var>foo</var></package> that has a dependency <example> Depends: python </example> It should install modules somewhere inside <file>/usr/lib/python/site-packages/</file> and use <tt>#!/usr/bin/python</tt> for programs. The <file>postinst</file> script should create symlinks in all <file>/usr/lib/pythonX.Y/site-packages/</file> directories that point to its <file>/usr/lib/python/site-packages/</file> files and compile them. </p> <p> NOT YET SUPPORTED: It's errorprone if the package itself supplies these scripts. And the package cannot know when a new upstream Python version is installed. So the <package>python<var>X</var>.<var>Y</var></package> must provide these scripts, which is not yet done. </p> <p> The packaged source must declare <tt>Build-Depends</tt> on one <package>python<var>X</var>.<var>Y</var>-dev</package> package. XXX: Or build-depend on each Python version, so that only checked modules are uploaded? </p> <p> TODO: Should policy demand that these packages must have a dependency on <package>python (<= <var>X</var>.<var>Y+1</var></package>)? </p> </item> </enumlist> </p> </sect1> </sect> <sect id="package_names"> <heading>Module Package Names</heading> <p> Python module packages should be named for the primary module provided. The naming convention for a module <tt>Foo</tt> is <package>python-<var>foo</var></package> for the package for the default Python version (the <em>default module package</em>). (Packages which include multiple modules may additionally include provides for those modules using the same convention.) </p> <p> Python module packages packaged for one particular version of Python (<em>versioned modules packages</em>) should be named <package>python<var>X</var>.<var>Y</var>-foo</package>. </p> <p> A module package providing a module for python version, which is not the default python version, must not be named <package>python-<var>foo</var></package>, it has to be named <package>python<var>X</var>.<var>Y</var>-foo</package>. </p> </sect> <sect id="dependencies"> <heading>Dependencies</heading> <p> Packaged modules available for the default Python version as described in <ref id="default_version"> must depend on "<package>python (>= <var>X</var>.<var>Y</var>), python (<< <var>X</var>.<var>Y+1</var>)</package>". If they require other modules to work, they must depend on the corresponding <package>python-foo</package>. They must not depend on any <package>python<var>X</var>.<var>Y</var>-foo</package>. </p> <p> Packaged modules available for one particular version of Python must depend on the corresponding <package>python<var>X</var>.<var>Y</var></package> package instead. If they need other modules, they must depend on the corresponding <package>python<var>X</var>.<var>Y</var>-foo</package> packages, and must not depend on any <package>python-foo</package>. </p> </sect> <sect id="bytecompilation"> <heading>Modules Bytecompilation</heading> <p> If a package provides any binary-independent modules (<file>foo.py</file> files), the corresponding bytecompiled modules (<file>foo.pyc</file> files) and optimized modules (<file>foo.pyo</file> files) must not be shipped in the package. Instead, they should be generated at the package post-installation, using e.g. <example> PYTHON=python2.3 if which $PYTHON >/dev/null 2>&1; then DIRLIST="/usr/lib/python2.3/site-packages" for i in $DIRLIST ; do $PYTHON -E -O /usr/lib/$PYTHON/compileall.py -q $i $PYTHON -E /usr/lib/$PYTHON/compileall.py -q $i done fi </example> and removed in the package's pre-removal script, e.g. <example> dpkg -L python2.3-somemodule | awk '$0~/\.py$/ {print $0"c\n" $0"o"}' | xargs rm -f >&2 </example> </p> <p> If you use debhelper, it is a good idea to use <tt>dh_python</tt> just before <tt>dh_installdeb</tt>, in which case you must Build-depend on <package>debhelper (>= 4.1.67)</package>. It removes undesired files and generates those scripts automatically. It also helps automating the package's dependencies generation, using the <tt>${python:Depends}</tt> substitution variable. </p> </sect> <chapt id="programs"> <heading>Python Programs</heading> <sect id="version_indep_progs"> <heading>Programs using the default python</heading> <p> Programs that can run with any version of Python should be started with <tt>#!/usr/bin/python</tt>. They must also specify a dependency on <package>python</package>, with a versioned dependency if necessary. </p> <p> If the program needs the python module <tt>Foo</tt>, it must depend on <package>python-foo</package>. In the case where <package>python-foo</package> does not exist, but a <package>python<var>X</var>.<var>Y</var>-foo</package> package exists for the current python version, it can depend on "<package>python<var>X</var>.<var>Y</var>-foo, python (>= <var>X</var>.<var>Y</var>), python (<< <var>X</var>.<var>Y+1</var>)</package>" so that dependencies are robust upon the next major python upgrade. </p> <p> You're free to use <tt>#!/usr/bin/env python</tt>, if you'd like to give the user a chance to override the Debian Python package with a local version, but it is not recommended. </p> <sect1 id="current_version_progs"> <heading>Programs Shipping Private Modules</heading> <p> A program using <file>/usr/bin/python</file> as interpreter can come up with private python modules. These modules should be installed in <tt>/usr/lib/site-python/<var>module</var></tt>, <tt>/usr/lib/python<var>X</var>.<var>Y</var>/site-packages/<var>module</var></tt> (where python<var>X</var>.<var>Y</var> is the current python version). </p> <p> If the private modules would pollute the name space in <tt>sys.path</tt>, the modules should be installed in <tt>/usr/lib/<var>package</var></tt> (for architecture any) or <tt>/usr/share/<var>package</var></tt> (for architecture all). In this case, the directory should be added to <tt>sys.path</tt> at the program startup. </p> <p> Such a package must depend on "<package>python (>= <var>X</var>.<var>Y</var>), python (<< <var>X</var>.<var>Y+1</var>)</package>". </p> <p> The rules explained in <ref id="bytecompilation"> apply to those private modules: the bytecompiled modules must not be shipped with the package, they should be generated in the package's postinst, using the current default python version, and removed in the prerm. </p> <p> TODO: Currently there is no mechanism to automatically recompile such modules when the default python version changes. The required dependency on "<package>python (>= <var>X</var>.<var>Y</var>), python (<< <var>X</var>.<var>Y+1</var>)</package>" ensures the package is upgraded, and hence recompiled, when the default python version changes. In the future, a mechanism may be introduced to automatically recompile such modules when the <package>python</package> package is upgraded, allowing such packages which support several python versions to only depend on "<package>python (>= <var>min.version</var>)</package>". </sect1> </sect> <sect id="version_dep_progs"> <heading>Programs Using a Particular Python Version</heading> <p> Programs which require a specific version of Python must start with <tt>#!/usr/bin/python<var>X</var>.<var>Y</var></tt>. They must also specify a dependency on <package>python<var>X</var>.<var>Y</var></package> and on any <package>python<var>X</var>.<var>Y</var>-foo</package> package providing necessary modules. They must not depend on any <package>python-foo</package> package. </p> <p> Again, if you're using <tt>#!/usr/bin/env python<var>X</var>.<var>Y</var></tt>, please be aware that a user might override the Debian Python package with a local version. </p> <p> If the program ships private python modules, these modules should be installed in <tt>/usr/lib/python<var>X</var>.<var>Y</var>/site-packages/<var>module</var></tt> (where python<var>X</var>.<var>Y</var> is the same python version the program uses) or <tt>/usr/lib/<var>package</var></tt>. In the latter case, this directory should be added to <tt>sys.path</tt> at the program startup. They must not be shipped in <tt>/usr/lib/site-python/</tt>. The latter case is recommended, if the private modules would pollute the name space in <tt>sys.path</tt>. </p> <p> The bytecompiled versions of the modules must not be shipped in the package, and they should be generated as explained in <ref id="bytecompilation">, using the same python<var>X</var>.<var>Y</var> version as the program. </p> </sect> </chapt> <chapt id="embed"> <heading>Programs Embedding Python</heading> <sect id="build_embedded"> <heading>Building Embedded Programs</heading> <p> Programs which embed a Python interpreter must declare a <tt>Build-Depends</tt> on <package>python<var>X</var>.<var>Y</var>-dev</package>, where python<var>X</var>.<var>Y</var> is the python version the program builds against. It should be the current default python version unless the program doesn't work correctly with this version. </p> </sect> <sect id="embedded_deps"> <heading>Embedded Python Dependencies</heading> <p> Dependencies for programs linking against the shared Python library will be automatically created by <prgn>dpkg-shlibdeps</prgn>. The <tt>libpython<var>X</var>.<var>Y</var>.so.<var>Z</var></tt> library the program is built against is provided by the <package>python<var>X</var>.<var>Y</var></package> package. </p> </sect> </chapt> <chapt id="other"> <heading>Interaction with Locally Installed Python Versions</heading> <p> As long as you don't install other versions of Python in your path, Debian's Python versions won't be affected by a new version. </p> <p> If you install a different subrelease of the version of python you've got installed, you'll need to be careful to install all the modules you use for that version of python too. </p> </chapt> <appendix id="build_dependencies"> <heading>Build Dependencies</heading> <p> Build dependencies for Python dependent packages must be declared for every Python version that the package is built for. To build for a specific version, add the versioned dependencies; to build for the default version, add the unversioned dependency. Architecture dependent packages must depend on the <package>-dev</package> package; for architecture independent packages, it may be sufficient to depend on the <package>python</package> or <package>python<var>X</var>.<var>Y</var></package> package. </p> <p> Build-Depend on at least: <example> Build-Depends: python1.5 Build-Depends: python2.1 Build-Depends: python2.2 (>= 2.2.3) Build-Depends: python2.3 (>= 2.3-1) Build-Depends: python (>= 2.3) Build-Depends: python1.5-dev (>= 1.5.2-18.6) Build-Depends: python1.5-distutils Build-Depends: python2.1-dev (>= 2.1.1-1.4) Build-Depends: python2.2-dev (>= 2.2.3) Build-Depends: python2.3-dev (>= 2.3-1) Build-Depends: python-dev (>= 2.3) </example> </p> </appendix> <appendix id="upgrade"> <heading>Upgrade Procedure</heading> <p> This section describes the procedure for the upgrade when the default python version is changed in the <tt>unstable</tt> distribution, requiring recompilation of many python-related packages. </p> <p> <enumlist> <item> <p> The Debian Python maintainer decides for the new default Debian Python version and announces the upgrade. </p> </item> <item> <p> Upload of the python core metapackages <package>python</package>, <package>python-dev</package>, <package>python-doc</package> and several <package>python-<var>module</var></package>, depending on the new <package>python<var>X</var>.<var>Y</var></package>, <package>python<var>X</var>.<var>Y</var>-dev</package> and so on. </p> <p> These new packages will make uninstallable all python packages depending on the previous <package>python</package> with a dependency requiring version less than <var>X</var>.<var>Y</var>. </p> </item> <item> <p> From this point, all these python modules/packages which are uninstallable have to be rebuilt against the new python version, fixing dependencies and build-dependencies. </p> <p> NMUs are allowed after notifying the package maintainer to have all these packages rebuilt in a reasonable timeline. </p> </item> <item> <p> File bug report against packages and/or make NMU's for packages that are not adapted by their maintainer. </p> </item> <item> <p> If a package doesn't work with the new python version, make it use the older version as described in <ref id="version_dep_progs">. </p> </item> <item> <p> When all packages are updated (or removed), the new <package>python</package> packages can migrate to <tt>testing</tt> together with all packages depending on it. </p> </item> </enumlist> </p> </appendix> </book> </debiandoc>