= PyProject =

The Python package build system is split into frontends and backends according to protocols found in several PEPs; chiefly [[https://www.python.org/dev/peps/pep-0517/|517]] and [[https://www.python.org/dev/peps/pep-0660/|PEP 660]]. In modern usage, this system is orchestrated through the '''`pyproject.toml`''' file.

<<TableOfContents>>

----



== Legacy ==

The original packaging system depended on specially-crafted `setup.py` scripts.

Almost universally, these required that the third-party `setuptools` package be installed.

{{{
#!/usr/bin/env python3
from setuptools import setup
from pathlib import Path

this_directory = Path(__file__).parent
long_description = (this_directory / "README.md").read_text()

setup(
  name='my-project',
  version=1.0.2,
  description='This is the short description',
  long_description=long_description,
  long_description_content_type='text/markdown',
  license='GPL',
  author='John Doe',
  author_email='jdoe@example.com',
  url='example.com/my-project',
  install_requires=[
      'toml>=0.10.2',
  ],
  entry_points={
    'console_scripts': [
      'my-project-cli = my-project:main',
    ],
  },
)
}}}

To pull the version from a source control mechanism, snippets like this were used:

{{{
import subprocess

ver = subprocess.run(['git', 'describe', '--tags'], stdout=subprocess.PIPE).stdout.decode().strip()

setup(
  ...
  version=ver,
  ...
)
}}}

Such a package is installed by running `python setup.py install`.

To reduce the size of built packages, source code files to be included were sometimes dynamically selected with snippets like:

{{{
from glob import glob

setup(
  ...
  package_data={
    'my-project': [
      'Makefile',
      'README.md',
    ] + [f[5:] for f in glob('static/**', recursive=True)]
  },
  ...
)
}}}



=== Distributions ===

There are fundamentally two types of package distributions: source distributions ('''sdist''') and binary distributions ('''bdist''').

The command for creating an sdist is `python setup.py sdist`. The distribution would be placed in a `dist/` subfolder.

There have been several different bdist formats over time. One of the earliest was '''`egg`''' files, which was favored by the `easy_install` package installer. Eggs files were named like `my-package-1.0.2-py3.6.egg` and were specially-crafted ZIP archives. The command for creating an wheel was `python setup.py bdist_egg`. The egg would be placed in a `dist/` subfolder, build artifacts would be placed in a `build/` subfolder, and parsed information was collected in a subfolder named like `my-package.egg-info/`.

The modern standard bdist format is '''`wheel`''' files, which is favored by [[Python/Pip|pip]]. Wheel files are named like `my-project-1.0.2-py3-none-any.whl` and are also specially-crafted ZIP archives. The command for creating a wheel is `python setup.py bdist_wheel`, which depended on the `wheel` package being installed. Otherwise the same subfolders were used, including `my-package.egg-info/`.

Note that the supported types of distributions came from the standard library's `distutils` module. This was obscured over time by the popularity of `setuptools`.

----



== PEPs 517, 660, and 518 ==

[[https://peps.python.org/pep-0518/|PEP 518]] introduced the '''`pyproject.toml`''' file, intended to declare a package's build backend.

To continue using the legacy build system (i.e., `setup.py`), the following was sufficient.

{{{
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
}}}

New packaging tools were encouraged to adopt this file specification and read from separately-named tables. (Note: in the TOML specification, `[table-name]` declares a table and any lines following it are attributes to that table.)



=== Setup.cfg ===

While `pyproject.toml` was adopted, `setuptools` did not embrace it. As a half-way step, a declarative configuration language was introduced for the '''`setup.cfg`''' file.

A minimal `setup.cfg` looked like:

{{{
[metadata]
name = my-project
version = 1.0.2
description = This is the short description
long_description = file: path/to/my/long/description
license = GPL
author = John Doe
author_email = jdoe@example.com
url = example.com/my-project

[options]
packages = my-project
python_requires = >= 3.6
install_requires =
    toml >= 0.10.2

[options.entry_points]
console_scripts =
    my-project-cli = my-project:main

[options.package_data]
my-project =
    Makefile
    README.md
    static/*
}}}

This enabled the use of minimal shim `setup.py` files:

{{{
#!/usr/bin/env python
import setuptools
if __name__ == "__main__":
    setuptools.setup()
}}}

With this pair of files, `setup.py install` remained a functional build command.

Note that the ability to include globbing patterns under `[options.package_data]` was added much later. Previously, the recommendation was to ''not'' migrate off of `setup.py`.

----



== PEPs 631 and 621 ==

[[https://peps.python.org/pep-0631/|PEP 631]] designed the '''`[project]`''' table for `pyproject.toml`.

{{{
[project]
name = "my-project"
description = "This is the short description"
readme = "path/to/my/long/description"
version = "1.0.2"
authors = [ { name = "John Doe", email = "jdoe@example.com" } ]
urls = { homepage = "example.com/my-project" }
license = { file = "path/to/my/license" }
requires-python = ">=3.6"
dependencies = [
    "toml >= 0.10.1",
]

[project.scripts]
my-project-cli = "my-project:main"
}}}

Note the similarities to the `setup.cfg` syntax. Some notable changes to highlight:
 * `long_description`
   * a long standing practice was to read a project's `README` file and provide this as a `long_description`
   * now, just provide a path to the `README` file
 * `author`, `author_email`, `maintainer`, `maintainer_email`
   * `authors` and `maintainers` have replaced these
   * this design greatly simplifies the specification of multiple individuals
 * `license`
   * while a license may continue to be provided as a string, the syntax has changed (`license = { text = "GPL" }`)
   * the current recommendation is to provide a path to the license file
   * future PEPs may build on this design
 * `entry_points`
   * `project.scripts` exists as a more-or-less perfect mapping
   * an analogous `[projects.gui-scripts]` exists for scripts that should only be called in a graphical setting

This PEP was ultimately superseded by the specification of [[https://peps.python.org/pep-0621/|PEP 621]].



=== Dependencies ===

The dependencies attribute can be heavily customized. As an example, this was the `pyproject.toml` of `docker-compose` at one point:

{{{
[project]
dependencies = [
  'cached-property >= 1.2.0, < 2',
  'distro >= 1.5.0, < 2',
  'docker[ssh] >= 4.2.2, < 5',
  'dockerpty >= 0.4.1, < 1',
  'docopt >= 0.6.1, < 1',
  'jsonschema >= 2.5.1, < 4',
  'PyYAML >= 3.10, < 6',
  'python-dotenv >= 0.13.0, < 1',
  'requests >= 2.20.0, < 3',
  'texttable >= 0.9.0, < 2',
  'websocket-client >= 0.32.0, < 1',

  # Conditional
  'backports.shutil_get_terminal_size == 1.0.0; python_version < "3.3"',
  'backports.ssl_match_hostname >= 3.5, < 4; python_version < "3.5"',
  'colorama >= 0.4, < 1; sys_platform == "win32"',
  'enum34 >= 1.0.4, < 2; python_version < "3.4"',
  'ipaddress >= 1.0.16, < 2; python_version < "3.3"',
  'subprocess32 >= 3.5.4, < 4; python_version < "3.2"',
]

[project.optional-dependencies]
socks = [ 'PySocks >= 1.5.6, != 1.5.7, < 2' ]
tests = [
  'ddt >= 1.2.2, < 2',
  'pytest < 6',
  'mock >= 1.0.1, < 4; python_version < "3.4"',
]
}}}

Note that the syntax for version specification is inherited from [[https://peps.python.org/pep-0440/#version-specifiers|PEP 440]].



=== Package Data ===

`setuptools` continues to support limiting package data. This functionality is moved to a separately-named table.

{{{
[tool.setuptools.package-data]
my-project = ["Makefile", "README.md", "static/*"]
}}}

Note that globbing is now supported. It is necessary to use Unix-style paths for all glob patterns, even when building on Windows.



=== Setuptools SCM ===

`setuptools_scm` is an extension of `setuptools` for pulling version information dynamically from source control mechanisms.

First, `version` '''must''' be removed from the `[package]` table and declared as dynamic.

{{{
[project]
# version = "1.0.2"
dynamic = ["version"]
}}}

A minimal template for `pyproject.toml` becomes:

{{{
[build-system]
requires = ["setuptools>=64", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"

[tool.setuptools_scm]
# presence enables setuptools-scm
}}}

Note that another feature of `setuptools_scm` is automatic Package Data determination. Built packages only include files tracked by the source control mechanism. To disable this feature, you must disable the Package Data functionality entirely.

{{{
[tool.setuptools]
include-package-data = false
}}}



=== Distributions ===

In the modern build system, distributions have been pared down to sdist and wheel.

Furthermore, `distutils` is deprecated and will be removed from the standard library. The `setuptools` project has adopted a large subset of its API as a compatibility module. In other words, building a package distribution will soon require external dependencies.

The new recommended method for building package distributions is installing the `build` package and running `python -m build`. This creates both an sdist and a wheel, and specifically builds the wheel from the sdist. (Note that `setuptools` and `wheel` are dependencies of `build`.)



----
CategoryRicottone