Don't use Poetry, if you don't have to.

(10. Dezember 2023, en)

In my brain the title translates to »basically never use Python Poetry«. But why?

Python Poetry is a neat tool. It fixes the majority of dependency issues, adds some bells and whistles at the side and seems to be “hip”. But my gut says: I’m not sure. Huh? »Why?« I might ask, but the thing with gut feelings is that you cannot really get this question answered directly.

Let the search begin.

A good starting point is »Why not tell people to “simply” use pyenv, poetry or anaconda«. It outlines some major reasons for avoiding Poetry, for me the two main reasons are:

using Poetry is not simple: it is another layer of abstraction on top of existing tooling. It comes with its own quirks and issues. You need to read the docs before you can use it as first time user.
not more, but less dependencies: Poetry is itself already dependency-heavy. So you now have your project’s dependencies plus all dependencies of Poetry. That needs to be installed, maintained and updated. The episode »It dependencies« of the Podcast changelog reminded me of this.
it is a maintenance burden: developers need to update the lock file from time to time, which is just extra work nobody is waiting for!
it clutters your git commits with lock file changes
onboarding new devs is more work
Setting up CI/CD and Docker containers is more work

A huge thing is predictability. A lock file can reduce the chance of your code breaking in unpredictable ways. Personally, I cannot remember where the requirements specs in requirements.txt or pyproject.toml were not enough. But the pip-tools statement makes sense:

In building your Python application and its dependencies for production, you want to make sure that your builds are predictable and deterministic. – pip-tools README

I still question myself: do I need it? I’m not so sure. To make it bold:

It is not essential to have .lock in the initial phase of your project

New projects: my standard setup

I conclude: it is sufficient to keep a normal requirements.txt (or pyproject.toml) with versions pinned to the minor version or whatever suits you. The default ingredients are Python packages

Python packages

If possible I try to group my code (even if it is only for internal use) into Python packages. I use a fairly standard pyproject.toml

# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
[tool.pytest.ini_options]
pythonpath = ["src", "tests"]
norecursedirs = [
    "tests/testkit"
]

[tool.ruff]
line-length = 111

[build-system]
requires = ["setuptools", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"

[tool.coverage.paths]
source = ["src"]

[project]
name = "something-py"
dynamic = ["version"]
description = "does something"
authors = [{ name = "Something Ltd.", email = "something@bargsten.org" }]
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
    "pyyaml",
    "msgspec",
    "pyspark~=3.3.1",
]
# https://spdx.org/licenses/
license = { text = "Apache-2.0" }
# or
# license = { file = "LICENSE" }

[project.urls]
homepage = "https://github.com/jwbargsten/..."
repository = "https://github.com/jwbargsten/..."

[project.optional-dependencies]
dev = [
    "ruff",
    "pytest>=7",
]
docs = [
    "sphinx",
    "myst-parser",
    "sphinx-sitemap",
    "pydata-sphinx-theme",
    "sphinx-autodoc-typehints",
    "myst-nb",
]
build = [
    "build",
    "twine",
    "tomli"
    "setuptools_scm",
]



# optional
[tool.setuptools_scm]
# write_to = "src/something/_version.py"

# optional
[tool.setuptools.package-data]
# "something.assets" = ["*.png"]

# optional
[project.scripts]
# something = "something.cli:main"

I have a slight aversion to pre-commit, so I skip it. The aversion stems from the belief that developers should be able to choose their own workflow. All sorts of pre-commits, hooks, etc. work against that when working in a team. My workflow falls into the category: first make a mess, then make it clean. Having my messy commits checked against all types of syntax and formatting rules slows me down. Instead I usually »refactor« my commits before I create a PR. So I don’t use hooks or pre-commits. I put everything into a Makefile. Check out »Managing your Python project with a Makefile« for more details on Python and Makefiles

Setup for a (web)app

Even simpler here, I use requirements.txt with pyproject.toml (for ruff settings) orchestrated by a Makefile in a virtual environment.

Done.