dwas.predefined

This module contains common predefined steps to help reduce duplication.

Note

If you think a predefined solution should be provided for a particular utility, please submit an issue, or contribute!

Attention

All predefined steps here have safe defaults, and will not modify any source code unless told so. Please make sure you understand the underlying tools when configuring them.

Inventory

Functions

black	Run the Black formatter against your python source code.
coverage	Run coverage.py to generate coverage reports.
docformatter	Run docformatter against your python source code.
isort	Run the isort formatter against your python source code.
mypy	Run mypy against your python source code.
package	Build a python package that follows PEP 517, and install it in dependent venvs.
pylint	Run pylint against your source code.
pytest	Run pytest.
ruff	Run Ruff against your python source code.
sphinx	Run sphinx.
twine	Run twine against the provided packages.
unimport	Run the Unimport formatter against your python source code.

Functions

dwas.predefined.black(*, files: Sequence[str] | None = None, additional_arguments: list[str] | None = None) → Step

Run the Black formatter against your python source code.

By default, it will depend on ["black"], when registered with dwas.register_managed_step().

Parameters:

• files – The list of files or directories to run black against. Defaults to ["."].

• additional_arguments – Additional arguments to pass to the black invocation. Defaults to ["--check", "--diff", "-W1"].

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

In order to verify your code but not change it, for a step named black:

register_managed_step(dwas.predefined.black())

Or, in order to automatically fix your code, but only if requested:

register_managed_step(
    dwas.predefined.black(additional_arguments=[]),
    # NOTE: this name is arbitrary, you could omit it, or specify
    #       something else. We suffix in our documentation all
    #       operations that will have destructive effect on the source
    #       code by ``:fix``
    name="black:fix",
    run_by_default=False,
)

Note that if you have other steps that will overwrite some of your files, you might want to order them so they don’t conflict. For example, assuming that you also use dwas.predefined.isort():

register_managed_step(
    dwas.predefined.isort(additional_arguments=["--atomic"]),
    name="isort:fix",
    run_by_default=False,
)
register_managed_step(
    dwas.predefined.black(additional_arguments=[]),
    name="black:fix",
    run_by_default=False,
)

This will ensure that both steps don’t step on each other and make black run second.

dwas.predefined.coverage(*, reports: list[list[str]] | None = None) → Step

Run coverage.py to generate coverage reports.

By default, it will depend on ["coverage"], when registered with dwas.register_managed_step().

Parameters:

reports – A list of parameters to pass to coverage to generate reports. Defaults to [["report", "--show-missing"]]

This step leverages artifacts named coverage_files provided by other steps to provide reports.

Example:

Here is a fully fledged example that packages source code, runs pytest and generates coverage out of it:

# One step to generate the package
dwas.register_managed_step(dwas.predefined.package())

# One step to run pytest across multiple python versions
dwas.register_managed_step(
    dwas.parametrize("python", ["3.9", "3.10", "3.11"])(
        dwas.predefined.pytest()
    ),
    dependencies=["pytest', "pytest-cov"],
    requires=["package"]
)

# And finally generate xml report and one on stdout.
# This will combine all coverage info from the previous pytest runs.
dwas.register_managed_step(
    dwas.predefined.coverage(
        reports=[
            ["xml", "-o", "reports/coverage.xml"],
            ["report", "--show-missing"],
        ],
    ),
    requires=["pytest"],
)

Returns:

The step so that you can add additional parameters to it if needed.

dwas.predefined.docformatter(*, files: Sequence[str] | None = None, additional_arguments: list[str] | None = None, expected_status_codes: list[int] | None = None) → Step

Run docformatter against your python source code.

By default, it will depend on ["docformatter"], when registered with dwas.register_managed_step().

Parameters:

• files – The list of files or directories to run docformatter against. Defaults to ["."].

• additional_arguments – Additional arguments to pass to the docformatter invocation. Defaults to ["--recursive"].

• expected_status_codes – Status codes that are acceptable from docformatter. Defaults to [0]. When formatting in place, you might want to set to [0, 3], as docformatter always returns 3 if a file was modified.

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

In order to verify your code but not change it, for a step named docformatter:

dwas.register_managed_step(
    dwas.predefined.docformatter(files["src", "tests", "dwasfile.py", "setup.py"])
)

Or, in order to automatically fix your code, but only if requested:

dwas.register_managed_step(
    dwas.predefined.docformatter(
        # NOTE: this name is arbitrary, you could omit it, or specify
        #       something else. We suffix in our documentation all
        #       operations that will have destructive effect on the source
        #       code by ``:fix``
        name="docformatter:fix",
        additional_arguments=["--in-place"],
        expected_status_codes=[0, 3],
        run_by_default=False,
        files=["src,", "tests", "dwasfile.py", "setup.py"],
    )
)

dwas.predefined.isort(*, files: Sequence[str] | None = None, additional_arguments: list[str] | None = None) → Step

Run the isort formatter against your python source code.

By default, it will depend on ["isort[colors]"], when registered with dwas.register_managed_step().

Parameters:

• files – The list of files or directories to run isort against. Defaults to ["."].

• additional_arguments – Additional arguments to pass to the isort invocation. Defaults to ["--check-only", "--diff"].

Returns:

The step so that you can add additional parameters to it if needed.

Tip

isort can be quite slow to find all files it needs to handle, you might want to limit the number of directories searched.

Examples:

In order to verify your code but not change it, for a step named isort:

dwas.register_managed_step(
    dwas.predefined.isort(files["src", "tests", "dwasfile.py", "setup.py"])
)

Or, in order to automatically fix your code, but only if requested:

dwas.register_managed_step(
    dwas.predefined.isort(
        additional_arguments=["--atomic"],
        files=["src,", "tests", "dwasfile.py", "setup.py"],
    ),
    # NOTE: this name is arbitrary, you could omit it, or specify
    #       something else. We suffix in our documentation all
    #       operations that will have destructive effect on the source
    #       code by ``:fix``
    name="isort:fix",
    run_by_default=False,
)

dwas.predefined.mypy(*, files: Sequence[str] | None = None, additional_arguments: list[str] | None = None) → Step

Run mypy against your python source code.

By default, it will depend on ["mypy"], when registered with dwas.register_managed_step().

Parameters:

• files – The list of files, directories or packages to run mypy against. Defaults to ["."].

• additional_arguments – Additional arguments to pass to the mypy invocation. Defaults to [].

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

dwas.register_managed_step(
    # Only run for sources, not tests/ or setup.py
    dwas.predefined.mypy(files=["./src"]),
    dependencies=["mypy", "types-requests"],
)

dwas.predefined.package(*, isolate: bool = True) → Step

Build a python package that follows PEP 517, and install it in dependent venvs.

Warning

This step is not suitable for building wheels with C extensions. If you have such requirements, please see the manylinux project, or other similar initiatives.

This step does not require dependencies by default.

Parameters:

isolate – Whether to create a new virtual environment for building the package, or run it in the one created for the step. Setting this to False does bring a measurable speedup, but might lead to under-declared build dependencies. Defaults to True.

Returns:

The step so that you can add additional parameters to it if needed.

This leverages uv build in order to build a source distribution and a universal wheel (assuming there are no c-extensions).

When this step is used as a requirement for another step, it will also install the wheel inside it.

This allows you to only build the sdist and wheel once, and then install your package in various different other steps, speeding up your whole pipeline. It also helps ensuring that your packaging is correct, as you end up running against the same version as if you were installing your project from e.g. pypi.

Tip

Using isolate=False will speedup measurably this step, but reduces your confidence in not under-declaring build dependencies.

Examples:

In order to generate a step package:

# NOTE: we use just register_step as there is no need for dependencies
dwas.register_step(dwas.predefined.package())

Or, if you want your step to run faster:

# NOTE: we use register_managed_step as we now need dependencies
dwas.register_managed_step(
    dwas.predefined.package(isolate=False),
    # We could read those from pyproject.toml directly, but we'd need
    # to install a toml reading library, so let's live with duplication
    # for now.
    dependencies=["setuptools>=61.0.0", "wheel"],
)

dwas.predefined.pylint(*, files: Sequence[str] | None = None, additional_arguments: Sequence[str] | None = None) → Step

Run pylint against your source code.

By default, it will depend on ["pylint"], when registered with dwas.register_managed_step().

Parameters:

• files – The list of files or directories to run pylint against. Defaults to ["."].

• additional_arguments – Additional arguments to pass to the pylint invocation. Defaults to [].

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

dwas.register_managed_step(
    dwas.predefined.pylint(files=["./src", "./tests"]),
    # Install both test and package dependencies to make pylint
    # find them
    dependencies=["requests", "pytest", "pylint"],
)

dwas.predefined.pytest(*, args: Sequence[str] | None = None) → Step

Run pytest.

By default, it will depend on ["pytest"], when registered with dwas.register_managed_step().

Parameters:

args – arguments to pass to the pytest invocation. Defaults to [].

Returns:

The step so that you can add additional parameters to it if needed.

Tip

If you use pytest-cov, it will also automatically expose a coverage_files artifact that can be used by dependent steps, for an example, see coverage()

Examples:

For running pytest with the a specific version of python, with your code in the PYTHONPATH:

dwas.register_managed_step(dwas.predefined.pytest(), python="3.9")

Or, for a more concrete example, across multiple versions of python and testing your installed application:

# Setup a "package" step, to install the source code automatically
# for the tests
dwas.register_managed_step(dwas.predefined.package())

dwas.register_managed_step(
    dwas.parametrize("python", ["3.9", "3.10", "3.11"])(
        dwas.predefined.pytest()
    ),
    dependencies=["pytest", "pytest-cov"],
    requires=["package"],
)

Leveraging dwas.parametrize(), this generates 4 different steps: pytest, which is a step group which depends on the other pytest[3.9], pytest[3.10] and pytest[3.11] which each run pytest with the given version of python.

Similarly, if you wanted to test multiple multiple versions of a python package, with different versions of python:

dwas.register_managed_step(
    dwas.parametrize("python", ["3.9", "3.10", "3.11"])(
        dwas.parametrize(
            "dependencies",
            [["pytest", "django==3.0"], ["pytest", "django==4.0"]],
            ids=["django3", "django4"],
        )(dwas.predefined.pytest()),
    requires=["package"],
)

dwas.predefined.ruff(*, files: Sequence[str] | None = None, additional_arguments: list[str] | None = None) → Step

Run Ruff against your python source code.

By default, it will depend on ["ruff"], when registered with dwas.register_managed_step().

Parameters:

• files – The list of files or directories to run ruff against. Defaults to ["."].

• additional_arguments – Additional arguments to pass to the ruff invocation. Defaults to ["check"]. Defaults to ["--check", "--diff", "-W1"].

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

In order to verify your code but not change it, for a step named ruff:

register_managed_step(dwas.predefined.ruff())

Or, in order to automatically fix your code, but only if requested:

register_managed_step(
    dwas.predefined.ruff(additional_arguments=["check", "--fix"]),
    # NOTE: this name is arbitrary, you could omit it, or specify
    #       something else. We suffix in our documentation all
    #       operations that will have destructive effect on the source
    #       code by ``:fix``
    name="ruff:fix",
    run_by_default=False,
)

Similarly, if you want to use ruff to format your code you could do:

# To check the formatting
register_managed_step(
    dwas.predefined.ruff(additional_arguments=["format", "--diff"]),
    name="ruff:format-check",
)
# To autoformat
register_managed_step(
    dwas.predefined.ruff(additional_arguments=["format"]),
    name="ruff:format",
)

dwas.predefined.sphinx(*, builder: str | None = None, sourcedir: Path | str | None = None, output: Path | str | None = None, warning_as_error: bool | None = None) → Step

Run sphinx.

By default, it will depend on ["sphinx"], when registered with dwas.register_managed_step().

Parameters:

• builder – The sphinx builder to use. Defaults to "html".

• sourcedir – The directory in which the conf.py resides. Defaults to ".".

• output – The directory in which to output the generated files. If None, will keep the data in the cache. Defaults to None.

• warning_as_error – Turn warnings into errors Defaults to False.

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

For running sphinx with a specific version of python, with your conf.py under docs/, outputting the html files under _build/docs:

dwas.register_managed_step(
    dwas.predefined.sphinx(sourcedir="docs", output="_build/docs"),
    python="3.9"
)

Or, to run doctests, linkchecks and build the output to _build/docs, requiring the current package to be installed (see dwas.predefined.package()):

register_managed_step(
    dwas.parametrize(
        ("builder", "output"),
        [
            ("html", "_build/docs"),
            # We don't care about the output of those two here.
            ("linkcheck", None),
            ("doctests", None),
        ],
        ids=["html", "linkcheck", "doctests"],
    )(dwas.predefined.sphinx()),
    requires=["package"],
)

dwas.predefined.twine(*, additional_arguments: list[str] | None = None) → Step

Run twine against the provided packages.

By default, it will depend on ["twine"], when registered with dwas.register_managed_step().

This step will ask the steps it requires for artifacts named sdists and wheels and will run against those.

Parameters:

additional_arguments – Additional arguments to pass to the twine invocation. Defaults to ["check", "--strict"].

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

Examples here assume that you use the package() step like:

# This creates a 'package' step
dwas.register_managed_step(dwas.predefined.package())

In order to make sure your distribution files are ready to be published:

dwas.register_managed_step(
    dwas.predefined.twine(),
    name="twine:check",
    requires=["package"],
)

And if you want to be able to use dwas twine:publish to publish:

dwas.register_managed_step(
    dwas.predefined.twine(
        additional_arguments=[
            "upload",
            "--verbose",
            # This is not required, if you don't want to gpg-sign your
            # distribution files
            "--sign",
            "--non-interactive",
        ],
    ),
    name="twine:upload",
    passenv=["TWINE_REPOSITORY", "TWINE_USERNAME", "TWINE_PASSWORD"],
    requires=["package", "twine:check"],
    run_by_default=False,
)

dwas.predefined.unimport(*, files: Sequence[str] | None = None, additional_arguments: list[str] | None = None) → Step

Run the Unimport formatter against your python source code.

By default, it will depend on ["unimport"], when registered with dwas.register_managed_step().

Parameters:

• files – The list of files or directories to run unimport against. Defaults to ["."].

• additional_arguments – Additional arguments to pass to the unimport invocation. Defaults to ["--check", "--diff", "--gitignore"].

Returns:

The step so that you can add additional parameters to it if needed.

Examples:

In order to verify your code but not change it, for a step named unimport:

dwas.register_managed_step(dwas.predefined.unimport())

Or, in order to automatically fix your code, but only if requested:

dwas.register_managed_step(
    dwas.predefined.unimport(
        # NOTE: `--gitignore` here is not required, but probably a good idea
        additional_arguments=["--remove", "--gitignore"],
    ),
    # NOTE: this name is arbitrary, you could omit it, or specify
    #       something else. We suffix in our documentation all
    #       operations that will have destructive effect on the source
    #       code by ``:fix``
    name="unimport:fix",
    run_by_default=False,
)

Note that if you have other steps that will overwrite some of your files, you might want to order them so they don’t conflict. For example, assuming that you also use dwas.predefined.isort():

dwas.register_managed_step(
    dwas.predefined.unimport(
        # NOTE: `--gitignore` here is not required, but probably a good idea
        additional_arguments=["--remove", "--gitignore"],
    ),
    name="unimport:fix",
    run_by_default=False,
)
dwas.register_managed_step(
    dwas.predefined.isort(additional_arguments=["--atomic"]),
    name="isort:fix",
    requires=["unimport:fix"],
    run_by_default=False,
)

This will ensure that both steps don’t step on each other and make unimport run first.