clone,
status, diff, add,
commit, and push)
.git/hooks/ directory of your
repository, but excluded from version control
git commit
.git/hooks/pre-commit,
pre-commit and include the hook
configuration in version control.
pre-commitpre-commit is "a multi-language package manager for
pre-commit hooks" (source).
pre-commit installs its own script as the
pre-commit hook executable for your repository.
pre-commit,
which in turn runs any checks you specify using a YAML file.
pre-commit package in your repository.
.pre-commit-config.yaml.
pre-commit install itself as your pre-commit
hook.
$ python3 -m pip install pre-commit
Defined in .pre-commit-config.yaml at the repository
root:
The repos section lists the repositories to use:
This represents the configuration for a single repository:
The URL of the repository to clone:
The tag or commit hash to clone at:
The hooks to run from that repository:
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
Source:
How to Set Up Pre-Commit Hooks
by Stefanie Molin
While the .pre-commit-config.yaml file lives in version
control, each contributor must run this locally:
The check-yaml hook we included will make sure that the
.pre-commit-config.yaml file (and any other YAML files
in this repository) is valid YAML. Try committing it:
There are more hooks provided by the repository that we are currently using. Take a moment to look through their documentation, and add some additional hooks to the setup that are relevant to your work. Be sure to commit your changes.
Adding these two hooks ensures that your scripts are actually executable:
We are free to use hooks from multiple repositories. A non-exhaustive list of popular hooks along with tips for searching for compatible repositories can be found at pre-commit.com/hooks.html.
Let's add
ruff
to lint and format our Python code, since it is "10-100x faster
than existing linters (like Flake8) and formatters (like
Black),"*
and speed very is important with pre-commit hooks.
ruff homepage
as of June 16, 2024
ruff to .pre-commit-config.yaml
Create a new entry under the repos key:
The ruff pre-commit hook is in a separate repository:
The tag or commit hash to clone at:
The hooks to run from that repository:
Use args to pass command line arguments to the hook:
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.4.10
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix, --show-fixes]
- id: ruff-format
This once again triggers an update to the environment used for running the checks:
All hooks that were added or modified hooks need to be updated in the environment:
This will be slower the first time, but the environment persists until modification:
Hook order in the configuration file matters – hooks run in the order you specify and may make conflicting changes:
ruff pre-commit hooks
Create a new file called example.py with the following
contents, and try to commit it:
import re
def my_function(a):
"""My function."""
pass
No TOML or YAML files to check:
No issues with these checks:
ruff modified a file, which is an automatic failure:
ruff found an unused import when linting:
ruff summarizes its findings (sometimes we have to
fix it):
ruff also made a change while formatting the file:
$ git diff example.py
diff --git a/example.py b/example.py
index ebb29eb..ad4c89e 100644
--- a/example.py
+++ b/example.py
@@ -1,5 +1,3 @@
-import re
-
def my_function(a):
"""My function."""
pass
Look at the setup information for the
numpydoc-validation hook at
numpydoc.readthedocs.io/en/latest/validation.html, and add it to your configuration. Be sure to commit your changes.
Tip: You can run the following to check for any formatting errors:
$ pre-commit validate-config .pre-commit-config.yaml
Adding the numpydoc-validation hook:
- repo: https://github.com/numpy/numpydoc
rev: v1.7.0
hooks:
- id: numpydoc-validation
Run all hooks on staged changes:
Run all hooks on example.py:
Run only the numpydoc-validation hook on all files:
Run numpydoc-validation hook on
example.py:
example.py fails the docstring checks:
Without any additional configuration, all numpydoc checks are run:
Output abbreviated.
Two options:
For behavior that you only want to happen when the tool is run as a
pre-commit hook (we want the issues fixed automatically here, but
perhaps when we run ruff on our own, we just want it to
report on what it would have changed):
For settings that should be applied for every invocation of the tool:
Configure the numpydoc-validation hook to ignore the
checks ES01, EX01, SA01, and SS06. Then, address any issues in
example.py so that it passes the checks.
First, specify which checks to report in the configuration file:
# pyproject.toml (this tool supports pyproject.toml)
[tool.numpydoc_validation]
checks = [
"all", # report on all checks
"ES01", # but don't require an extended summary
"EX01", # or examples
"SA01", # or a see also section
"SS06", # and don't require the summary to fit on one line
]
Then, update the docstrings in example.py:
"""Utility functions."""
def my_function(a):
"""
My function.
Parameters
----------
a : int
The value to use.
"""
pass
The docstring checks are strict, but things like the test suite won't end up in our documentation, so we are creating extra work. Let's restrict when the docstring checks will run:
Tip: Inclusive filtering on the file name is also supported with
files. File type filtering is also available. See the
pre-commit documentation
for all supported options.
Run pre-commit autoupdate to update all hooks to their
latest versions (the rev key in the YAML):
Warning: Make sure these new versions work with your setup by
running pre-commit run --all-files.
--no-verify when committing to bypass the
checks.
pre-commit run --all-files to run your hooks in
CI/CD workflows (or you can use
pre-commit.ci).
Only allow committing one file at a time:
exif-stripperThis hook removes metadata from images, and the check is implemented to run on a single file at once.
Simplified to only strip EXIF data.
Source:
stefmolin/exif-stripper
Come up with your own check. It doesn't matter if something already exists for your idea. If you can't think of anything, try creating a check that enforces a file naming convention that you follow.
Write your code in the
src/your_pkg/your_module.py file, renaming both the
directory and file as you see fit.
Require filenames to be at least three characters (without the extension):
Contents of
src/filename_validation/validate_filename.py
We will use argparse for this example:
At a minimum, we need to accept filenames:
We can then parse the received arguments:
Run your check(s) on each item in args.filenames:
Return 1 if there were any failures, 0 otherwise:
exif-stripper
Create the ArgumentParser:
Parse the received arguments:
Process each file, storing the outcomes (these are Boolean values):
Aggregate the results, returning 1 if there were any failures, 0 otherwise:
Source:
stefmolin/exif-stripper
Create a CLI for the check function you created in the previous exercise. Add an optional command line argument to modify how your check behaves.
You can put this in the same file from the previous exercise or in
another file in the same directory (e.g.,
src/your_pkg/cli.py).
Here, we continue with the hook requiring filenames of a certain length:
Most of this is identical to the example:
But, this time we add the minimum length as an optional argument:
When we run the checks on each file, we also pass it in:
Contents of src/filename_validation/cli.py
exif-stripperLet's look at the file structure for a Python package:
For this example, the repository name is
exif-stripper:
Configuration file to have pre-commit run hooks on
our code:
Configuration file for our new hook (for the
strip-exif hook, here):
Important files for any codebase:
This package uses the src-layout:
Tests for the logic to make this easier to maintain:
The pyproject.toml file is used to install this:
Based on
stefmolin/exif-stripper
pyproject.toml
This is a template pyproject.toml file:
Name your distribution (likely the name of your repository):
Provide a version (can also be done dynamically):
Provide information about your project:
Include all dependencies required to use your hook here:
Optional dependencies for development of the hook go here, instead:
Creating a package is recommended for testing and reuse:
Create an executable for calling your hook's CLI:
Based on
stefmolin/pre-commit-example
When the exif_stripper package is installed, an entry
point (executable) called, strip-exif, is created
automatically, which when run calls the main() function
in the exif_stripper.cli module:
scripts.strip-exif = "exif_stripper.cli:main"
Source:
stefmolin/exif-stripper
Using the provided template, populate the
pyproject.toml file for your hook from the previous
exercise. Be sure to create an entry point that will hit the CLI.
Confirm that you can pip install it (note that you will
need to create any files referenced in the
pyproject.toml, such as README.md).
Configuration goes in
.pre-commit-hooks.yaml as a list of hooks:
The hook ID (we can use this to run just this hook):
Display name for the hook when the checks are run:
(Optional) Description of what your hook does:
The entry point or executable to run, which can also include arguments:
The language your hook is written in (so
pre-commit can install it):
(Optional) Restrict this hook to only run on certain file types:
Source:
stefmolin/exif-stripper
Create a .pre-commit-hooks.yaml file for configuring
your hook from the previous exercise. Note that, by default, hooks
will run in parallel, if you need to run in serial, specify
require_serial: true. Be sure to consult
pre-commit.com/#creating-new-hooks
for a listing of the configuration options.
Tip: You can run the following to check for any formatting errors:
$ pre-commit validate-manifest .pre-commit-hooks.yaml
Continuing the example from previous exercises, we can use the
following as our .pre-commit-hooks.yaml file for the
hook to validate the length of the filename:
- id: validate-filename
name: validate-filename
description: This hook checks filename length.
entry: validate-filename
language: python
pre-commit try-repo. (pre-commit.com/#developing-hooks-interactively)
.pre-commit-config.yaml file in another repository,
and set the rev value to the most recent commit hash.
Then, you can run your hook with pre-commit run.
Test that your hook is properly configured.
Start by creating a file that will fail the check:
Run pre-commit try-repo passing in the file:
Now, pre-commit will generate a configuration for us:
Think of this as the .pre-commit-config.yaml file
that will be used:
The repo points to the path we passed to
pre-commit try-repo:
The rev is set to the most recent commit hash:
The hooks are pulled out of .pre-commit-hooks.yaml:
Just like before, pre-commit will need to set up the
environment:
As expected, x.py fails the
validate-filename check:
There are several points of failure when building and maintaining a hook, so it is best to set up testing in layers:
pre-commit to run your hooks in your CI/CD
workflows.
Tip: See the references slide at the end for some examples.
Make it easy for people to get started with your hook by including a
setup snippet in your README. If you have command line
arguments and/or support configuration files, mention some common
configurations. Note that you should create tags so people don't
have to use commit hashes.
Source:
stefmolin/exif-stripper
pyproject.toml.
pre-commit (see the list at
pre-commit.com/#supported-languages).
exif-stripper
pytest:
test_cli.py
ci.yml
pyproject.toml (from the
numpydoc-validation
hook)
I hope you enjoyed the workshop. You can follow my work on the following platforms:
