Process, Analyze, and Transform Python Code with ASTs
You've likely used a tool like black, flake8, or ruff to lint or format your code,
or a tool like sphinx to document it, but you probably do not know how they accomplish
their tasks. These tools and many more use Abstract Syntax Trees (ASTs) to analyze and
extract information from Python code. An AST is a representation of your code's structure
that enables you to access and manipulate its different components, which is what makes it
possible to automate tasks like code migrations, linting, and docstring extraction.
In this workshop, you'll learn how to use the Python standard library's ast module to
parse and analyze code. Using just the standard library, we will implement a couple of
common checks from scratch, which will give you an idea of how these tools work and help you
build the skills and confidence to use ASTs in your own projects.
Workshop outline
This section will introduce attendees to abstract syntax trees (ASTs) and some of the ways they are used in the Python ecosystem. We will learn how to parse source code into an AST, what that structure looks like, and understand the situations in which we cannot generate an AST.
Due to the structure of the AST, we must traverse it to
process, analyze, and transform it. In this section, we will learn how to perform the
traversal using ast.walk(), ast.NodeVisitor, and ast.NodeTransformer to conduct
some common linting tasks.
Tracking context during the AST traversal makes it possible to use information about a node's ancestry (as well as its descendants) to make decisions about how to process it. In this final section, we will build an import linter capable of flagging unused imports, along with masked and missing names, by tracking context (e.g., scope, times accessed, etc.) during traversal.
Live sessions
I have presented this workshop 2 times at conferences around the world. Click a location on the map to see the conference(s) I have presented this workshop at.
Click a on the map to see more information.