Type Stubs for Native Python Modules
For Kodiak we needed to find html comment positions inside a string of markdown. All the existing Python markdown parsers didn’t expose the necessary AST node positions, but the Rust package, pulldown-cmark did.
With a little more Rust code and pyo3, we get
a working Python module as a .whl
.
The only downside is that pyo3 doesn’t generate type stubs (.pyi
files) for the package and since the module
is a binary, using the module can’t be type checked.
So what can we do?
-
Write the type stub by hand
For
markdown-html-finder
it looks like this:markdown_html_finder/__init__.pyi
from typing import List, Tuple def find_html_positions(markdown: str) -> List[Tuple[int, int]]: ...
-
Add the type stub to the generated
.whl
Wheel provides a CLI for unpacking and packing wheels.
./.venv/bin/wheel unpack markdown_html_finder-0.1.0.whl # need the py.typed file so type checkers will use types # see: https://www.python.org/dev/peps/pep-0561/#packaging-type-information touch markdown_html_finder/py.typed cp -R ./markdown_html_finder markdown_html_finder-0.1.0 ./.venv/bin/wheel pack markdown_html_finder-0.1.0
-
Install the updated
.whl
and using the module will now be type checked. 🎉
For a more extensive example, checkout the build script in the
markdown-html-finder
repo.