Bazel PyO3 Rust-Python Integration

This note ties together concepts from the other Bazel notes into a concrete use case: building Rust code via PyO3 into a shared library that Python can import.

What We’re Building

The goal: write Rust code using PyO3, compile it into a .so file, and have import my_module work from Python — all managed by Bazel. This involves two Bazel rule sets (rules_rust and rules_python) that don’t know about each other, stitched together with a macro or custom rule.

rules_rust: Compiling Rust

rules_rust provides several rules for different Rust artifact types. The one we care about is rust_shared_library, which produces a cdylib — a C-compatible shared library. This is what Python’s extension module loading mechanism (dlopen()) expects.

When Bazel analyzes a rust_shared_library target, the rule implementation:

1. Resolves the Rust toolchain to get the rustc binary and standard library
2. Reads CrateInfo providers from each dependency to learn their compiled output paths
3. Declares an action that runs rustc --crate-type=cdylib with the right --extern flags, inputs, and output
4. Returns its own CrateInfo (for other Rust targets), DefaultInfo (universal), and CcInfo (for C/C++ interop)

Proc macros

PyO3 heavily uses proc macros (#[pyfunction], #[pymodule], #[pyclass]). These are Rust compiler plugins — code that runs inside rustc during compilation to transform your code. Because they run on the build machine (not the target machine), they must be compiled for the execution platform (see exec vs target). In rules_rust, this happens automatically: proc_macro_deps are built with cfg = "exec". The generated BUILD files from crate_universe handle this correctly.

rules_python: Hermetic Python

rules_python provides:

• Hermetic Python interpreters — Bazel downloads and manages CPython. No dependency on system Python.
• Core rules: py_library, py_binary, py_test
• pip integration for PyPI packages

The Python toolchain gives your rule access to the interpreter binary and headers:

python_tc = ctx.toolchains["@rules_python//python:toolchain_type"]
py_runtime = python_tc.py3_runtime
py_runtime.interpreter  # the python3 binary
py_runtime.headers      # Python.h and friends — needed by PyO3 at compile time

The Three Problems to Solve

Problem 1: The .so filename

rust_shared_library produces libmy_module_rs.so by default. Python expects a very specific name:

• Version-specific: my_module.cpython-311-x86_64-linux-gnu.so
• Stable ABI: my_module.abi3.so (works across Python versions — simpler)

If the filename is wrong, import my_module silently fails with ModuleNotFoundError. You need to rename the output. The simplest approach is a genrule:

genrule(
    name = "my_module_rename",
    srcs = [":my_module_rs"],
    outs = ["my_module.abi3.so"],
    cmd = "cp $< $@",
)

For a more robust solution (computing the platform-specific suffix from the Python toolchain at Analysis time), you’d write a custom rule instead of using a macro, since macros can’t access toolchains.

Problem 2: The extension-module feature

On Linux, Python extension modules get Python symbols at runtime from the interpreter process itself (loaded via dlopen() with RTLD_GLOBAL). If the extension also links libpython.so, you get duplicate symbols and crashes. The extension-module Cargo feature tells PyO3 to skip linking libpython.

Platform differences:

• Linux: don’t link libpython (the extension-module feature handles this)
• macOS: need -undefined dynamic_lookup linker flags
• Windows: must link against python3.lib

The Python Stable ABI (abi3) is worth using: it makes the extension work across Python 3.x versions without recompilation, and simplifies the output filename to my_module.abi3.so.

# Cargo.toml
pyo3 = { version = "0.20", features = ["extension-module", "abi3-py311"] }

Problem 3: Making the .so findable at runtime

Python needs the .so on sys.path. Bazel needs to put it in the runfiles tree — the set of files available when a program runs (as opposed to files produced during the build). Two mechanisms work together:

1. The data attribute on py_library adds files to runfiles:

   py_library(name = "my_module", data = [":my_module_rename"])

2. The imports attribute adds directories to sys.path:

   py_library(name = "my_module", data = [...], imports = ["."])

   imports = ["."] says “add this package’s directory to sys.path.” Since the .so is in that directory (via data), import my_module finds it.

The Complete Solution

Macro

# tools/pyo3_module/defs.bzl
def pyo3_module(name, srcs, module_name, deps = [], visibility = None, **kwargs):
    rs_name = name + "_rs"
    so_name = module_name + ".abi3.so"
 
    rust_shared_library(
        name = rs_name,
        srcs = srcs,
        deps = deps + ["@crate_index//:pyo3"],
        edition = "2021",
        visibility = ["//visibility:private"],
    )
 
    native.genrule(
        name = name + "_rename",
        srcs = [":" + rs_name],
        outs = [so_name],
        cmd = "cp $< $@",
        visibility = ["//visibility:private"],
    )
 
    py_library(
        name = name,
        data = [":" + name + "_rename"],
        imports = ["."],
        visibility = visibility,
        **kwargs,
    )

Usage

# rust/my_module/BUILD.bazel
pyo3_module(
    name = "my_module",
    srcs = glob(["src/**/*.rs"]),
    module_name = "my_module",
    visibility = ["//python:__subpackages__"],
)

# python/my_app/BUILD.bazel
py_binary(
    name = "app",
    srcs = ["app.py"],
    deps = ["//rust/my_module:my_module"],
)

How it all connects

py_binary("app")
  └── py_library("my_module")           ← public API, provides runfiles + sys.path
        └── genrule("my_module_rename") ← renames .so to correct filename
              └── rust_shared_library("my_module_rs")  ← compiles Rust to cdylib
                    └── @crate_index//:pyo3            ← resolved by crate_universe

Data flow at runtime: the .so propagates upward via DefaultInfo.runfiles (see cross-language bridge pattern). Python rule sets and Rust rule sets don’t know about each other — they communicate through DefaultInfo and file artifacts.

Common Pitfalls

Pitfall	Symptom	Fix
Wrong .so filename	ModuleNotFoundError	Verify filename matches Python’s expected pattern
Missing imports on py_library	ModuleNotFoundError (file exists but not on sys.path)	Add imports = ["."]
.so not in runfiles	Builds fine, fails at runtime	Use data attribute (auto-adds to runfiles)
Missing extension-module feature	Crashes at import from duplicate symbols	Add to PyO3’s Cargo features
ABI mismatch	Crashes with different Python version	Use abi3 stable ABI