Uncovering concurrency issues, testing and debugging
Until now, the GIL has allowed developers to ignore C safety issues when writing parallel programs, since the GIL ensured that all thread execution was serialized, allowing for simultaneous access to Python objects and state defined in the interpreter.
The new free-threaded model ensures that Python code access originating from other Python code frames is safe and is guaranteed to not produce any major interpreter crash, as opposed to unrestricted C code access, which can present any of the common C thread-safety issues.
Usually, concurrency issues arise when two or more threads try to modify the same value in memory. In Python, this commonly occurs when a class or function defines native shared state, either via an attribute or a variable that can be modified from native code in each thread execution scope.
The most common issues related to concurrency in the context of free-threaded CPython extensions are either dirty reads/writes to global/shared C state, unexpected behavior due to simultaneous access to C calls that are not thread-safe, and finally, major runtime crashes due to memory allocation issues and forbidden pointer lookups. While the first case depends on the actual implementation of the algorithm/routine and may produce unintended results, it would not cause a fatal crash of the interpreter, as opposed to the last two cases.
In order to discover, handle and debug concurrency issues at large, there are several strategies, which we will summarize next.
Ensure that an extension module is free-threaded compliant
We highly suggest reading the detailed guide presented on Porting Extension Modules to Support Free-Threading
Testing scenarios
In order to check that a function or class has no concurrency issues, it is
necessary to define test functions that cover such cases. For such scenarios, the
standard threading
library defines several low-level parallel primitives that
can be used to test for concurrency, while the concurrent.futures
module
provides high-level constructs.
For example, consider a method MyClass.call_unsafe
that has been flagged as having concurrency issues since it mutates attributes
of a shared object that is accessed by multiple threads. We can write a test for
it using first low-level primitives:
"""test_concurrent.py"""
# Low level parallel primitives
import threading
# High level parallel constructs
from concurrent.futures import ThreadPoolExecutor
# Library to test
from mylib import MyClass
def test_call_unsafe_concurrent_threading():
# Defines a thread barrier that will be spawned before parallel execution
# this increases the probability of concurrent access clashes.
n_threads = 10
barrier = threading.Barrier(n_threads)
# This object will be shared by all the threads.
cls_instance = MyClass(...)
results = []
def closure():
# Ensure that all threads reach this point before concurrent execution.
barrier.wait()
r = cls_instance.call_unsafe()
results.append(r)
# Spawn n threads that call call_unsafe concurrently.
workers = []
for _ in range(0, n_threads):
workers.append(threading.Thread(target=closure))
for worker in workers:
worker.start()
for worker in workers:
worker.join()
# Do something about the results
assert check_results(results)
def test_call_unsafe_concurrent_pool():
# Defines a thread barrier that will be spawned before parallel execution
# this increases the probability of concurrent access clashes.
n_threads = 10
barrier = threading.Barrier(n_threads)
# This object will be shared by all the threads.
cls_instance = MyClass(...)
def closure():
# Ensure that all threads reach this point before concurrent execution.
barrier.wait()
r = cls_instance.call_unsafe()
return r
with ThreadPoolExecutor(max_workers=n_threads) as executor:
futures = [executor.submit(closure) for _ in range(n_threads)]
results = [f.result() for f in futures]
# Do something about the results
assert check_results(results)
Given the non-deterministic nature of parallel execution, such tests may pass
from time to time. In order to reliably ensuring their failure under concurrency,
we recommend using pytest-repeat
, which enables the --count
flag in the
pytest
command:
# Setting PYTHON_GIL=0 ensures that the GIL is effectively disabled.
PYTHON_GIL=0 pytest -x -v --count=100 test_concurrent.py
We advise to set count
in the order of hundreds and even larger, in order to
ensure at least one concurrent clash event.
Debugging tests that depend on native calls
If your code has native dependencies, either via C/C++ or Cython, gdb
(or lldb
) can be used as follows:
# Setting PYTHON_GIL=0 ensures that the GIL is effectively disabled.
PYTHON_GIL=0 gdb --args python my_program.py --args ...
# To test under pytest
PYTHON_GIL=0 gdb --args python -m pytest -x -v "test_here.py::TestClass::test_method"
# Using LLDB (under LLVM/clang)
PYTHON_GIL=0 lldb -- $(which python) my_program.py
# Using LLDB (and pyenv)
PYTHON_GIL=0 lldb -- $(pyenv which python) $(pyenv which pytest) -x -v "test_here.py::TestClass::test_method"
When Python is run under gdb
, several Python integration commands will be
available, such commands start with the py-
prefix. For instance, the py-bt
allows to obtain a Python interpreter backtrace whenever the debugger hits a native
frame, this allows to improve the tracking of execution between Python and native
frames1.
For more information about gdb
and lldb
commands, we encourage reading
the GDB to LLDB command map page in the
official LLVM docs.
Cython debugging
Since Cython produces intermediate C/C++ sources that then are compiled into native
code, stepping through may get difficult if done solely from the C source file.
In order to get through such difficulty, Cython includes the cygdb
extension,
which enables gdb
to go through large sections of C code that are equivalent to
a single Cython declaration.
Enabling cygdb
requires the compilation of Cython sources with the --gdb
flag. After the sources are compiled and linked, it can be used as follows:
# For example, running the tests of scikit-image.
# build/cp313td/ contains the trace files generated by Cython to be compatible
# with cygdb
PYTHON_GIL=0 cygdb build/cp313td/ -- --args python -m pytest -x -v skimage/
Since cygdb
requires the Python interpreter version used to compile gdb
to match the one to be used during the execution of the script, recompiling gdb
will be necessary in order to ensure the most complete debugging experience.
We recommend the gdb
compilation instructions
provided by the Linux from scratch project.
cygdb
defines a set of commands prefixed by cy
that replace the usual gdb
commands. For example cy run
will start the program with the Cython debugging
extensions enabled, cy break
will define a breakpoint on a function with the
Cython definition name, cy next
will step over a Cython line, which is equivalent
to several lines in the produced C code.
Detecting issues in CPython
If a debugging session suggests that an error/bug is incoming from CPython,
we recommend installing a debug instance. The easiest way to accomplish this
is via pyenv
:
pyenv install --debug --keep 3.13t-dev
This command will not only install a debug distribution of CPython, but also
will ensure that the source files are kept as well, such files will be loaded
by gdb
/lldb
at the moment of debugging. For more information regarding
CPython installation sources, please visit the
Installing a free-threaded Python page.
Frequently seen errors and how to fix them
These are error messages that we see come up often when working with code or development workflows that have not been updated to accommodate the free-threaded build. We also provide suggested fixes. Please send in pull requests to the repository for this document if you run into any confusing free-threading-specific errors that you suspect apply to other libraries and aren't covered here.
Cython compilation errors: unknown type name '__pyx_vectorcallfunc'
This happens if you try to build a Cython extension for the free-threaded build
using the current stable release of Cython (3.0.10 at the time of writing). The
current stable release of Cython does not support the free-threaded build. You
must either build Cython from the master
branch on
Github or use the nightly wheel:
pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple cython
See the porting guide for more detail about porting Cython code to work under free-threading.
You may wonder why you are able to install a wheel for the current Cython
release at all. This is because Cython ships a pure-python wheel tagged with
py2.py3-none-any
, which pip will install if it cannot find another wheel that
is compatible. A future version of Cython will ship a wheel with compiled code
that supports the free-threaded build.
The current nightly wheel is a pure-python build, so it works on all architectures. The pure-python version of Cython is usually only marginally slower than a compiled version, so you should default to installing the wheel in CI instead of compiling Cython, which can take up to a few minutes on some CI runners.
Cython compilation errors arising from static_assert
This error arises from a limitation in Cython that exposes a CPython implementation detail. Under clang, the issue manifests in error messages like the following example, generated by building scikit-learn with a C99 compilation environment:
In file included from sklearn/__check_build/_check_build.cpython-313t-darwin.so.p/sklearn/__check_build/_check_build.pyx.c:4315:
In file included from /path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_frame.h:13:
/path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_code.h:537:15: error: expected parameter declarator
static_assert(COLD_EXIT_INITIAL_VALUE > ADAPTIVE_COOLDOWN_VALUE,
^
/path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_backoff.h:125:33: note: expanded from macro 'COLD_EXIT_INITIAL_VALUE'
#define COLD_EXIT_INITIAL_VALUE 64
^
In file included from sklearn/__check_build/_check_build.cpython-313t-darwin.so.p/sklearn/__check_build/_check_build.pyx.c:4315:
In file included from /path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_frame.h:13:
/path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_code.h:537:15: error: expected ')'
/path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_backoff.h:125:33: note: expanded from macro 'COLD_EXIT_INITIAL_VALUE'
#define COLD_EXIT_INITIAL_VALUE 64
^
/path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_code.h:537:14: note: to match this '('
static_assert(COLD_EXIT_INITIAL_VALUE > ADAPTIVE_COOLDOWN_VALUE,
^
/path/to/python/3.13.0b3t/include/python3.13t/internal/pycore_code.h:537:1: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
static_assert(COLD_EXIT_INITIAL_VALUE > ADAPTIVE_COOLDOWN_VALUE,
^
int
Free-threading support in Cython relies CPython internals. In particular, Cython
will generate code that in some situations includes an internal CPython header,
pycore_frame.h
. This header, in turn, includes another header pycore_code.h
that makes use of static_assert
, a construct defined in the C11 standard. This
is problematic for a compiler assuming compliance with C99.
In practice we have found the easiest way to fix this is to update the standard used to build C code to either C11 or C17. In the meson build configuration for a simple C project, this looks like:
project(
'my-project',
'c',
default_options: [
'c_std=c17',
],
)
Note that C17 corrected defects, removed optional requirements, and did not add any new features compared with C11, so requiring C11 support may be more onerous than C17. The CPython codebase currently assumes a compiler with at least partial C11 support, and we have found for all major compilers (including MSVC) that updating the assumed C standard is not a big problem as of mid-2024. There are some C11 features that MSVC and other compilers are missing, so please open an issue if this ends up being a big problem for your project or a subset of your userbase.
-
This feature is not correctly working on
lldb
after CPython 3.12. ↩