Pebble: threads and processes without the boilerplate

One decorator, three execution models

Pebble provides a neat API to manage threads and processes within an application, and the readme gets to the point in a handful of lines.

The basic case runs a job in a separate thread and waits for its results:

    from pebble import concurrent

    @concurrent.thread
    def function(foo, bar=0):
        return foo + bar

    future = function(1, bar=2)

    result = future.result()  # blocks until results are ready

The second example is the same code with asyncio support, where the decorator comes from the asynchronous module and the call is awaited:

    import asyncio

    from pebble import asynchronous

    @asynchronous.thread
    def function(foo, bar=0):
        return foo + bar

    async def asynchronous_function():
        result = await function(1, bar=2)  # blocks until results are ready
        print(result)

    asyncio.run(asynchronous_function())

The third adds the thing the standard library does not give you directly, a timeout on the job itself, and the error handling that goes with it:

    from pebble import concurrent
    from concurrent.futures import TimeoutError

    @concurrent.process(timeout=10)
    def function(foo, bar=0):
        return foo + bar

    future = function(1, bar=2)

    try:
        result = future.result()  # blocks until results are ready
    except TimeoutError as error:
        print("Function took longer than %d seconds" % error.args[1])
    except Exception as error:
        print("Function raised %s" % error)
        print(error.traceback)  # traceback of the function

Three things are worth noticing in that last block. The timeout is a parameter on the decorator rather than something you pass at call time, so it is a property of the job definition. The exception carries the traceback of the function, as an attribute, which is the part that matters when the failure happened in another process. And the process variant is the one that gets killed on timeout, which is why the timeout lives in the decorator rather than on the future.

Pools with max_tasks, restarts and done callbacks

The pool API is where Pebble goes further than the standard library, and the example in the readme shows the three features that matter.

    from pebble import ProcessPool
    from concurrent.futures import TimeoutError

    TIMEOUT_SECONDS = 3

    def function(foo, bar=0):
        return foo + bar

    def task_done(future):
        try:
            result = future.result()  # blocks until results are ready
        except TimeoutError as error:
            print("Function took longer than %d seconds" % error.args[1])
        except Exception as error:
            print("Function raised %s" % error)
            print(error.traceback)  # traceback of the function

    with ProcessPool(max_workers=5, max_tasks=10) as pool:
        for index in range(0, 10):
            future = pool.schedule(function, index, bar=1, timeout=TIMEOUT_SECONDS)
            future.add_done_callback(task_done)

The pool is used as a context manager, so it shuts down when the block ends. The two constructor arguments set the shape of the pool: max_workers is how many processes run at once, and max_tasks is how many tasks a single worker handles before it is recycled. That second parameter is the interesting one. A worker that has run ten tasks is replaced, which means a long lived pool does not accumulate state in a worker process indefinitely, and it means a worker that has died is quietly replaced rather than taking the pool down with it.

The documentation summary says pools support workers restart, timeout for long running tasks and more, and the example shows the third of those, the per schedule timeout. The timeout here is passed to the schedule call rather than the decorator, so it is per task rather than per function, which is what you want in a pool where different tasks have different budgets.

The done callback is the completion handling. It receives the future, and because it is a callback rather than a try block around a result call, it fires for every scheduled task without you having to write the error handling again each time.

Why not just use concurrent.futures

The standard library has a process pool, so the question for any reader is why add a dependency. The readme does not make the argument for you explicitly, but the example above answers it.

Three capabilities stand out. First, the timeout on a process job. In concurrent.futures, a ProcessPoolExecutor cannot cancel a task that has already started, so a job that hangs holds a worker forever unless you shut the executor down. Pebble's decorator timeout kills the offending process, and the future raises with the traceback attached.

Second, max_tasks and worker restart. The standard pool has no equivalent, so a worker process that has grown large or has picked up bad state stays in the pool. Pebble recycles it on a schedule.

Third, the asyncio flavour. The asynchronous module lets the same decorated function be awaited from a coroutine, and the readme points out that it supports asyncio. The standard library's executor approach requires wrapping the future in an asyncio future and running the blocking call in a thread anyway, which reintroduces the thread pool you were trying to avoid.

The trade-off runs the other way as well. Pebble is a third party library, with LGPL licensing and its own API to learn, where concurrent.futures is in the standard library and its semantics are guaranteed. A team with a straightforward fan-out workload and no problematic tasks should use the standard library, and a team with jobs that hang, or workers that need recycling, has a reason to look here.

Where a decorator on a process changes the failure model

The shift from a thread to a process is invisible in the API and very visible in behaviour, and it is worth spelling out because it is where most of the surprises in a library like this come from.

In a thread, an exception raised in the job propagates to the caller and the thread is reusable. In a process, the job runs in a separate interpreter, so an exception has to be transported back across the process boundary, and that transport is what the traceback attribute on the exception is for. The readme's example prints it explicitly, which is the correct thing to do when the failure happened somewhere else.

A timeout has the same character. A thread that hangs holds a thread; a process that hangs holds a process and the memory that goes with it. Killing it is the only way to reclaim that, which is why the timeout lives on the decorator where it is a property of the job rather than something you might forget at a call site.

The flip side is that a process pool gives you real parallelism rather than concurrency. Work that is CPU bound and pure Python gets the benefit of running on more than one core, which no amount of threading will give you in CPython, because the GIL holds the lock. Work that is IO bound does not need a process at all, and paying the cost of a process for it is usually the wrong call.

That is the practical decision the library is helping you make, and the fact that both a thread and a process decorator take the same shape means you can try the cheaper option first and change one word when you find the job is genuinely CPU bound.

A small package with a real release history

The packaging is plain, and the release history suggests a project with users rather than a weekend experiment.

The setup file is a classic setuptools one. The version is read out of the package source at build time by scanning the lines of the package's init file for the version assignment, so the version lives in one place rather than being duplicated in the manifest. The package data includes a py.typed marker, which is what makes the library typed for consumers running a type checker, and the package is marked as not zip safe. The Python requirement is 3.10 or newer, and the classifier lists the development status as production or stable, along with the LGPL identifier.

The licence is LGPL, and the setup file records it as the GNU Lesser General Public License. The author is Matteo Cafasso, and the description in the manifest matches the readme's own phrase about threading and multiprocessing eye-candy, with keywords covering thread, process, pool and decorator.

The release cadence is steady. The three most recent versions are 5.2.3 from 2026-10-04, 5.2.2 from 2026-09-04 and 5.2.1 from 2026-07-19. The repository is not archived, the default branch is main, and the last push was on 2026-10-10, six days before the current date.

The repository layout is small and conventional for a Python package: a pebble/ directory for the code, a test/ directory, a doc/ directory, a MANIFEST.in, a .pre-commit-config.yaml, a .readthedocs.yaml for the documentation build, and a README.rst, which is what the setup file reads for the long description. The documentation is published at pebble.readthedocs.io and the package on PyPI, with the source on GitHub.

Editorial conclusion

Pebble suits a Python developer who wants a process pool with real timeouts and worker restarts without writing the supervision logic, or who wants the same decorator to work under asyncio. It does not suit a project that needs the standard library only, and the pool semantics differ enough from concurrent.futures that you should read the documentation before swapping one for the other. Before adopting it, check that the timeout and error semantics match what your code expects, because a timeout in a process job kills the worker rather than raising in the caller, and note that the last push was on 2026-10-10 with the 5.2.3 release on 2026-10-04.

Frequently asked questions

Official sources

1. Issues
2. License: LGPL-3.0
3. noxdafox/pebble on GitHub
4. README
5. Releases