Library / SDK
ross/requests-futures avatar
ross/requests-futures

Eight workers, one shared Session: inside requests-futures

Asynchronous Python HTTP Requests for Humans using Futures

2,222 stars158 forksPythonNOASSERTION

At a glance

What is it?
requests-futures wraps the python requests library in concurrent.futures, so the same call sites return a Future instead of a Response. The wrapper stays deliberately small, and its default 8-worker pool shares one Session object that requests itself does not treat as thread-safe.
Who is it for?
requests-futures is the right pick when your call sites already speak requests and the fan-out is simple: dozens of independent GETs whose bodies you resolve through result(). Leave it out of code that streams large responses, that shares a cookie jar or session.headers across in-flight calls without giving each worker its own Session, or that needs subprocesses without module-scope classes.
Can I use it commercially?
Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
Is it still maintained?
Yes. The repository last received commits 18 days ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on October 20, 2026, and from our analysis. They are not legal advice.

Editorial analysis

FuturesSession returns a Future instead of a Response

requests-futures keeps the shape of the requests library, requires Python 3.10 or newer, and changes one thing at the call site. A `FuturesSession` built from `requests_futures.sessions` exposes the same verbs as `requests.Session`, and every call hands back a `Future`. Nothing about URL handling, the response object or the status code changes. What changes is that the call stops blocking, so you collect futures and resolve them when you want the payload, normally through `result()`.

That shift has one consequence worth stating plainly: exceptions move. A connection error or a timeout is no longer raised where the call was made. It is thrown at the moment the future is resolved, so an exception handler wrapped around the call site does not catch it and the same handler wrapped around `result()` does. Anything you had guarded per call has to be relocated to the resolve step, and a return position that used to hold a status code now holds a future object.

Eight workers share one Session by default

A session creates a `ThreadPoolExecutor` with 8 workers by default, and that default executor holds a single `requests.Session` shared across every worker thread. The README states plainly that `Session` is not thread-safe, and that mutating shared state such as the cookie jar or `session.headers` from several in-flight requests is a real hazard.

Read further and the README contradicts itself in a way worth knowing before you adopt it. The same document claims the api of requests.Session is preserved without modifications beyond returning a future rather than a response, then warns that the shared object is unsafe to touch from all 8 threads at once. An unchanged API surface is not unchanged runtime behaviour, and the default configuration hands you the unsafe arrangement. Nothing in the package removes that exposure. The documentation index points to a full thread-safety discussion for what to do instead, and separately lists sizing `max_workers` against the connection pool, including with a supplied `session=`.

Reading the first responses with as_completed

No install command appears in the README, so start from the packaging metadata rather than a shell transcript. pyproject.toml names the distribution, declares the floor for the interpreter, and lists exactly one runtime dependency.

toml
name = "requests-futures"
description = "Asynchronous Python HTTP for Humans."
requires-python = ">=3.10"
dependencies = [
    "requests>=1.2.0",
]

The first real use is two GET calls against httpbin followed by a resolve of each future in turn. Both calls return immediately in submission order, and the resolve line is where each response is actually waited for, yielding a normal Response you can read exactly as in synchronous code.

python
from requests_futures.sessions import FuturesSession

session = FuturesSession()
# first request is started in background
future_one = session.get('http://httpbin.org/get')
# second requests is started immediately
future_two = session.get('http://httpbin.org/get?foo=bar')
# wait for the first request to complete, if it hasn't already
response_one = future_one.result()
print('response one status: {0}'.format(response_one.status_code))
print(response_one.content)

Two prints report the status code and body of each call. If you prefer results in completion order, collect the futures into a list and walk it with `as_completed`, attaching your own metadata by setting an attribute on the future, for example `future.i = i`, then reading that attribute back inside the loop.

Leaving the with block cancels queued requests

Fan-out code tends to leak work. When a loop submits more requests than it needs answers for, the queued futures keep occupying workers long after the results you wanted have landed. The session context manager exists for that case.

python
from requests_futures.sessions import FuturesSession
with FuturesSession(max_workers=1) as session:
    future = session.get('https://httpbin.org/get')
    future2 = session.get('https://httpbin.org/delay/10')
    future3 = session.get('https://httpbin.org/delay/10')
    response = future.result()

Leaving the block calls `close()`, which shuts down the session's own executor with `cancel_futures=True`, so every queued request that never started is cancelled in one sweep rather than skipped one at a time. In this trace `max_workers=1` means the first call has finished when the block exits, while the two delay paths never ran, so both are cancelled together. One boundary is worth keeping in mind: a request already running when `close()` fires is left to finish instead of being killed.

ProcessPoolExecutor brings a pickling contract with it

A `ProcessPoolExecutor` can be handed to the session in place of the thread pool. The README frames that as useful when per-request memory is heavy enough that cycling the interpreter is needed to hand memory back to the operating system.

python
from concurrent.futures import ProcessPoolExecutor
from requests_futures.sessions import FuturesSession

session = FuturesSession(executor=ProcessPoolExecutor(max_workers=10))
# ... use as before

The cost is a serialization contract with no counterpart under threads. Everything submitted has to be picklable, and any `FuturesSession` subclass used this way must be importable at module scope, because a class defined inside a function or a `__main__` script body cannot be imported by the child process. When something in the request cannot be pickled, `FuturesSession` raises a `RuntimeError` pointing back to the documentation rather than letting the raw pickling failure surface. That makes the process pool a real alternative with a different approach, and a narrower one, but the README does not compare the wrapper against any async HTTP client, so the executor you supply is the only trade-off it evaluates.

Where pyproject.toml and the repository page disagree

Two of this project's own files describe it differently. GitHub's classification of the licence is Other, described as a custom licence it cannot classify with a pointer to the LICENSE file, while pyproject.toml declares `Apache-2.0` and adds `license-files` pointing at that same LICENSE file. Read the file itself before deciding which terms apply to your use.

toml
license = "Apache-2.0"
license-files = ["LICENSE"]
requires-python = ">=3.10"
dynamic = ["version"]

The drift shows up in one line of text too. The README title reads "Asynchronous Python HTTP Requests for Humans", the repository description appends "using Futures", and the pyproject description field drops the word Requests altogether. Nothing functional hangs on it, but it shows the three strings a packager reads have drifted apart. At the repository root the surrounding tooling is visible too: `pyproject.toml`, `requirements.txt`, `docs/`, `tests/`, `script/`, `requests_futures/`, `AGENTS.md`, `MANIFEST.in` and `.git_hooks_pre-commit`. The last push was on 2026-09-07 and the repository has no GitHub releases.

The documentation index names the failure modes

The README closes with a short index of what lives in the full documentation, and that index is the clearest statement of where this design strains. The entries name `as_completed` with a timeout, error handling across the future boundary (submit-time versus `result()`-time), retries through a mounted `HTTPAdapter` and urllib3's `Retry`, sizing `max_workers` against the connection pool including with a supplied `session=`, streaming responses with `stream=True` and why that interacts badly with background work, then a final line on sharing one executor that the document cuts off mid-word.

Four of those entries are hazards rather than features. A timeout that governs the wait rather than the request is a different contract from the one requests users expect. Retries have to be bolted on through an adapter because the wrapper does not retry on your behalf. Streaming is called out as interacting badly with background work, which rules it out for large downloads. And the connection pool stays finite, so adding workers past the connection limit converts queueing into latency instead of throughput.

Pinned dependencies across five Python versions

Upgrade cost here is mostly a lockfile question, and the answer sits in `requirements.txt`. That file is fully pinned, carries a header telling you not to edit it by hand, and opens with the environment marker `x-python-version-not-supported`, which the project uses to fail outside the interpreters it targets. Those five are 3.10 through 3.14, the same versions listed as classifiers in pyproject.toml and as target versions for the formatter.

The pin worth watching is the runtime dependency. pyproject.toml only asks for `requests>=1.2.0`, while the development file pins `requests==2.34.2`, so the floor your users can install is far below the version the maintainers build against. Alongside it, `httpx==0.28.1` is pinned next to `requests==2.34.2` in the same development environment, and the test extra pulls a local httpbin service.

toml
test = [
    "pytest>=6.2.5",
    "pytest-cov>=3.0.0",
    "pytest-httpbin>=2.0.0",
]

Regenerate that pinned list with `./script/update-requirements` rather than editing it, and keep your own pin above the floor if you depend on session behaviour introduced after 1.2.0. The build asks for setuptools 77 with the `setuptools.build_meta` backend, the formatter is pinned between black 26.0.0 and 27.0.0, and its configuration sets a line length of 80 with string normalisation disabled, which matches the single-quoted literals in this project's own examples.

Editorial conclusion

requests-futures is the right pick when your call sites already speak requests and the fan-out is simple: dozens of independent GETs whose bodies you resolve through result(). Leave it out of code that streams large responses, that shares a cookie jar or session.headers across in-flight calls without giving each worker its own Session, or that needs subprocesses without module-scope classes. Before adopting it in a service, check how your own error handling treats submit-time failures, because this wrapper moves them to the future boundary.

Frequently asked questions

Does requests-futures need Python 3.10 or newer?

Yes. The project requires Python 3.10 or newer, and pyproject.toml sets the interpreter floor alongside classifiers for 3.10 through 3.14. Anything outside that range trips the x-python-version-not-supported marker in requirements.txt.

How many requests run at once in requests-futures?

A ThreadPoolExecutor is created with 8 workers by default. You can set the count with max_workers, or pass an executor of your own, such as ProcessPoolExecutor(max_workers=10).

Is the shared requests Session in requests-futures safe across threads?

Not by default. The default executor shares one requests.Session across all worker threads, and Session is not thread-safe, so mutating the cookie jar or session.headers from several in-flight requests is a hazard. The documentation points to a thread-safety discussion for the alternative.

Where do I get requests-futures and how is it installed?

The package source sits in requests_futures/ and pyproject.toml defines the build with setuptools and the setuptools.build_meta backend. The README gives no install command, so the packaging metadata is the only installation detail recorded.

Can I use ProcessPoolExecutor with requests-futures?

Yes, a ProcessPoolExecutor can be supplied in place of a ThreadPoolExecutor. Everything submitted must be picklable and any FuturesSession subclass used that way must be importable at module scope; unpicklable objects raise a RuntimeError that points back to the documentation.

How do I cancel requests that have not started in requests-futures?

Use the session as a context manager. Leaving the with block calls close(), which shuts down the session's executor with cancel_futures=True so queued requests are cancelled together, while a request already running is left to finish.

Official sources

  1. Issues
  2. README
  3. ross/requests-futures on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/ross-requests-futures.svg)](https://hysenlabs.com/projects/ross-requests-futures)

Was this useful?