hiredis-py: a C reply parser bolted onto Python

A parser that never touches the socket

The scope of this library is narrow, and the README is explicit about the boundary. The hiredis module contains the Reader class, that class is responsible for parsing replies from the stream of data read from a Redis connection, and it does not contain functionality to handle I/O.

That single sentence defines the project. You feed bytes in, you get Python objects out, and the connection management stays with whatever else you are using. In practice that means redis-py or another client owns the socket, and hiredis-py is a pluggable parser inside it. The repository description says the same thing in two words: Python wrapper for hiredis, where hiredis is the C library that carries the protocol parsing code.

The primary target, in the README's phrasing, is speeding up the parsing of multi bulk replies, which is the RESP form used for lists, sets, hashes and arrays. A single inline string is cheap to parse whatever parser you use, which is why the benchmark numbers later on are almost flat for SET and GET and steep for long LRANGE results.

The project lives in the redis GitHub organisation, ships on PyPI under the name hiredis, and is written as a C extension alongside the Python glue. The build configuration in setup.py compiles the extension from src/ together with the vendored hiredis sources under vendor/hiredis, naming alloc, async, hiredis, net, read, sds and sockcompat.

Three lines to parse your first reply

Installation is a single pip command, since the package is on PyPI:

pip install hiredis

The usage example in the README shows the whole lifecycle. You construct a Reader, feed it a chunk of the RESP stream, and pull replies out of it:

>>> reader = hiredis.Reader()
>>> reader.feed("$5\r\nhello\r\n")
>>> reader.gets()
b'hello'

The mechanics are worth understanding because they are what every higher level client has to respect. Reader.feed takes a string argument that is appended to an internal buffer. Reader.gets reads that buffer and returns a reply when the buffer contains a full reply. If a single call to feed contains multiple replies, gets has to be called multiple times to extract all of them.

When the buffer does not contain a full reply, gets returns False. That is the signal that more data is needed and feed should be called again before calling gets again. A parser returning False as a normal, expected value rather than raising is a design choice that propagates into every call site, and it is why a client cannot simply treat every gets result as a reply.

Also note what is not in the example: no connection, no timeout, no retry. The Reader is a state machine over bytes, and everything around it belongs to the client library you already have.

A sentinel object for partial buffers and RESP3

The False return value has a cousin. The README describes an alternative where you provide a custom sentinel object via a parameter, and calls it out as useful for the RESP3 protocol where native boolean types are supported. The reason is obvious once you look at the two representations: a real RESP3 boolean reply and the parser's not enough data marker would otherwise be indistinguishable.

Here is the pattern in full, including the case of a feed that carries only part of a multi bulk reply:

>>> reader.feed("*2\r\n$5\r\nhello\r\n")
>>> reader.gets()
False
>>> reader.feed("$5\r\nworld\r\n")
>>> reader.gets()
[b'hello', b'world']
>>> reader = hiredis.Reader(notEnoughData=Ellipsis)
>>> reader.gets()
Ellipsis

The first feed carries the array header and only the first element, so there is no complete reply to hand back. The second feed completes it and gets returns the assembled list of two byte strings.

The last two lines swap the marker for Ellipsis and show the same incomplete state reported differently. A sentinel chosen by the caller is a small thing that removes a class of bug, because a boolean reply can then be distinguished from a truncated buffer without type sniffing.

Unicode decoding you configure once per Reader

By default the Reader returns bytes, which is what the first example shows with b'hello'. It can also decode bulk data into text, and the decoding settings are given when the Reader is initialised:

>>> reader = hiredis.Reader(encoding="utf-8", errors="strict")
>>> reader.feed(b"$3\r\n\xe2\x98\x83\r\n")
>>> reader.gets()
'☃'

Two keyword arguments drive this: encoding sets the codec, and errors sets the handler. Decoding of bulk data is attempted with the specified encoding and error handler. If the error handler is strict, which is the default, a UnicodeDecodeError is raised when the data cannot be decoded, and the README notes that this is identical to Python's default behaviour. Other valid values for errors include replace, ignore and backslashreplace, which are the standard Python handlers.

There is a second failure mode with a different exception type. When the specified encoding cannot be found, a LookupError is raised when calling gets for the first reply with bulk data. That is worth knowing because it surfaces at reply time rather than at construction time, so a typo in the encoding argument stays silent until data actually arrives.

Choosing the encoding up front also means the parser pays the decode cost in C rather than decoding each bulk string later in Python, which is part of why the extension exists at all.

ReplyError is returned, ProtocolError is raised

The two error paths behave differently, and confusing them is a common source of bugs in client code.

A protocol error occurs when the stream is corrupt, for example because multiple threads are using the same socket or some other condition breaks the stream. In that case hiredis.ProtocolError is raised. Because the buffer is read in a lazy fashion, it is only raised when gets is called and the first reply in the buffer contains an error. And there is no way to recover from a faulty protocol state, so the I/O code feeding data to the Reader should probably reconnect. That word probably is the README's, and it is the right amount of uncertainty: the parser cannot resynchronise a stream whose framing is broken.

Redis error replies, the -ERR kind, are handled the other way round. For those, the custom error class hiredis.ReplyError is returned, but not raised. The value comes back as the result of gets, and the caller decides what to do with it. That keeps existing except clauses working, since a client that expected a return value still gets one.

If you need different exception types, Reader can be initialised with the protocolError and replyError keywords. Those keywords take a class that is a subclass of Exception, and when they are not provided the Reader uses the default error types. So a codebase with its own error hierarchy can have the parser raise and return its own classes instead of hiredis.ProtocolError and hiredis.ReplyError.

The benchmark numbers, and what they actually measure

The repository contains a benchmarking script in the benchmark directory. It uses gevent to have non-blocking I/O and redis-py to handle connections, and the benchmarks are run with a patched version of redis-py that uses hiredis-py when it is available. All benchmarks are done with 10 concurrent connections.

The reported results are in the README. For SET key value plus GET key, redis-py reaches 11.76 Kops and redis-py with hiredis-py reaches 13.40 Kops, an improvement of 1.1x. The list tests tell a different story, with list entries of 5 bytes. LRANGE list 0 9 goes from 4.78 Kops to 12.94 Kops, a 2.7x improvement. LRANGE list 0 99 goes from 0.73 Kops to 11.90 Kops, a 16.3x improvement. LRANGE list 0 999 goes from 0.07 Kops to 5.83 Kops, an 83.2x improvement.

The README draws the conclusion itself: throughput improvement for simple SET and GET is minimal, but the larger the multi bulk reply, the larger the improvement.

Read those numbers with care, because the shape of the benchmark drives them. The list entries are 5 bytes each, so the comparison measures the cost of building thousands of small Python objects from a large reply, not throughput of Redis itself. The patched redis-py also means the baseline is stock redis-py, and both sides use gevent for I/O, so the difference is confined to parsing. If your workload returns small strings, the 1.1x figure is the relevant one, and it is not a reason to change anything.

Building the extension and the licence question

Building from source needs a recursive checkout of submodules, a built hiredis, and gcc. The README gives the whole sequence:

git clone --recurse-submodules https://github.com/redis/hiredis-py
python setup.py build_ext --inplace
python -m pytest

The requirements are Python 3.9 or newer, with the development headers present. On Ubuntu and Debian systems the README says to install them with apt-get install python3-dev.

The build configuration in setup.py shows how portable the extension tries to be. The compiler arguments are -std=c99 on non Windows platforms, Windows links against ws2_32, and the linker flag -Wl,-Bsymbolic is applied away from Windows and macOS. The package metadata sets python_requires to >=3.9 and ships both hiredis.pyi and py.typed, so type checkers can see the module.

The licence has a wrinkle worth flagging. The project metadata records the licence as MIT, and setup.py declares license="MIT", while the README states that the code is released under the BSD license, after the licence of hiredis, and the badge in the README points at the BSD 3-Clause text. The vendored hiredis sources are the origin of that. If you are shipping this inside a product, read the LICENSE file in the repository rather than relying on either label, since the two statements in the project itself differ.

On maintenance, the record is recent and steady. The last push was on 2026-10-15, and the release tags show v3.5.0 on 2026-10-15, v3.4.2 on 2026-09-22 and v3.4.1 on 2026-08-07.

The alternative is stock redis-py

There is no competing parser to compare against, because the alternative is not another C library but simply not using one.

Stock redis-py parses RESP in pure Python, and the benchmark table in the README is exactly that comparison: redis-py against redis-py with hiredis-py substituted. The difference in approach is that the pure Python parser builds each reply object with ordinary Python code, while the extension hands the parsing work to the C implementation of hiredis and returns the results directly. For a single inline string the difference is small, 11.76 Kops against 13.40 Kops in the README's own measurement, and for a reply with a thousand small list entries the difference is the 83.2x figure.

A second alternative is to change the data you fetch rather than the parser. If a list of a thousand entries is being read one range at a time, a different access pattern that fetches fewer items wins more than any parser change, since the cost is in constructing the reply rather than in moving it.

The practical question is therefore about deployment, not speed. hiredis-py is a compiled extension, so it needs a wheel for your platform or a compiler in the image. A pure Python environment, a platform without a prebuilt wheel, or a policy that forbids native extensions rules it out, and in those cases the 1.1x on simple commands is not worth the operational cost. Where the extension is deployable, the decision comes down to reply size: the difference between 1.1x and 83.2x in the same table is entirely in how much data the reply carries.

Editorial conclusion

hiredis-py suits a team already using redis-py whose workload returns large multi bulk replies and whose deployment can install a compiled extension, since the documented throughput gap is narrow for simple SET and GET and wide for long lists. It does not suit a service that must run on any machine without a compiler or prebuilt wheel, a pure Python environment, or a team that wants one library to own connections, because the Reader deliberately does none of that. Verify first by running the benchmark script in the benchmark directory against your own payload sizes, then by confirming that pip install hiredis succeeds on your target images, since the Reader only pays off when replies are large enough for the C parser to matter.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. redis/hiredis-py on GitHub
5. Releases