lwesp: an AT command parser for talking to ESP modules in C

What the library is for

LwESP is a lightweight AT command parser library for communicating with ESP8266, ESP32 and the other series of Wi-Fi modules from the same vendor, using AT commands. It is written in C11 and is system platform agnostic.

The use case is worth stating precisely, because there are two ways to use these chips and the library only serves one of them.

When an ESP32 is your microcontroller, the wireless stack runs on the chip itself and you talk to it through the vendor's own functions. When the same chip is sold as a bare radio module and soldered onto a board with a different microcontroller, the wireless stack still runs on the module, and the only way to talk to it is a command interface over a serial link. That interface is the AT command set, and somebody has to write the parser that speaks it.

So the library is that parser, and the readme says what it sits on top of: the modules run the vendor's official AT command software and communicate with the host device over a UART or a serial peripheral interface protocol.

The target platforms are the interesting part. The main targets are embedded system devices like ARM Cortex-M, AVR and PIC, and the library is developed under a Windows host, with examples for the microcontroller, Windows and POSIX platforms. It also works under Windows, Linux and Mac environments. So the same code that drives a radio module from a microcontroller can be compiled on a desktop for development and testing, which is the arrangement that makes embedded networking libraries pleasant to work with.

The stated aim is to be lightweight and versatile, and the feature list is consistent with that. The language is C with the fixed width integer types from the standard header, so the code does not depend on the size of a native integer, which on an microcontroller with an unusual architecture is a real portability question. The supported hardware covers the current modules from the vendor, including the variants with different radios.

And the documentation is hosted separately, with the readme pointing at it for implementation details, which is the right division for a project where the manual is longer than the readme.

Configuration, and the platform argument

Two features on the list are about the shape of the library rather than about what it does, and they are the ones that make it usable on a microcontroller.

The first is that different configurations are allowed to optimise the user's requirements. That sounds like a sentence about build flags, and it is, but the phrasing matters: optimisation is a goal of the configuration, not an accident of it. A library that ships a single configuration with everything compiled in is a library that costs a microcontroller a kilobyte of flash and a few hundred bytes of RAM for features the project will never use. A configuration-driven library lets the integrator choose, and the choice is part of the interface rather than a set of undocumented macros.

The second is platform independence, and the readme backs it with a claim that is unusual and worth taking seriously: the library is developed under the Windows platform, and examples are available for the microcontroller, Windows and POSIX platforms. Developed on a desktop, tested on the desktop, ported to the target. Most embedded libraries are the other way round, which means the desktop is a place you port to for convenience and the real target is where the bugs are. Building the library the other way round means the common build is the one that most people will run.

The portability work is also visible in the repository itself. There is a formatting configuration, which tells you the project is formatted with a specific tool and that the formatting is part of the contribution rules rather than a matter of taste. There is a set of presets for the build system, which is how a project gives you known-good configurations without a page of prose. There is a module description file, which is the format the platform's own module manager uses, so the library can be installed the same way as everything else on that platform. And there is an editor directory, a set of snippets, and a directory of certificates.

That last one is worth a moment's thought. A library that can talk to a Wi-Fi module and can therefore open TLS connections needs somewhere to put the trust anchors, and a repository that carries a certificates directory is a project that has thought about the difference between a trusted certificate on a desktop and a trusted certificate on a device with no certificate store.

The threading model

The most substantive design decision in the feature list is the one about operating systems, and it is written in a way that is unusually candid about what is and is not supported.

The library supports operating system implementations with advanced inter-thread communication, or a real-time operating system, and then the readme says: currently only the operating system mode is supported.

So there are two modes in the design, one of them implemented and the other not, and the difference between them is threads. In the real-time operating system mode, the library uses two different threads to process user input and received data. A producer thread collects user commands from application threads and starts command execution, and a process thread processes received data from the ESP device.

That split is the right one for this problem, and it is worth understanding why. The two directions of a serial conversation have completely different characteristics. Outbound, the application hands over a command and the library needs to send it, possibly after waiting for the module to be ready. Inbound, the module is talking at any time, and the responses have to be matched to the commands that are waiting for them. If a single thread did both, then every inbound byte would have to be handled on the same thread that is trying to send a command, which means either the application blocks on a slow network while a response is sitting unread, or the command machinery is reentrant in a way that is very hard to get right on a microcontroller.

Splitting them means the sending side is a queue and the receiving side is a parser, and neither blocks the other. The producer thread is where the application hands over work, and the process thread is where the module's output is turned into answers to the outstanding commands. The inter-thread communication is the part that the feature list describes as advanced, and on an embedded system that communication is the thing that needs the real-time operating system's primitives, which is why the feature is listed as advanced rather than as ordinary.

The trade is memory and determinism. Two threads mean two stacks, and the module's responses have to be buffered somewhere while they wait to be matched. On a microcontroller with tens of kilobytes of RAM, that is a real budget, and it is the reason the configuration-driven build matters: a project that cannot afford the threaded mode needs a build that leaves it out.

And the readme's list of supported platforms includes a bare mode, since the threading is described as being about the operating system implementations specifically, which implies the library also has a mode where there is no operating system underneath and the application drives it directly.

The connection API, and the applications on top

The library provides a connection API based on the sequential model, for connections in both client and server mode. The name of the model is a reference to the connection abstraction that the system's own socket layer exposes, and the reason it matters is that the API becomes familiar.

A socket-based connection API has a shape: you open a connection, you send and receive on it, you close it. Any engineer who has used that API in a system library already knows how to use this one, and more importantly, the code that consumes the library's connections can be written against a familiar shape. The alternative, a callback-driven or a state-machine API, would be faster in some situations and much harder to use in all of them.

The sequential model also matches the hardware. A radio module is a serial device that processes one command at a time, and a connection that needs several sockets open at once is a connection that needs several commands queued. The API's shape follows the device's shape, which is the right way round.

On top of that core, the readme lists the applications that are included rather than left to the user, and they are the ones that every embedded networking project ends up writing.

There is an HTTP server with support for dynamic files, which means the server can serve files from a file system that is mounted or generated at run time rather than from a compiled-in array, which is what you need for a device whose firmware is updated but whose web interface changes. There is an MQTT client, which is the other protocol that shows up in every embedded project. And there is an optional command line module with ready-made commands for common operations, which is the embedded equivalent of a diagnostic shell and which saves every project from writing one.

The remaining features are the vendor's own AT command surface, exposed rather than reinvented. There is Wi-Fi protected setup management, so a device can be put into pairing mode without a firmware update. There is custom DNS setup and a hostname for the DHCP server, so a device on a network announces itself. There is ping. There is optional IPv6 support. There are modules for multicast name resolution, for network time synchronisation, and for a vendor-specific provisioning mechanism that puts a device onto a wireless network from a phone.

And there is direct access to the module's own flash and its non-volatile storage, with read, write and erase, which is how you store credentials, calibration data and counters on a device that has no file system of its own.

That last group is the one that shows the scope most clearly. This is not a networking library that treats the module as a black box; it is a way of driving the module's whole feature set from C, including the parts that are about storage rather than about networking.

The repository and the examples

The examples directory is the most useful part of the repository for a newcomer, and its structure is the structure of the library's portability claim.

There is a directory for a desktop POSIX example, one for a Windows example, and one for a microcontroller example, with a shared library directory that the three have in common. So the same application code, the same library calls, three different platform layers, and a directory of platform-specific material that is where the differences live. That is what platform independence looks like when it is actually done, and it is a much better demonstration than a claim in the readme.

The other directories in the repository follow the same logic. There is a development directory, a documentation directory, and a directory whose name suggests the web content, plus a set of module and preset files for the build system, a change log, a list of authors, a licence, and a file of outstanding work.

The presence of a list of authors as a separate file rather than a section in the readme is a small thing that says the project is collaborative and has been for a while. The release history says the same thing: a first release in the spring of 2021 and a development version in the following year, with the default branch being the development one rather than a released one.

That branch choice is worth noticing, because it tells you how the project expects to be used. A library whose default branch is the development branch is a library where the released tag is a snapshot and the tip is where the work is, and where a user who wants the newest fix is expected to work from the tip rather than from a release. For a library whose host platform is a microcontroller toolchain, that is often the only practical arrangement, because the release cadence of the host's own tooling is faster than a library's release cadence, and a library that only releases when the host releases cannot keep up.

The build system deserves a note too. There is a build file at the root and a directory of modules for it, and there are presets, which is the modern arrangement for a project that has to support several toolchains. A library targeting half a dozen architectures and two operating systems cannot have a single build invocation that works everywhere, and presets are how a project ships the known-good combinations instead of a paragraph describing them.

And the contribution rules are unusually specific for an embedded library, which is where most of the friction lives. Fresh contributions are welcome, and the instructions are three steps: fork the repository, follow the project's C style and coding rules and use the formatting tool to format the code, and open a pull request against the development branch. The explicit mention of a formatting tool is the one that matters in practice, because an embedded project with a strict style guide and no automation rejects contributions on formatting grounds constantly, and a rule that says to run the tool is cheaper for the maintainer than a rule that says to follow the style.

Editorial conclusion

lwesp fits an embedded engineer who has chosen a bare ESP radio module over a microcontroller with the radio built in, and who wants the connection API to look like the one their other networking code already uses. It does not fit somebody who wants the ESP as a system-on-chip with its own wireless stack, because the whole point of the library is that the stack lives on the other end of a serial link. Verify first that your module's firmware is the official AT command software, since the library is written against that specific set of commands, that the transport you use is one the library's platform layer supports, because the UART and SPI links are not interchangeable, that you can afford the memory the netconn buffers take, since a connection-based API keeps state, and that the threading model matches your system, because the operating system mode uses two threads and the bare mode uses none.

Frequently asked questions

Official sources

1. License: MIT
2. MaJerle/lwesp on GitHub
3. Project website
4. README
5. Releases