NimbleParsec: parser combinators that compile to binary matching

Composers write parsers, the compiler writes the clauses

NimbleParsec is a simple and fast library for text-based parser combinators, and the mechanism is stated in two sentences: combinators are composed programmatically and compiled into multiple clauses with binary matching. The word programmatically is the important one. Most parser combinator libraries in dynamic languages build an interpreter that walks a data structure at runtime, and this one does not, it generates code. The benefit list gives three consequences of that choice. Performance comes because compiling to binary matching uses many Erlang VM optimisations to generate fast parser code with low memory usage. Composability comes because the library does not rely on macros for building and composing parsers, so they are fully composable, and the only macros are defparsec/3 and defparsecp/3, which emit the compiled clauses. And there is no runtime dependency, because after compilation the generated clauses have no runtime dependency on NimbleParsec, which means you can compile parsers and avoid imposing a dependency on the users of your library.

A date and time parser, and the clauses it becomes

The worked example is a date and a time, which is the right size for showing the whole shape. You define date by piping a four digit integer, then ignoring a dash, then a two digit integer, then a dash, then another two digit integer, and time the same way with colons and an optional trailing Z. Then defparsec combines them with a T separator:

defmodule MyParser do
  import NimbleParsec

  date =
    integer(4)
    |> ignore(string("-"))
    |> integer(2)
    |> ignore(string("-"))
    |> integer(2)

  time =
    integer(2)
    |> ignore(string(":"))
    |> integer(2)
    |> ignore(string(":"))
    |> integer(2)
    |> optional(string("Z"))

  defparsec :datetime, date |> ignore(string("T")) |> concat(time), debug: true
end

MyParser.datetime("2010-04-17T14:12:34Z")
#=> {:ok, [2010, 4, 17, 14, 12, 34, "Z"], "", %{}, {1, 0}, 20}

The result tuple shows what you get back: the parsed values, the rest of the input, a context map, the line and column, and the number of bytes consumed. The debug: true option prints the generated clauses, and what it prints is the point of the whole library, because a function that pattern matches a fixed set of bytes in a single clause, with the digit range checks inlined into the guard, is what makes the library fast.

The generated clause is hand-written parser code

The debug output is worth reading as a whole because it shows exactly what you are being charged for. The first clause matches fourteen bytes in one pattern, with a guard checking that each byte is between the character codes for zero and nine, and then it folds them into integers with the arithmetic written out in full. The library's own comment on this is that it generates highly inlined code, comparable to hand-written parsers, and that this gives NimbleParsec an order of magnitude performance gains compared to other parser combinators. Further performance can be gained by giving the inline: true option to defparsec/3. The clauses that do not match fall through to an error clause returning an error tuple with the remaining input and the position, and the optional Z is handled by a clause that matches the Z and a sibling clause that does not, which is how the optional combinator becomes two functions.

Reusing a combinator inlines it again, and that is the cost

The performance considerations section explains the other side of aggressive inlining, and it is a real constraint rather than a footnote. The library works by aggressively inlining code, and if you define date and time combinators and then use them in two different combinations, each node appears twice and is therefore compiled twice. The consequence is stated directly: reusing combinators over and over again can lead to memory usage during compilation as well as high compile times. The fix is to encapsulate combinators and reuse them with defpcombinatorp:

defcombinatorp :date, ...
defcombinatorp :time, ...

date_then_time = concat(parsec(:date), parsec(:time))
time_then_date = concat(parsec(:time), parsec(:date))
defparsec :combinations, choice([date_then_time, time_then_date])

By using parsec(:date) and parsec(:time) you point to previously defined and compiled combinators, which leads to better compile-time performance. The distinction is that defcombinatorp compiles the combinator once into a reusable function, while parsec calls it, so the cost is paid once rather than per use site. If you are writing a grammar where the same sub-grammar appears in several places, this is the difference between a grammar that compiles quickly and one that does not.

It is a set of primitives, not a framework

The stated goal is to focus on a set of primitives for writing efficient parser combinators, and the composition aspect means you are supposed to be able to use those primitives to implement higher level combinators yourself. That is a deliberate boundary. The library gives you integer, string, ignore, optional, concat and choice, plus the two macros that turn a composed structure into compiled clauses, and it leaves the grammar design to you. The scope also has a stated boundary at the other end: the library does not handle low-level binary parsing, and for those cases the recommendation is Elixir's bitstring syntax, which is the language's own tool for matching on binary patterns. So a project that needs to parse both a text format and a binary framing cannot use NimbleParsec for both halves, and has to reach for the bitstring syntax in the same codebase, which is a reasonable division of labour but worth knowing before you commit.

Installation, the family it belongs to, and the licence question

The installation is a dependency line in mix.exs:

def deps do
  [
    {:nimble_parsec, "~> 1.0"}
  ]
end

The tilde requirement means the 1.x line, which is the right constraint for a library that generates code into your modules. The repository belongs to a family of libraries from Dashbit, all prefixed Nimble, and the README lists them: NimbleCSV for CSV parsing, NimbleOptions for validating and documenting high-level options, NimbleOwnership for resource ownership tracking, NimblePool as a tiny resource pool implementation, NimblePublisher as a minimal filesystem-based publishing engine with Markdown support and code highlighting, and NimbleTOTP for generating time-based one time passwords. The tree is small, with a lib directory, an examples directory, a test directory, a mix.exs, a .formatter.exs and a CHANGELOG, and the examples are a simple language parser, a simple math parser and a simple XML parser. The licence field on the repository is not one GitHub can classify, but the licence text in the README is the Apache License 2.0, with copyright 2018 Plataformatec and copyright 2020 Dashbit, which reflects the project changing hands. The last push was on 2026-08-13.

Editorial conclusion

NimbleParsec suits an Elixir project that parses a text format at volume and cares about both speed and not imposing a dependency on the users of its own library, since the compiled clauses have no runtime dependency on the library. It does not suit a project that needs to parse binary data, because the library explicitly does not handle low-level binary parsing and points at Elixir's bitstring syntax instead. Before adopting it, read the performance section, because reusing a combinator in several places inlines it each time, and the fix is defpcombinatorp rather than a smaller grammar.

Frequently asked questions

Official sources

1. dashbitco/nimble_parsec on GitHub
2. Issues
3. README