JuliaDiff/ForwardDiff.jl: forward mode differentiation for Julia, and the mode choice that decides everything

What forward mode actually computes

ForwardDiff implements methods to take derivatives, gradients, Jacobians, Hessians and higher order derivatives of native Julia functions, or any callable object really, using forward mode automatic differentiation.

The mechanism is the dual number, and the standard example makes the shape of it clear.

julia> using ForwardDiff

julia> f(x::Vector) = sin(x[1]) + prod(x[2:end]);  # returns a scalar

julia> x = vcat(pi/4, 2:4)
4-element Vector{Float64}:
 0.7853981633974483
  2.0
  3.0
  4.0

julia> ForwardDiff.gradient(f, x)
4-element Vector{Float64}:
  0.7071067811865476
  12.0
   8.0
   6.0

julia> ForwardDiff.hessian(f, x)
4×4 Matrix{Float64}:
 -0.707107  0.0  0.0  0.0
  0.0       0.0  4.0  3.0
  0.0       4.0  0.0  2.0
  0.0       3.0  2.0  0.0

The function is ordinary Julia. Nothing is annotated, no derivative rules are written by hand, and the gradient and the Hessian are exact to floating point precision rather than estimated.

The performance claim is stated in the README in a measured way rather than an absolute one: while performance can vary depending on the functions you evaluate, the algorithms implemented by ForwardDiff generally outperform non-AD algorithms such as finite differencing in both speed and accuracy. That is the argument for automatic differentiation over numerical differentiation in general, and it holds because forward mode carries the derivative alongside each value as it evaluates, rather than re-evaluating the function at perturbed points and subtracting two nearly equal numbers.

The repository is a standard Julia package layout, with `src/`, `test/`, `ext/` for package extensions, `benchmarks/` and `docs/`, and a `Project.toml`.

Scalar to vector is the case forward mode wins

The README does something most automatic differentiation documentation does not, which is tell you when not to use the package.

Functions that map a vector to a scalar are the best case for reverse mode automatic differentiation, and the example `f` above is exactly such a function. But ForwardDiff may still be a good choice if `x` is not too large, as it is much simpler.

The best case for forward mode differentiation is a function which maps a scalar to a vector, and the example is:

julia> g(y::Real) = [sin(y), cos(y), tan(y)];  # returns a vector

julia> ForwardDiff.derivative(g, pi/4)
3-element Vector{Float64}:
  0.7071067811865476
 -0.7071067811865475
  1.9999999999999998

The reason is the cost model, and it is worth understanding rather than memorising. Forward mode propagates a value and its derivative together, so a function from a vector of n inputs to a scalar carries n derivative components through every operation it performs, which is a cost proportional to n per evaluation. Reverse mode instead propagates the derivative of the final scalar backwards through the computation, and its cost is proportional to the number of operations, which is usually far smaller.

So the number of independent inputs drives the forward mode cost, and the number of operations drives the reverse mode cost. A function with one input and a hundred operations is a forward mode case, and a function with a thousand inputs and ten operations is a reverse mode case. The Jacobian API makes the same point from the other direction, since the map from inputs to outputs is where forward mode shines:

julia> ForwardDiff.jacobian(x) do x  # anonymous function, returns a length-2 vector
         [sin(x[1]), prod(x[2:end])]
       end
2×4 Matrix{Float64}:
 0.707107   0.0  0.0  0.0
  0.0       12.0  8.0  6.0

The JuliaDiff project maintains a web page listing the other automatic differentiation packages, so the decision is documented rather than a matter of tribal knowledge.

The version 1.0 change that fixed a class of bugs

The first section of the README is an upgrade note, and it is the most useful paragraph in the file.

Equality on `Dual` numbers now requires both the real and the dual part to match. The stated reason is that the change removes, in the words of the note, a large number of bugs where the structure of, for example, non-zero values in an array was inspected, leading to erroneous derivatives.

The bug class is worth spelling out, because it is a general hazard of automatic differentiation. A dual number is a pair, a real part and a tangent part, and code that asks whether such a number is zero, or compares it to another, or uses it as a dictionary key, is asking a question that the old equality answered with the real part alone. So a function that skips a term when the input is zero, evaluated in dual arithmetic, would silently drop the derivative of that term. The result is a derivative that is wrong in a way that looks like a plausible number, which is the worst kind of wrong.

The note is careful about the consequence. This might cause slightly different behaviour in programs, but should in general be more correct than previously. That is an honest framing of a breaking change, and it is the sort of change that a package should make deliberately rather than drift into, since the old behaviour was a trap for anyone who wrote control flow on a differentiated value.

If you are upgrading from a pre-1.0 version and your code inspects the structure of values returned from a differentiated computation, that code is what needs review.

DiffRules, and the limits of the derivative table

ForwardDiff relies on DiffRules for the derivatives of many simple functions such as `sin`.

That sentence is the whole story of how the library works, and it is worth unpacking. Forward mode cannot invent a derivative for a function it has never seen. It carries the derivative through arithmetic that it knows, and for anything else it needs a rule: a way of computing the derivative of a primitive function. DiffRules is the table of those rules for many simple functions such as `sin`.

The practical consequence is that the quality of your derivative depends on the coverage of the rule table for the functions you call. Elementary functions are covered, and functions you define yourself are handled because their derivative is computed from their internals, but a call into a package that uses an unsupported special function, or that reaches for `ccall` into a C library, is a place where the derivative either errors or is wrong.

That is the general weakness of forward mode and it is worth knowing before you commit: the failure mode is not a graceful fallback, it is either a method error at the call site or a silently wrong number, and the second is the reason the version 1.0 equality change mattered. The same applies to code that branches on values, since a branch taken on the real part is a branch that the derivative does not follow.

For a package written to be differentiated, the requirement is that it avoids non differentiable constructs, or provides its own rules, and the ecosystem's rule table is the place where such a rule gets added.

The shape of the project and its release rhythm

A few details about how the package is put together, for anyone planning to contribute or to depend on it.

The documentation is published with the usual stable and dev split, both on the juliadiff.org site, and the repository has a `docs/` folder that is the source of it. The CI badge and the codecov badge are both present, so the test suite and the coverage are part of the routine. The `ext/` directory is for Julia package extensions, the modern mechanism for optional dependencies, and the `benchmarks/` directory is where the performance claims are tracked.

The project also asks for a citation, which is unusual in a README and tells you something about how the maintainers see the work. The requested reference is the paper Forward-Mode Automatic Differentiation in Julia by Revels, Lubin and Papamarkou, and the bibtex entry is provided:

@article{RevelsLubinPapamarkou2016,
    title = {Forward-Mode Automatic Differentiation in {J}ulia},
   author = {{Revels}, J. and {Lubin}, M. and {Papamarkou}, T.},
  journal = {arXiv:1607.07892 [cs.MS]},
     year = {2016},
      url = {https://arxiv.org/abs/1607.07892}
}

The release history is steady, with v1.4.4 on 2026-08-04, v1.4.5 on 2026-08-06 and v1.4.6 on 2026-09-11, and the last push to the repository on 2026-10-14. The default branch is `master`, the licence file is `LICENSE.md`, and the repository is not archived, so the package is being maintained even though the major version has been 1.0 for a while.

Editorial conclusion

Adopt ForwardDiff.jl when your Julia function maps a scalar to a vector, or when the input dimension is small enough that the cost of a forward sweep is irrelevant, since those are the cases the package's own documentation identifies as its strength. Do not adopt it for a large vector to scalar function where you need a gradient, because reverse mode is the better tool there and the README says so. Verify first that your function is written in a way the dual number arithmetic can carry, that every non linear primitive it calls has a derivative rule available through DiffRules, and that you have read the version 1.0 upgrade note, since the change to equality on dual numbers can alter the behaviour of code that inspected the structure of an array of them.

Frequently asked questions

Official sources

1. Issues
2. JuliaDiff/ForwardDiff.jl on GitHub
3. README
4. Releases