fpga-network-stack: TCP/IP and RoCEv2 implemented in HLS for Xilinx FPGAs

A network stack as HLS and RTL

The project is a scalable network stack for FPGAs that supports TCP/IP, RoCEv2 and UDP/IP at rates from 10 to 100 Gbit/s. The interesting structural choice is that the protocol logic is written in high level synthesis rather than in RTL by hand, with the tree split into an hls directory for the synthesizable C++ and an hdl directory for the parts that have to be hand written, alongside an ip directory for the packaged IP, xilinx-cmake for the vendor specific build glue, and doc, constraints, scripts, tools and waveforms directories. That split is what makes the throughput claim plausible. A TCP stack written in HLS can be compiled to a pipelined data path rather than a soft processor, so the design behaves like hardware, and the interfaces are all AXI4-Stream, so the modules compose with anything else in a Xilinx pipeline.

The toolchain and the boards that work out of the box

Two prerequisites are listed, and they set the size of the commitment. You need Xilinx Vivado 2022.2 and cmake 3.0 or higher, so this is not a project you build on a laptop with a package manager, it is a project you build inside a vendor tool flow. The repository uses git submodules, and the readme is explicit that you either clone with them or update them later:

git clone --recurse-submodules [email protected]/this/repo.git
git submodule update --init --recursive

The boards supported out of the box are the Xilinx VC709, the Xilinx VCU118 and the Alpha Data ADM-PCIE-7V3. That list is short and specific, and it tells you two things about the design. The first is that the platform is a PCIe attached host card rather than an embedded system, and the second is that the cmake platform identifier you will pass, xilinx_u55c_gen3x16_xdma_3_202210_1, names a specific Xilinx device and a specific DMA generation, which means the stack is tied closely to one family of hardware and one host interface. If your card is not on that list, expect work.

Building the HLS cores and installing them as IP

The build is cmake, then make. You create a build directory, configure with the platform and data width, and run the ip target:

mkdir build
cd build
cmake .. -DFNS_PLATFORM=xilinx_u55c_gen3x16_xdma_3_202210_1 -DFNS_DATA_WIDTH=64

The make ip target builds the HLS IP cores and installs them into the IP repository, which is the step that makes them visible to a Vivado project. The configuration surface is where most of the design decisions live, and the readme tabulates the options. FNS_DATA_WIDTH sets the data width of the network stack in bytes and takes 8, 16, 32 or 64, which is the central trade-off, since a wider bus moves more data per cycle but costs more resources and timing margin. FNS_ROCE_STACK_MAX_QPS sets the maximum number of queue pairs the RoCE stack can support, with 500 as the listed value, and FNS_TCP_STACK_MAX_SESSIONS does the same for TCP sessions. FNS_TCP_STACK_MSS sets the maximum segment size, and the TCP-specific switches turn on fast retransmit, window scaling and the DDR bypass on the receive path, while FNS_TCP_STACK_NODELAY_EN toggles Nagle's algorithm on or off.

Four targets for working on one module

You do not have to build everything to work on one part. The readme shows the per-module flow, using the TCP module as the example, where you change into the module directory, create a build directory and configure it the same way. Then there are four make targets, and the comments in the readme name the underlying HLS commands, which is a useful hint about what each one costs in time. make csim runs C simulation, which is csim_design, and it is fast because nothing is synthesised. make synth runs synthesis, csynth_design, which is where you find out whether the design fits in the device and meets timing. make cosim runs co-simulation, cosim_design, which is the slow and thorough check that the synthesised design behaves like the C model. make ip exports the IP, export_design, which is the step that produces the packaged core. The order is a sensible workflow, and it is the one any HLS project follows, but it is worth knowing that co-simulation is the expensive one and should be reserved for changes that need it.

The TCP API, expressed as AXI4-Stream channels

All the interfaces use AXI4-Stream, and packets travel on a small struct with a data word, a keep byte mask and a last flag. The application API is a set of streams with typed payloads. Opening a connection means sending an ipTuple with a 32-bit address and a 16-bit port to the open connection request stream, and the stack answers on the response stream with an openStatus carrying a session identifier and a success flag. Closing a connection means sending the session identifier, and the readme is honest that there is no completion notification, only a guarantee that the connection is closed eventually. Opening a listening port takes a port number, which must be in the active range of 0 to 32767, and the stack answers with a boolean. Notifications arrive on their own stream and say either that data is available or that a connection was closed, carrying the session identifier, a length, the address, the destination port and a closed flag. Reading data is a request for a session and a length, answered with a metadata value and the data itself. Writing data has the shape of a credit scheme, the application asks to send, the stack answers with the remaining space and an error code, and if the answer is negative the application retries. The readme also links Wavedrom waveform files for the receive and transmit handshakes, which is the fastest way to understand the ordering.

RoCEv2, the Coyote lineage and a reference platform

The RoCE part is the one with a performance claim attached. The new RDMA version, dated 02/2024, is adapted from the one used in Coyote and is fully compatible with the RoCEv2 standard, which means it can communicate with standard NICs such as Mellanox cards rather than only with another FPGA. The README states that it is proven to run at 100 Gbit/s with low latency and high throughput. The compatibility point is the one that matters in practice, because an RDMA stack that only talks to itself is a research artefact, while one that interoperates with commodity hardware is deployable. For a worked example, the project points at DavOS, the Distributed Accelerator OS, which includes either the TCP/IP stack or the RoCEv2 stack with DMA to host memory, and that is where to look if you want to see the stack in a system rather than in isolation.

Where the project stands

Two facts frame any decision to depend on this. The first is that the repository publishes no GitHub releases, so there is no tagged version to pin and no changelog of releases to read, and the last push was on 2026-08-03, which means the tree is being worked on but the versioning is informal. The second is the shape of the code itself. The stack is a set of compile-time parameters rather than a runtime-configurable library, so changing the maximum segment size or the number of queue pairs means a rebuild, and a deployment that needs to adapt at runtime is not what this is. There is a LICENSE.md, a CONTRIBUTING.md, a top level CMakeLists.txt and a per-vendor xilinx-cmake directory, and the constraints, doc and tools directories are where the board specific work lives. For a project that exists to put a real network stack into an FPGA datapath, that is a coherent set of trade-offs, and the honest way to evaluate it is to read the interface definitions first, because the API is the part you will be writing against.

Editorial conclusion

The stack is worth adopting if you are building a custom FPGA data path that has to terminate real network traffic, because writing a TCP stack in RTL is a project in itself and this one is parameterised for the trade-offs you need to make. It is the wrong choice if you are not already inside a Vivado flow, since the toolchain requirement is heavy and the option set is Xilinx specific. Before you start, read the interface definitions in the readme rather than the module names, since the API is expressed as AXI4-Stream streams, and check the supported board list, because out of the box support covers the VC709, the VCU118 and the Alpha Data ADM-PCIE-7V3 only.

Frequently asked questions

Official sources

1. fpgasystems/fpga-network-stack on GitHub
2. Issues
3. License: BSD-3-Clause
4. README