# perftest: the RDMA bandwidth and latency micro-benchmark suite > The linux-rdma/perftest repository is a collection of micro-benchmarks written over uverbs, covering send, RDMA read, RDMA write, and atomic operations, each with a bandwidth and a latency variant, plus raw Ethernet tests. It ships with a detailed methodology section that explains what the numbers mean and where they are not to be trusted. **linux-rdma/perftest** — Infiniband Verbs Performance Tests - Repository: https://github.com/linux-rdma/perftest - Stars: 1,038 · Forks: 431 - Language: C - License: NOASSERTION - Published: 2026-10-19 · Updated: 2026-10-19 · Language: en - Canonical page: https://hysenlabs.com/projects/linux-rdma-perftest ## A micro-benchmark suite, not an application simulation perftest is a collection of tests written over uverbs, intended for use as a performance micro-benchmark. The README says the tests may be used for hardware or software tuning as well as for functional testing, and that is the scope. The tests are paired by operation and by metric. Send has ib_send_bw and ib_send_lat. RDMA Read has ib_read_bw and ib_read_lat. RDMA Write has ib_write_bw and ib_write_lat. RDMA Atomic has ib_atomic_bw and ib_atomic_lat. And there is a set for native Ethernet, which the README notes is for when working with MOFED2, named raw_ethernet_bw and raw_ethernet_lat, with the later table listing raw_ethernet_send_lat and raw_ethernet_send_bw. The critical caveat is stated in the benchmarks description section, and it is the sentence that should stop you from over-reading the output. The benchmarks generate a synthetic stream of operations, which is very useful for hardware and software benchmarking and analysis, but they are not designed to emulate any real application traffic. Real application traffic may be affected by many parameters, and hence might not be predictable based only on the results of those benchmarks. That is the right mental model. This is a measurement of the fabric and the adapter, not of your application. A number that looks good here tells you the hardware can move bytes that fast, and nothing about whether your application will. The project is a C codebase with an autotools build, and the primary language is listed as C. The licence is a custom one that GitHub cannot classify, so the COPYING file at the root is the text that governs. The recent releases follow a date-stamped scheme, perftest-26.07.8 from 2026-09-28 and perftest-26.04.17 from 2026-05-31, and the last push to the repository was on 2026-10-15. ## Building it with autotools The installation is a clone and a standard autotools sequence, and the README is explicit that the perftest directory appears in your current directory after cloning. ``` git clone https://github.com/linux-rdma/perftest.git cd perftest ./autogen.sh ./configure make make install ``` The configure step takes an optional prefix if you want to install somewhere specific, with the example given as ./configure --prefix=. After the install, all of the tests appear both in the perftest directory and in the install directory, which is a useful detail because it means you can run a test from the build tree without installing at all. The note about CUDA support appears in the installation section and points to the GPUDirect usage section under the special feature detailed explanation in tests, which tells you that GPUDirect is a supported configuration rather than an experiment. The repository layout matches a classic autotools project. There is configure.ac, Makefile.am, autogen.sh, a man/ directory for the manual pages, src/ for the sources, a debian/ directory for packaging, and two helper scripts at the top level, run_perftest_loopback and run_perftest_multi_devices, which are wrappers for running a test over a loopback interface and across several devices. The encrypt_credentials.c and gen_data_enc_key.c files at the root relate to the credential handling used by the RDMA CM and related paths, and perftest.spec is the packaging specification. ## The methodology section, and what the numbers actually mean The notes on testing methodology section is the most valuable part of the README, because it tells you where the numbers are weak. Four of its points matter in practice. First, the benchmarks use the CPU cycle counter to get timestamps without a context switch, and some CPU architectures, such as Intel's 80486 or older PPC, do not have that capability. So the measurement method itself is architecture dependent. Second, the latency benchmarks measure round-trip time but report half of that as one-way latency, which means the results may not be accurate for asymmetrical configurations. If your fabric is not symmetric, that assumption is wrong and the reported number is wrong with it. Third, the bandwidth accounting differs by direction. On all unidirectional bandwidth benchmarks, the client measures the bandwidth. On bidirectional benchmarks, each side measures the bandwidth of the traffic it initiates, and at the end of the measurement period the server reports its result to the client, which combines them. Reading a bidirectional result without knowing this produces the wrong conclusion about which direction is the bottleneck. Fourth, the latency tests report minimum, median and maximum, and the documentation explains why the median is the one to look at: it is typically less sensitive to high latency variations than an average, and the first value measured is usually the maximum because of warmup effects. The documentation is also careful about iterations. The default of 1000 iterations is described as pretty good, long sampling periods have very limited impact on measurement accuracy, and the program keeps data structures with a memory footprint proportional to the number of iterations, so a very high iteration count can degrade the measured performance for reasons that have nothing to do with the devices under test. If a high count is strictly necessary, the recommendation is to use the -N flag, which is the no peak option. ## Running a test, and the two errors people make The running tests section is short and the first thing it lists is the prerequisites, which are the kernel version and a set of matching kernel modules: the kernel module must match libibverbs, librdmacm, libibumad, and libmath, and the linux kernel module must match pciutils. A version mismatch between the user space library and the kernel module is the classic cause of a benchmark that will not start. The invocation itself is a server and a client: ``` Server: ./ Client: ./ ``` The server address is an IPv4 or IPv6 address, and you can use the IPoIB address if IPoIB is configured. The help flag lists the available options. Two mistakes are called out. The first is the one that catches everybody: the same options must be passed to both server and client. The tests compare configurations, and a mismatch between the two sides means you measured something other than what you intended. The second is the note marked as important, that when running the benchmarks over an InfiniBand fabric, a Subnet Manager must run on the switch or on one of the nodes in the fabric, prior to starting the benchmarks. Without a Subnet Manager the fabric has no routing and the tests will not complete. The common options are worth skimming. The port defaults to 18515. The connection type accepts RC, UC, UD, XRC, DC, or SRD and defaults to RC. The MTU defaults to the active_mtu reported by ibv_devinfo. The IB device defaults to the first one found, and the port to 1. And there is an option to connect QPs with rdma_cm and run the test on those QPs, which is the difference between testing the raw verbs path and testing the connection manager path. ## Bandwidth modes, histograms, and long running measurements There are three ways to run a bandwidth benchmark, and they suit different questions. A bandwidth benchmark may be run for a number of iterations, or for a fixed duration. The -D flag instructs the test to run for the specified number of seconds, and the --run_infinitely flag makes the program run until interrupted by the user, printing the measured bandwidth every 5 seconds. The infinite mode with periodic reporting is the one to use for a soak test, where the interesting question is not the peak but whether the number holds after an hour. The histogram option is the other feature to know about. The -H option in latency benchmarks dumps a histogram of the results, and the README suggests viewing it with xgraph, ygraph, r-base, PSPP, or other statistical analysis programs. Given the point about the median being more stable than the average, and the point about the first measurement being the maximum because of warmup, a histogram is how you actually see the distribution rather than three summary numbers. The comparison to a general purpose network benchmark is instructive. A test that sends fixed size messages in a loop measures the fabric's behaviour under a synthetic load, and the numbers are highly reproducible. That reproducibility is exactly what you want for tuning, where the question is whether a change to the adapter, the firmware, or the queue settings moved the number. What it does not tell you is how your application will perform, because a real application has a message size distribution, a concurrency profile, and a response pattern that the loop does not reproduce. ## Where the numbers get posted, and what the project is for The suite's purpose is stated in the overview and it is narrower than a general benchmarking programme. The tests are a micro-benchmark, for hardware or software tuning and for functional testing, and the README asks you to post results and observations to the openib-general mailing list, with the contact details at openib.org. That request is a good indicator of how the tool is used in practice. Someone suspecting that their adapter is underperforming, or that a cable or a switch port is degrading, runs these tests and compares the numbers against what the hardware should do. The output is a small set of comparable figures, produced by a tool everyone in the field has, which is what makes them comparable at all. A bespoke benchmark script produces numbers that only mean something on the machine that wrote it. The same is true of the common options, which is why the option list matters more than it appears. The connection type option lets you compare reliable connection against unreliable connection or datagram, which is the comparison that tells you whether a fabric problem is a transport problem. The MTU option, defaulting to the active MTU from the device, lets you test whether raising the MTU helps. The RDMA CM option lets you separate the verbs performance from the connection manager performance, which is a question that comes up whenever a comparison between a software stack and a hardware implementation disagrees. The reference point for all of this is the hardware. A bandwidth number only means something relative to the link rate of the adapter and the port on the switch, and the perftest suite is the instrument that lets you compare your two. ## Conclusion Adopt perftest if you are tuning an RDMA fabric, comparing adapters, or establishing a baseline for an SLO, since it is the reference measurement tool for that job. Do not treat its numbers as an application benchmark, because the suite states it is not designed to emulate real application traffic. Verify first that a Subnet Manager is running on the switch or on a node in the fabric, since without one the benchmarks over InfiniBand will not work at all. ## FAQ ### What tests does perftest provide? Send, RDMA Read, RDMA Write, and RDMA Atomic, each with a bandwidth and a latency variant, giving ib_send_bw, ib_send_lat, ib_read_bw, ib_read_lat, ib_write_bw, ib_write_lat, ib_atomic_bw, and ib_atomic_lat. There are also raw Ethernet tests, raw_ethernet_send_lat and raw_ethernet_send_bw, for when working with MOFED2. ### How do I build perftest from source? Clone the repository, cd into the perftest directory, run ./autogen.sh, then ./configure, which accepts an optional --prefix for a specific install directory, then make and make install. All of the tests appear in the perftest directory and in the install directory. ### What must be true before I run an RDMA benchmark? The kernel module has to match libibverbs, librdmacm, libibumad, and libmath, and the linux kernel module has to match pciutils. Over an InfiniBand fabric a Subnet Manager must also be running on the switch or on one of the nodes in the fabric before the benchmarks are started. ### How do I run a perftest server and client? The server is started as ./ and the client as ./ , with the address being IPv4 or IPv6 and the IPoIB address usable if IPoIB is configured. The same options must be passed to both server and client. ### How long should a perftest run be? The default of 1000 iterations is described as pretty good, and long sampling periods have very limited impact on measurement accuracy. For a fixed duration use the -D flag, and for a soak run use --run_infinitely, which prints the measured bandwidth every 5 seconds until interrupted. The -H option dumps a histogram of latency results. ## Sources - [Issues](https://github.com/linux-rdma/perftest/issues) - [linux-rdma/perftest on GitHub](https://github.com/linux-rdma/perftest) - [README](https://github.com/linux-rdma/perftest/blob/master/README.md) - [Releases](https://github.com/linux-rdma/perftest/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/linux-rdma-perftest