NFStream: flow-level network analysis in Python, built on nDPI and a CFFI engine
NFStream: a Flexible Network Data Analysis Framework.
At a glance
- What is it?
- NFStream turns packets into flow records with statistical features, encrypted application labels from nDPI, and optionally the process that owns each socket. It is a good fit for offline dataset building and reproducible ML feature extraction, and a poor fit if you need a single long-running capture process that survives restarts.
- Who is it for?
- Adopt NFStream if you are building flow-level datasets or training models where the feature computation has to be identical between training and deployment, and if your capture fits the offline pcap path or a Linux live interface. Do not adopt it if you need a supervised capture daemon with restart semantics, or if you are on a platform where AF_PACKET_V3 is unavailable and you expect the same throughput.
- Can I use it commercially?
- Yes, with conditions. LGPL-3.0 is a weak copyleft licence: you can use it inside commercial and closed-source software, but if you distribute changes to its own files, you must publish those changes under the same licence.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What NFStream is for, and who ends up using it
The README frames NFStream as a Python framework providing data structures for working with online or offline network data, with the stated goal of becoming a unifying network data analytics framework for researchers so that experiments are reproducible. That second goal is the interesting one. Most flow tooling produces records, but the exact definition of a flow, the timeout that closes it, and the way packet size statistics are aggregated differ between tools. If two papers compute the same named feature with different code paths, the comparison is not fair. NFStream's pitch is that the feature computation logic lives in one place, so a model trained on NFStream features can be deployed against NFStream features.
The audience follows from that. People building intrusion detection datasets, people extracting flow features for classification experiments, and people who want application identification on encrypted traffic without writing their own TLS fingerprinting. The topics list on the repository includes dataset-generation, deep-packet-inspection and traffic-classification, which matches the README's emphasis. It is not a packet capture tool in the Wireshark sense and it is not a NetFlow exporter that speaks the NetFlow protocol to a collector. It is a library you call from Python, and the output is Python objects or a dataframe.
The mechanism: nDPI for labels, CFFI for the computation engine
Two decisions define the architecture. First, deep packet inspection is delegated to nDPI, which the README credits for encrypted application identification and metadata fingerprinting across protocols such as TLS, SSH, DHCP and HTTP. NFStream does not implement its own classifier; it wraps nDPI and exposes the results as flow fields. That means the set of application labels you get is the set nDPI knows, and it moves when nDPI moves.
Second, the computation engine is CFFI based, with AF_PACKET_V3 and FANOUT named for Linux packet capture and multiprocessing for parallelism. The README also lists PyPy support. The practical consequence is that the packet-level work happens outside the CPython interpreter, so the per-packet cost is not dominated by Python bytecode. Statistical features are then produced in two flavours: post-mortem features such as minimum, mean, standard deviation and maximum of packet size and inter-arrival time, and early flow features such as the sequence of the first n packet sizes, inter-arrival times and directions. The early variant exists because a classifier that needs the whole flow cannot decide anything until the flow ends.
A third path, system visibility, probes the monitored system's kernel to find open Internet sockets and attach ground-truth process information such as process name and PID at the application level. The README calls this guaranteed ground-truth. That is a strong claim and it is worth reading as a statement about where the data comes from (the kernel's socket table) rather than a guarantee about every packet in every capture.
Getting it running: install, pcap, live capture, exports
The README documents installation through PyPI as the primary route, with a separate building-from-sources section, and the release history shows v6.6.0 published in February 2026 with v6.5.4 before it in November 2025. The usage examples in the README are built around a streamer object constructed over a source. For offline work the source is a pcap file; for online work it is an interface name. The streamer is then iterated, and each iteration yields a flow object whose attributes are the features.
Two export interfaces are documented: a Pandas export interface and a CSV export interface. The Pandas path is the one most people will use for dataset work, since it turns the stream into a dataframe directly. The CSV path exists for cases where you want the records on disk without loading pandas.
Extensibility goes through NFPlugins. The README's claim is that a new flow feature can be created within a few lines of Python, and there is a dedicated section in the table of contents. The machine learning section is split into training a model and running an ML powered streamer on live traffic, which is the deployment half of the reproducibility argument. There is also a live Jupyter notebook linked through Binder if you want to see the API surface before installing anything locally.
One thing the README does not spell out in the material available here is the full set of constructor parameters for the streamer. If you need to control flow timeouts, active/inactive thresholds or the number of capture workers, check the API reference on nfstream.org rather than assuming the defaults are what you want.
Where the design forces a trade-off
The early-features design is the clearest example. To emit the first n packet sizes and inter-arrival times, the framework has to hold state for flows that have not terminated, and it has to decide when to release that state. That decision is the flow timeout. Set it long and memory grows with the number of concurrent flows; set it short and long-lived sessions get split into multiple records, which changes the statistics you compute over them. NFStream gives you both post-mortem and early features, but it does not remove the underlying tension, and the README does not present it as a tension at all.
The system visibility path is the second one. Reading the kernel's socket table gives you process attribution that packet headers cannot provide, which is genuinely useful for labelling. It also ties the feature to the machine doing the capture and to the privileges of the capturing process. On a container host or a machine with heavy NAT, the mapping between a flow and a process is less obvious than the feature name suggests.
Third, the performance story is platform-specific in a way the feature list does not advertise. AF_PACKET_V3 and FANOUT are Linux mechanisms. The README says multiplatform and the CI badges cover Linux, macOS and Windows, so the code runs in all three, but the fast capture path described in the performance bullet is a Linux path. If you are on macOS or Windows, expect the same API with a different capture backend and do not carry Linux throughput expectations across.
Alternatives and how they differ in approach
The most direct comparison is to a NetFlow or IPFIX exporter such as softflowd or the flow export built into a router or switch. Those tools also produce flow records, but the record is defined by the export protocol: a fixed set of fields, a fixed template, and the statistics are whatever the exporter chose to compute. NFStream instead computes features in your process and hands them to you as Python objects, which means you can add fields through NFPlugins and you can join the result to a dataframe in the same script. The cost is that you own the capture host and the pipeline; an exporter pushes records to a collector and you never touch the packets.
A second comparison is to a packet analysis library such as Scapy. Scapy gives you packet-level access and full control, and you can build flows from it. NFStream gives you flows first, with the packet layer largely hidden behind the CFFI engine and nDPI. The difference shows up when you need something nDPI does not classify or a field the framework does not expose: with Scapy you write the parser, with NFStream you either find an NFPlugin hook or you are blocked.
The third is the nDPI library itself. NFStream is a Python wrapper around it plus flow accounting plus feature extraction. If you only need application identification and you are already in C, calling nDPI directly avoids a Python dependency and a framework's opinions about what a flow is.
Maintenance, licensing and the upgrade surface
NFStream is licensed LGPL-3.0. The practical implication for most users is that importing it as a library is the ordinary case the licence anticipates, while modifying it and redistributing, or statically linking it into a closed product, raises obligations that a permissive licence would not. That is a general description of the LGPL family, not legal advice, and the LICENSE file in the repository is the authority.
The release history in the material shows a gap worth noting: v6.5.3 is dated October 2022 and v6.5.4 is dated November 2025, with v6.6.0 in February 2026. Whatever the cause, it means the project has had both quiet stretches and recent activity, so pinning a version and reading the changelog before upgrading is the sane approach. Because deep packet inspection is delegated to nDPI, an NFStream upgrade can change which application labels your flows receive even if your own code is untouched. If your model was trained on a specific label set, that is the upgrade risk to watch, and it is not something you can detect from the NFStream version number alone. The repository is not archived and the last push recorded is August 2026.
Who should pick it up, and what to check first
Take NFStream if your work is offline pcap analysis at flow granularity, or a live capture on Linux where you control the host, and if you want the same feature code path in training and in deployment. The NFPlugin mechanism and the Pandas export are the two features that make it pleasant for dataset construction, and the nDPI integration saves you from writing TLS and SSH fingerprinting.
Leave it alone if what you actually need is a long-running capture daemon with well-defined restart behaviour, or if your environment is a shared host where process attribution is unreliable, or if you need protocol parsing that nDPI does not cover and you are not prepared to write it. In those cases a NetFlow exporter plus a collector, or Scapy with your own flow table, will fit better even though both require more work up front.
Before you build anything on top of it, run your own pcap through the streamer and look at the application labels nDPI assigns to your traffic, because that set is the ceiling on what your downstream classifier can distinguish. Then check the flow timeout behaviour against the session lengths you actually see, since that is what determines whether one TCP session becomes one record or several. Both checks are cheap and both change the design of whatever you build next.
Editorial conclusion
Adopt NFStream if you are building flow-level datasets or training models where the feature computation has to be identical between training and deployment, and if your capture fits the offline pcap path or a Linux live interface. Do not adopt it if you need a supervised capture daemon with restart semantics, or if you are on a platform where AF_PACKET_V3 is unavailable and you expect the same throughput. Before committing, verify three things against your own traffic: which nDPI application labels actually appear for your protocols, whether the system visibility path resolves the process names you expect on your kernel, and how the flow timeout defaults interact with the session lengths in your capture.
Community notes