FlowCarp identifies application-layer protocols in network traffic based on behavior instead of port numbers. FlowCarp reads packet data in the form of PCAP, PcapNG or TZSP streams and outputs information about the flows it finds, including the detected application layer protocol for each flow.
Latest version: 0.8.4
The protocol identification feature in FlowCarp is similar to what some vendors refer to as "App-ID" or "application identification" or "NBAR", which are methods that use deep packet inspection (DPI) to determine the traffic category for each flow. FlowCarp also inspects the application-layer data, but without protocol parsers or signature rules. Instead, it computes a few statistical measurements of the data and compares those measurements to known protocols. This special technique allows FlowCarp to identify proprietary and undocumented protocols, since it doesn't require any protocol specification.
FlowCarp takes packet data as input. The following packet input methods and formats are currently supported:
The input packets can be previously captured packets in a PCAP file on disk, but it is also possible to let FlowCarp analyze real-time traffic on the fly.
Read PCAP from disk:
Real-time view of local live capture:
Fetch live traffic from PCAP-over-IP service, like Fox-IT's pcap-broker:
Receive packets in real-time from remote MikroTik device:
The primary output from FlowCarp is flow data, i.e. information about TCP/UDP/ICMP connections in the input data. The flow output includes information about IP addresses, port numbers and — more importantly — the application-layer protocol for each flow. Many of the supported flow output formats also include additional information, such as timestamp, duration, number of bytes and packets transferred and the Community ID for each flow.
FlowCarp can also produce alerts when it detects application layer protocols that are tagged as "MALICIOUS". This includes traffic like malware command-and-control (C2), protocols used to steal or exfiltrate information from hacked devices as well as remote management tools that are often used by malicious actors to interact with hacked computers.
The following flow and alert output formats are supported:
| Flows | Alerts | |
|---|---|---|
| Comma-separated values (CSV) | ✅ | |
| Tab-separated values (TSV) | ✅ | |
| Suricata Eve JSON | ✅ | ✅ |
| Zeek conn log (JSON) | ✅ | |
| Zeek notice log (JSON) | ✅ |
The Eve JSON output is the most versatile, as it can contain both flows and alerts, as well as additional metadata. Eve and Zeek formatted JSON outputs can be parsed and filtered with jq. You can additionally render FlowCarp's Eve JSON output to an EveBox web UI.
FlowCarp includes built-in detection of several protocols, but you can create custom models for additional protocols. Collect at least 10 sessions where the protocol is in use, and save that traffic to a single PCAP file or to a dedicated directory containing only capture files with that protocol.
Pro-tip: collect 100+ sessions from different sources for higher precision
Create a local protocol model file with "--output model" like this:
Load a custom protocol into FlowCarp with "--model":
Example commands:
There is a FlowCarp demo server available on demo.flowcarp.com. This server runs in the "--webservice" mode with a commercial license that can identify over 600 protocols. The demo server is free to use if you want to try out FlowCarp's unique state-of-the-art protocol identification functionality.
Submit packet capture files to the demo server like this:
The server responds with a Suricata Eve JSON structure containing the detected flows, protocols and alerts. The jq tool can be used to filter the JSON data from FlowCarp. As an example, this command can be used to only show alerts from when it detects protocols tagged as MALICIOUS:
This jq query shows the top 10 most common protocols and their corresponding port numbers:
FlowCarp is released under a CC BY-ND 4.0 license, which means you are free to use the software for any purpose, even commercially. But there is a limitation on how much data that can be read with the free community edition. A company, organization or individual can only analyze up to 10 000 flows per day without a commercial license. FlowCarp continues extracting flow information from PCAP data when the daily limit is reached, but it does not identify additional protocols.
Another benefit with having a commercial FlowCarp license is that it can identify much more protocols than the free community edition. All FlowCarp users get access to the free community protocol package, which can identify the following common and malicious protocols:
Common protocols
Malicious protocols
The commercial protocol packages, on the other hand, can identify up to 600+ protocols. The most advanced protocol package can even detect what we call sub-protocols, which are protocols inside of other protocols. This sub-protocol detection enables users to differentiate between different types of HTTP based communications like Windows Update, CRL, OCSP or WebSocket.
FlowCarp's sub-protocol identification feature can also detect many malicious protocols that run on top of HTTP, such as AdaptixC2, DCRat, Formbook, LummaC2, NetSupport RAT, RedLine and StealC. The sub-protocol identification feature can even detect what's running inside of a TLS encrypted session, without having to decrypt the traffic! Identification of sub-TLS protocols is much more complex than for HTTP, which is why the precision is lower. But it can usually accurately identify many legitimate as well as malicious protocols that run on top of TLS, regardless of which port is being used.