Session data is one of the six kinds of network security monitoring (NSM) data available to detect and respond to intrusions, and for troubleshooting, measuring and operating your customers' networks. (The other forms of NSM data are alert, full content, statistical, transaction, and extracted content.) NetFlow is Cisco's preferred method for providing session data, although the open source community has software to generate and collect NetFlow records as well. In this article I will demonstrate how to deploy an open source NetFlow probe and an open source NetFlow collector, as well as briefly describe and compare NetFlow v5 and v9.
For demonstration purposes, I deployed Damien Miller's open source NetFlow probe Softflowd and Peter Haag's open source NetFlow collector/analyzer suite Nfdump on a FreeBSD 7.2 system. I installed Softflowd using the FreeBSD net-mgmt/softflowd 0.9.8 port, and I installed Nfdump from source using the nfdump-1.6b-snapshot-20090619.tar.gz snapshot.
To use NetFlow, you need the following: 1) a probe to generate records; 2) a collector to accept and store them; and 3) an analyzer to read them. For this article I will run two NetFlow probes in parallel and two NetFlow collectors in parallel, in order to show NetFlow v5 and NetFlow v9 in action.
1. First, I set up the two NetFlow collectors using Nfdump's nfcapd program.
I run the first nfcapd instance in one terminal with it in the foreground so I can watch it, if necessary. I tell it to listen on port 9995 UDP and save what it sees in the /home/analyst/v5 directory.
In a second terminal, I start a second nfcapd instance and tell it to listen on port 9999 UDP while saving records to the /home/analyst/v9 directory.
2. For demonstration purposes, I start two NetFlow probes using softflowd.
I run the first softflowd instanc...
To continue reading for free, register below or login
To read more you must become a member of SearchNetworkingChannel.com
e in a third terminal and tell it to generate NetFlow v5 records, exporting them to 192.168.201.128 port 9995 UDP, where my first nfcapd listens.
I run the second softflowd instance in a fourth terminal and tell it to generate NetFlow v9 records, exporting them to 192.168.201.128 port 9999 UDP, where my second nfcapd listens.
3. After these records appear, I terminate all four programs using ctrl-C. I use the Nfdump analyzer to read the v5 and v9 records.
First, we see the v5 results:
Next we see the v9 results.:
As you can see, NetFlow records of both types record a flow start timestamp, a duration, source IP and port, destination IP and port, along with packet and byte counts. This is the basic sort of information that summarizes conversations on the network.
There's clearly a problem with the duration field for the last four v9 records. The purpose of this exercise was to demonstrate how to generate and collect NetFlow v5 and v9 records, but the result may actually be to demonstrate an issue with either Softflowd or Nfdump. This is a good reminder to validate tools before relying upon them in production environments.
The value of NetFlow v9
If we steer clear of the duration issues, however, the question might be: What value is NetFlow v9? The difference between the two formats is that v5 is a fixed format while v9 can be defined using templates. Softflowd does not appear to support defining arbitrary flow templates on the probe side, but another NetFlow probe, like a Cisco router, would support creating templates. Such configuration allows a network or security administrator to define the fields he or she wants to be exported. Refer to Cisco's document NetFlow v9 Export Format for more information on v9 templates. On the open source collector side, Nfdump has extended its support for various templates, so that is an advantage of using newer versions of that NetFlow suite.
Beyond NetFlow v9, the IETF is building a new Internet Protocol Flow Information Export (IPFIX) standard using NetFlow v9 as the base. The University of Waikato is building an IPFIX probe (called a "meter" in IPFIX-speak) called Maji. However, it only includes very simple IPFIX collectors with its suite, and no analyzers are bundled.
Cisco also appears to be pushing ahead with its new technology called Flexible NetFlow, which I first mentioned on my blog in 2006. Some commercial vendors appear to be working with Cisco to leverage this enhancement, which has the potential to turn FNF-capable devices into packet inspection and classification engines. Thus far, I have not seen any open source FNF implementations, since even NetFlow v9 has not yet been adopted by many organizations.
Richard Bejtlich is the founder of TaoSecurity, author of several books on network security monitoring, including Extrusion Detection: Security Monitoring for Internal Intrusions, and operator of the TaoSecurity blog.
