NetFlow and other flow-based technologies like sFlow, Jflow, and IPFIX are specifications for collecting certain network data for monitoring and reporting, with the network switch and/or router directly providing the data. This method of network monitoring has become very popular since you can use existing resources in the network to provide data that is for the most part already being processed by these devices. However, as the old saying goes, there’s no free lunch, and this pertains to flow-based monitoring as well. Knowing what the potential issues are can help you determine if flow-based network monitoring and reporting will meet your needs.

So Why All The Different Specifications?

As with many things in networking, flow data originated from a number of proprietary, vendor-specific, definitions. Fortunately, the overall definition of the basic component, the “flow”, is shared among all the various vendor definitions. A flow is sequence of packets that has the following seven identical characteristics: source IP address, destination IP address, source port, destination port, layer 3 protocol type, TOS byte, and input logical interface. By implication, a flow is unidirectional.*

What is unique among the different flow types is the definition of the flow records, or the structure of the data actually reported from the network device, and the way that data is obtained. The most common specifications for flow records include: NetFlow, IPFIX, sFlow, and, Jflow. By dissecting the NetFlow solution, we can develop a better understanding of how flow-based information may impact your network.

Stream of Packets: Size Matters

As the source of NetFlow data, a switch or router must communicate the data to another device on the network for further processing and analysis. This results in an additional stream of packets coming from the switch or router every minute or two, depending on how it is configured. The typical packet size is around 1500 bytes, which is relatively large. These packets come in spurts ranging from tens of Kilobytes to several Megabytes of traffic for each reporting interval, depending on how many flows are monitored by the switch or router for that given interval. NetFlow packets usually contain data for 5 to 10 flows per packet.

The bottom line is that flow-based analysis generates network traffic of its own, with the volume of traffic proportional to the size of the network segment being monitored. On highly utilized network segments, this could cause undesirable results.

Sampling: The Good and the Bad

Sampling is another important factor to consider when evaluating flow-based analysis systems. The default configuration for NetFlow is to monitor and develop flow records for 100% of the packets - no sampling. But it can be configured to “1 out of k” static sampling, or the network device itself can switch to a sampling mode when network traffic gets heavy. Sampling leads to inaccuracies in reporting, and these inaccuracies can vary substantially since it all depends which flows are being ignored through the sampling. If several small flows are ignored, overall accuracy is essentially unaffected. But if a large flow happens to be missed by the sampling, significant inaccuracies can result.

The Prime Directive

All flow-based solutions share hardware resources with the prime directive of your router or switch – forwarding packets. If your router or switch is heavily utilized, it will focus first on its prime directive, compromising flow-based reporting. This can create intermittent inaccuracies in your monitoring and reporting that are very difficult to detect, affecting your ability to collect essential information from your network at a time when it is needed most.

In addition, the flow records sent from the switch or router to the flow-based processor are based on UDP packets, an unreliable transport mechanism. There are no acknowledgments with UDP, so dropped packets result in missing and inaccurate flow-based data. Remember, each NetFlow packet reports on 5 to 10 flows, so for each dropped packets many flows are ignored. And this is most likely to happen when the network is busy, compounding your inability to get an accurate picture of the current state of the network.*

Do You Need Detailed Troubleshooting?

Flow-based data is very well suited for network monitoring and reporting, given the above limitations. But what about detailed network troubleshooting? Keep in mind that flow records report on just that – the flows. Once you identify a problem within a specific flow, what then? How can you see exactly what happened, packet by packet, so that you can determine the root cause of the issue? What if you need to see the contents of certain packets to determine whether or not errors are being reported, either in configuration or in payload data? For this, you will need to move to a packet-based network analysis solution. The nice thing about a packet-based solution is that it can report all the same data as a flow-based solution, without the limitations discussed above, and provide detailed drill-down into each individual conversation, as well as into each individual packet when that level of analysis is needed. And there are always occasions when it is!