There are a number of ways an attacker can circumvent the protection of network security monitoring. He can use evasion techniques to avoid detection, or use diversion techniques to distract the defender. Here are a couple methods.
Protocol misuse. NetFlow and layer 1/2/3 statistics track hardware addresses, IP addresses, and TCP/UDP ports. Application layer detail is generally not analyzed and tracked. Any packet sent over port 80 will be assumed an HTTP packet, anything over port 53 a DNS packet, and so on. An attacker can send information over alternate ports to mask their activities. Alternatively, some protocols can be directly misused to carry out an attacker’s aims. For example, see the OzymanDNS app that tunnels SSH and transfers files over the standard DNS protocol. When application layer tracking is not enabled, an attacker has a blind spot that they can use.
Kaminsky, D. (2004, July 29). Release!, from Dan Kaminsky’s Blog: http://dankaminsky.com/2004/07/29/51/
Payload obfuscation. An attacker can also create a blind spot by obfuscating (or disguising) their application layer traffic. If application layer analysis is enabled, it may be utilizing pattern matching for application layer analysis. The attacker has to modify the packet or its payload enough to no longer match the pattern. Perhaps the simplest method is fragmentation, where the IP packet is broken into fragments. Any one fragment will not match the pattern detection. When the fragments get to the host computer, the host re-assembles the packet. The attacker’s payload is then delivered undetected.
Schiffman, M. (2010, February 15). A Brief History of Malware Obfuscation, from Cisco:http://blogs.cisco.com/security/a_brief_history_of_malware_obfuscation_part_1_of_2/
Timm, K. (2002, May 05). IDS Evasion Techniques and Tactics, from Symantec: http://www.symantec.com/connect/articles/ids-evasion-techniques-and-tactics
Denial of Service. A solid NSM solution is one that performs application layer analysis, checks for fragmentation, and negates common obfuscation techniques. An attacker then has options. Think of the smash and grab crimes, where the criminal gets in, gets what they can, and gets out quickly. The equivalent is the attacker who triggers the NSM in one area to create a distraction while they attack in another area. For example, an attacker launches a Denial of Service attack on a network link unrelated to their real target. Alternatively, the DoS targets the NSM infrastructure itself. If the attack is a quick raid of the victim’s network, such methods may pay off.
In sum, attackers can hide in the blind spots, cover their tracks, or make diversions.