Intrusion Detection Systems
Misuse-based Intrusion Detection
[Roesch-1999]
M. Roesch,
“Snort - Lightweight Intrusion Detection for Networks”,
Proceedings of the 13th USENIX conference on System administration (LISA),
Seattle, Washington, USA, November 1999, pp. 229-238
ResiliNets Keywords: signature-based network intrusion detection
Abstract: "Network intrusion detection systems (NIDS) are an important part of any network security architecture. They provide a layer of defense which monitors network traffic for predefined suspicious activity or patterns, and alert system administrators when potential hostile traffic is detected. Commercial NIDS have many differences, but Information Systems departments must face the commonalities that they share such as significant system footprint, complex deployment and high monetary cost. Snort was designed to address these issues."
Notes: An open-source signature-based network intrusion detection system
[Paxson-1999]
V. Paxson,
“Bro: A System for Detecting Network Intruders in Real-Time”,
Computer Networks, vol. 31, (23-24), December 1999, pp. 2435-2463
ResiliNets Keywords: signature-based intrusion detection
Abstract: "We describe Bro, a stand-alone system for detecting network intruders in real-time by passively monitoring a network link over which the intruder's traffic transits. We give an overview of the system's design, which emphasizes high-speed (FDDI-rate) monitoring, real-time notification, clear separation between mechanism and policy, and extensibility. To achieve these ends, Bro is divided into an ``event engine that reduces a kernel-filtered network traffic stream into a series of higher-level events, and a ``policy script interpreter that interprets event handlers written in a specialized language used to express a site's security policy. Event handlers can update state information, synthesize new events, record information to disk, and generate real-time notifications via syslog. We also discuss a number of attacks that attempt to subvert passive monitoring systems and defenses against these, and give particulars of how Bro analyzes the six applications integrated into it so far: Finger, FTP, Portmapper, Ident, Telnet and Rlogin. The system is publicly available in source code form."
Notes: The best paper to read to get an overview of Bro, which is a very cleanly designed (and consequently easy to understand) network intrusion detection system.
[Dreger-Kreibich-Paxson-Sommer-2005]
Holger Dreger, Christian Keibich, Vern Paxson, Robin Sommer,
“Enhancing the Accuracy of Network-based Intrusion Detection with Host-based Context”,
Proceedings of the Conference on
Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA) 2005,
Vienna, Austria, July 2005.
ResiliNets Keywords: signature-based intrusion detection
Abstract: "In the recent past, both network- and host-based approaches to intrusion detection have received much attention in the network security community. No approach, taken exclusively, provides a satisfactory solution: network-based systems are prone to evasion, while hostbased solutions suffer from scalability and maintenance problems. In this paper we present an integrated approach, leveraging the best of both worlds: we preserve the advantages of network-based detection, but alleviate its weaknesses by improving the accuracy of the traffic analysis with specific host-based context. Our framework preserves a separation of policy from mechanism, is highly configurable and more flexible than sensor/manager-based architectures, and imposes a low overhead on the involved end hosts. We include a case study of our approach for a notoriously hard problem for purely network-based systems: the correct processing of HTTP requests."
Notes: A nice paper on providing host-based context to Bro (a network-based intrusion detection system.) The paper nicely describes the motivation for including host-based context when carrying out network-based intrusion detection.
[Sommer-Paxson-2003 (doi)]
R. Sommer, V. Paxson,
“Enhancing Byte-Level Network Intrusion Detection Signatures with Context”,
Proceedings of the 10th ACM conference on Computer and Communications Security (CCS),
Washington D.C., USA, 2003, pp. 262-271
Keywords: Bro, Network Intrusion Detection, Pattern Matching, Security, Signatures, Snort, Evaluation
Abstract: "Many network intrusion detection systems (NIDS) use byte sequences as signatures to detect malicious activity. While being highly efficient, they tend to suffer from a high false-positive rate. We develop the concept of contextual signatures as an improvement of string-based signature-matching. Rather than matching fixed strings in isolation, we augment the matching process with additional context. When designing an efficient signature engine for the NIDS bro, we provide low-level context by using regular expressions for matching, and high-level context by taking advantage of the semantic information made available by bro's protocol analysis and scripting language. Therewith, we greatly enhance the signature's expressiveness and hence the ability to reduce false positives. We present several examples such as matching requests with replies, using knowledge of the environment, defining dependencies between signatures to model step-wise attacks, and recognizing exploit scans.To leverage existing efforts, we convert the comprehensive signature set of the popular freeware NIDS snort into bro's language. While this does not provide us with improved signatures by itself, we reap an established base to build upon. Consequently, we evaluate our work by comparing to snort, discussing in the process several general problems of comparing different NIDSs."
Notes: Signature-based IDSs can generate high numbers of false positives. This paper describes how signatures can be extended using additional context, such as connection state and regular expressions, to improve intrusion detection accuracy. This is done using Bro.
Anomaly-based Intrusion Detection
[Lunt-Tamaru-Gilham-Jagannathan-Jalali-Neumann-Javitz-Valdes-Garvey-1992]
Fred Gilham, Peter Neumann, Alfonso Valdes, Teresa F. Lunt, Ann Tamaru, R. Jagannathan, Caveh Jalali, Harold S. Javitz & Thomas D. Garvey,
“A Real-time Intrusion-Detection Expert System (IDES) - Final Technical Report”,
Technical Report,
Computer Science Laboratory, SRI International, Menlo Park, CA, February 1992
ResiliNets Keywords: anomaly-based intrusion detection
Abstract: "SRI International has designed and developed a real-time intrusion-detection expert system (IDES). IDES is a stand-alone system that observes user behavior on one or more monitored computer systems and flags suspicious events. IDES monitors the activities of individual users, groups, remote hosts and entire systems, and detects suspected security violations, by both insiders and outsiders, as they occur. IDES adaptively learns users’ behavior patterns over time and detects behavior that deviates from these patterns. IDES also has a rule-based component that can be used to encode information about known system vulnerabilities and intrusion scenarios. Integrating the two approaches makes IDES a comprehensive system for detecting intrusions as well as misuse by authorized users. IDES has been enhanced to run under GLU, a platform supporting distributed, parallel computation; GLU enhances configuration flexibility and system fault tolerance. This final report is a deliverable item for work supported by the U.S. Navy, SPAWAR, which funded SRI through U.S. Government Contract No. N00039-89-C-0050."
Notes: One of the seminal works on anomaly-based intrusion detection.
[Sekar-Gupta-Frullo-Shanbhag-Tiwari-Yhang-Zhou-2002 (doi)]
R. Sekar, A. Gupta, J. Frullo, T. Shanbhag, A. Tiwari, H. Yang, S. Zhou,
“Specification-based Anomaly Detection: A New Approach for Detecting Network Intrusions”,
Proceedings of the 9th ACM conference on Computer and Communications Security (CCS) 2002,
Washington, DC, USA, 2002, pp. 265 - 274
ResiliNets Keywords: anomaly-based intrusion detection
Keywords: Intrusion detection, anomaly detection, network monitoring
Abstract: "Unlike signature or misuse based intrusion detection techniques, anomaly detection is capable of detecting novel attacks. However, the use of anomaly detection in practice is hampered by a high rate of false alarms. Specification-based techniques have been shown to produce a low rate of false alarms, but are not as effective as anomaly detection in detecting novel attacks, especially when it comes to network probing and denial-of-service attacks. This paper presents a new approach that combines specification-based and anomaly-based intrusion detection, mitigating the weaknesses of the two approaches while magnifying their strengths. Our approach begins with state-machine specifications of network protocols, and augments these state machines with information about statistics that need to be maintained to detect anomalies. We present a specification language in which all of this information can be captured in a succinct manner. We demonstrate the effectiveness of the approach on the 1999 Lincoln Labs intrusion detection evaluation data, where we are able to detect all of the probing and denial-of-service attacks with a low rate of false alarms (less than 10 per day). Whereas feature selection was a crucial step that required a great deal of expertise and insight in the case of previous anomaly detection approaches, we show that the use of protocol specifications in our approach simplifies this problem. Moreover, the machine learning component of our approach is robust enough to operate without human supervision, and fast enough that no sampling techniques need to be employed. As further evidence of effectiveness, we present results of applying our approach to detect stealthy email viruses in an intranet environment."
Notes: There are two main approaches to intrusion detection -- anomaly and signature-based detection. Anomaly detection-based techniques are typified with high false positive rates. This paper describes an approach to anomaly detection where the feature space that is monitored is constrained by a specification of normal behaviour (using EFSA), which is annotated with learnt probabilistic normal behaviour. One of the suggested benefits of this approach to intrusion detection is fewer false positives than standard anomaly detection approaches.
[Zhang-Lee-Huang-2003 (doi) .]
Y. Zhang, W. Lee, Y. Huang,
“Intrusion detection techniques for mobile wireless networks”,
ACM/Kluwer Wireless Networks Journal, vol.9, #5, September 2003, pp. 545 - 556
ResiliNets Keywords: anomaly-based intrusion detection
Keywords: Intrusion detection, intrusion response, cooperative detection, anomaly detection, mobile ad-hoc networks
Abstract: "The rapid proliferation of wireless networks and mobile computing applications has changed the landscape of network security. The traditional way of protecting networks with firewalls and encryption software is no longer sufficient and effective. We need to search for new architecture and mechanisms to protect the wireless networks and mobile computing application. In this paper, we examine the vulnerabilities of wireless networks and argue that we must include intrusion detection in the security architecture for mobile computing environment. We have developed such an architecture and evaluated a key mechanism in this architecture, anomaly detection for mobile ad-hoc network, through simulation experiments."
Notes: Interesting paper that discusses network intrusion detection in mobile ad-hoc networks. The argument for doing intrusion detection in these types of networks is reasonably well argued with some example attacks presented. An example is shown where device mobility information (from GPS, for example) is used as a way to detect anomalous behaviour. Using this kind of context (in addition to just network traffic events) is interesting Also, there are elements of cooperative detection presented, which differs from most of the traditional approaches to intrusion detection.
[Tapiador-Teodoro-Verdejo-2004 (doi) .]
Juan M. Estevez-Tapiador, Pedro Garcia-Teodoro, Jesus E. Diaz-Verdejo
“Anomaly detection methods in wired networks: a survey and taxonomy”,
Computer Communications, vol.27, #16, October 2004, pp. 1569–1584
ResiliNets Keywords: list
Keywords: Anomaly detection; Network intrusion detection; Computer and network security; Network management
Abstract: “Despite the advances reached along the last 20 years, anomaly detection in network behavior is still an immature technology, and the shortage of commercial tools thus corroborates it. Nevertheless, the benefits which could be obtained from a better understanding of the problem itself as well as the improvement of these mechanisms, especially in network security, justify the demand for more research efforts in this direction.
This article presents a survey on current anomaly detection methods for network intrusion detection in classical wired environments. After introducing the problem and elucidating its interest, a taxonomy of current solutions is presented. The outlined scheme allows us to systematically classify current detection methods as well as to study the different facets of the problem. The more relevant paradigms are subsequently discussed and illustrated through several case studies of selected systems developed in the field. The problems addressed by each of them as well as their weakest points are thus explained. Finally, this work concludes with an analysis of the problems that still remain open. Based on this discussion, some research lines are identified.”
Notes: importance and relevance to ResiliNets
[Patcha-Park-2007 (doi) .]
Animesh Patcha and Jung-Min Park
“An overview of anomaly detection techniques: Existing solutions and latest technological trends”,
Computer Networks, vol.51, #12, August 2007, pp. 3448–3470
ResiliNets Keywords: list
Keywords: Anomaly detection; Machine learning; Statistical anomaly detection; Data mining
Abstract: “As advances in networking technology help to connect the distant corners of the globe and as the Internet continues to expand its influence as a medium for communications and commerce, the threat from spammers, attackers and criminal enterprises has also grown accordingly. It is the prevalence of such threats that has made intrusion detection systems—the cyberspace’s equivalent to the burglar alarm—join ranks with firewalls as one of the fundamental technologies for network security. However, today’s commercially available intrusion detection systems are predominantly signature-based intrusion detection systems that are designed to detect known attacks by utilizing the signatures of those attacks. Such systems require frequent rule-base updates and signature updates, and are not capable of detecting unknown attacks. In contrast, anomaly detection systems, a subset of intrusion detection systems, model the normal system/network behavior which enables them to be extremely effective in finding and foiling both known as well as unknown or “zero day” attacks. While anomaly detection systems are attractive conceptually, a host of technological problems need to be overcome before they can be widely adopted. These problems include: high false alarm rate, failure to scale to gigabit speeds, etc. In this paper, we provide a comprehensive survey of anomaly detection systems and hybrid intrusion detection systems of the recent past and present. We also discuss recent technological trends in anomaly detection and identify open problems and challenges in this area.”
Notes: importance and relevance to ResiliNets
[Kemmerer-Vigna-2002 (doi) .]
Richard A. Kemmerer and Giovanni Vigna
“ Intrusion detection: a brief history and overview ”,
IEEE Security and Privacy, vol.35, #4, April 2002, pp. 27–30
ResiliNets Keywords: list
Keywords: data collection , data security , intrusion detection , network wide analysis , system effectiveness
Abstract: “The goal of intrusion detection is seemingly simple: to detect intrusions. However, the task is difficult, and in fact intrusion detection systems do not detect intrusions at all, they only identify evidence of intrusions, either while they are in progress or after the fact. The paper considers data collection issues, intrusion detection techniques, system effectiveness and network wide analysis.”
Notes: importance and relevance to ResiliNets
[Barford-Kline-Plonka-Ron-2002 (doi) .]
Paul Barford, Jeffery Kline, David Plonka, Amos Ron
“A signal analysis of network traffic anomalies”,
Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, IMW '02, vol. 2346/2002, pp. 71–82
ResiliNets Keywords: list
Keywords: Internet, management, measurement, network, monitoring, performance, reliability, availability, and serviceability, routing protocols
Abstract: “Identifying anomalies rapidly and accurately is critical to the efficient operation of large computer networks. Accurately characterizing important classes of anomalies greatly facilitates their identification; however, the subtleties and complexities of anomalous traffic can easily confound this process. In this paper we report results of signal analysis of four classes of network traffic anomalies: outages, flash crowds, attacks and measurement failures. Data for this study consists of IP flow and SNMP measurements collected over a six month period at the border router of a large university. Our results show that wavelet filters are quite effective at exposing the details of both ambient and anomalous traffic. Specifically, we show that a pseudo-spline filter tuned at specific aggregation levels will expose distinct characteristics of each class of anomaly. We show that an effective way of exposing anomalies is via the detection of a sharp increase in the local variance of the filtered data. We evaluate traffic anomaly signals at different points within a network based on topological distance from the anomaly source or destination. We show that anomalies can be exposed effectively even when aggregated with a large amount of additional traffic. We also compare the difference between the same traffic anomaly signals as seen in SNMP and IP flow data, and show that the more coarse-grained SNMP data can also be used to expose anomalies effectively.”
Notes: importance and relevance to ResiliNets
[Thottan-Ji-2003 (doi) .]
Marina Thottan, Chuanyi Ji
“Anomaly detection in IP networks”,
IEEE Transactions on Signal Processing, vol.51, #8, August 2003, pp. 2191–2204
ResiliNets Keywords: list
Keywords: adaptive signal processing, autoregressive processes, eigenvalues and eigenfunctions, network performance, network reliability
Abstract: “Network anomaly detection is a vibrant research area. Researchers have approached this problem using various techniques such as artificial intelligence, machine learning, and state machine modeling. In this paper, we first review these anomaly detection methods and then describe in detail a statistical signal processing technique based on abrupt change detection. We show that this signal processing technique is effective at detecting several network anomalies. Case studies from real network data that demonstrate the power of the signal processing approach to network anomaly detection are presented. The application of signal processing techniques to this area is still in its infancy, and we believe that it has great potential to enhance the field, and thereby improve the reliability of IP networks.”
Notes: importance and relevance to ResiliNets
