1998 DARPA Intrusion Detection Evaluation

The following three talks presented by MIT Lincoln Laboratory in December 1998 summarize the evaluation.

  • Introduction to the Evaluation [ppt]
  • Summary and Plans for 1999 [pdf]

The official guidelines for the 1998 DARPA evaluation were first made available in March 1998 and were updated throughout the following year.

  • Evaluation Schedule
  • Off-line Evaluation Plan [txt]
  • Off-line Evaluation Network Diagram [GIF] [PS] [ppt]
  • List of Simulation Network Hosts (Names and IP addresses)
  • Real-time Evaluation Plan [txt]
  • Real-time Evaluation Network [GIF] [PS] [ppt]

Documentation for the first sample of network traffic and audit logs that was first made available in February 1998.

Documentation for a four hour sample of network traffic and audit logs that was first made available in May 1998.

A list of attacks and a list of anomalies, with descriptions, provides further documentation of the seven weeks of training data used in the 1998 evaluation.

1999 DARPA Intrusion Detection Evaluation

The official guidelines for the 1999 DARPA evaluation. Numerous things were changed from the 1998 evaluation.

Other documents about the 1999 evaluation are available.

  • A Summary of the 1998 Evaluation with a Brief Outline of Changes for the 1999 Evaluation is available in PDF Format.

    A table of stealthy U2R attack instances, showing how each attack instance was made to be stealthy with respect to the network sniffer based Intrusion Detection systems.

An attack database is now available online. This attack taxonomy is based on the 1998 - 1999 training data and incorporates attack descriptions from Kris Kendall's thesis. The database includes attacks considered "new" in the 1999 Evaluation.

The Master's Thesis of Kris Kendall contains descriptions of all the attacks used in the 1998 evaluation and a useful taxonomy of attacks. The thesis is available on the publications page.

Detection Scoring Truth - List of all attack instances in the 1999 test data.

Identification Scoring Truth - Identification alert entries for all attack instances in the 1999 test data.

1999 Analysis of Windows NT Attacks

In early 2000 work was done to further analyze the detect-ability of all attacks run against the Windows NT host in the 1999 Windows NT event log auditing test data. We have compiled a table of all such attacks and the detection results in 1999 and provided a perl script that automatically locates the specific implementations of these attacks used in 1999.

  • Table of NT attack instances and detection results in 1999.
  • A Perl script for locating the 1999 NT attacks in the audit logs.

2000 Dataset One

  • List of hosts and operating systems used in this scenario.

Future Evaluations and Datasets

Plans for future Intrusion Detection Evaluations have been discussed. A significant effort is being made to step-back and ensure that evaluations of intrusion detection technology are appropriately designed and scaled to respond to the needs of DARPA and the research community. Early stages of planning were carried out in the Spring of 2000. Those helping with this early planning included DARPA, Principle Investigators in the DARPA Strategic Information Assurance (SIA) program, the Sandia Red-Team, and Lincoln Laboratory. A planning workshop, entitled the "Evaluation Re-think Workshop" was held on 23-24 May in Wisconsin. Slides from the Wisconsin meeting are available on a Schafer website.

The outcome of this meeting was that in the current year, Lincoln Laboratory was tasked to produce much needed off-line intrusion detection datasets. These datasets will provide researchers with extensive examples of attacks and background traffic. The Hawaii PI meeting presentation given at the SIA PI meeting gives the goals of and a detailed plan for producing the 2000 datasets.


top of page