1998 DARPA Off-Line Intrusion Detection Evaluation Schedule

Last Modification: 25 September 1998

This document describes the schedule for the DARPA 1998 off-line intrusion detection evaluation being conducted by Lincoln Laboratory. This revised schedule provides almost 1 1/2 months for sites to train systems with the complete set of training data and two weeks to run test data through systems and return results. The separate evaluation guidelines document provides further details on the off-line evaluation. For information on the real-time component of the DARPA 1998 evaluation being conducted by the Air Force Rome Laboratory contact Terry Champion (tgc@sappho.rl.af.mil).

07/09/98 – 9/14/98: DISTRIBUTE TRAINING DATA

Training data distributed from Lincoln Lab to all sites participating in off-line evaluation. All tcpdump and bsm data posted on the Lincoln Web Site, CD's mailed out with pascal system dump files.

10/9/98: OPTIONAL PRE-TEST

To help prepare for the final evaluation using unseen test data, we are offering all sites the ability to run a pre-test dry run. You would train your intrusion detection system using the first six weeks of training data, and then test that system using the final one week of training data. If you send the list files of the final one week back to us with your scores, we will evaluate your system and return the result. Each site will see only their own results to guarantee confidentiality. We highly recommend that sites participate in this pre-test. Results will provide feedback to each site and help us tune our analysis procedures. In addition, this will provide a sanity check that each site understands the result submission format, and it may uncover some issues concerning the definition and scoring of attacks.

10/22/98: SYSTEM DESCRIPTIONS DUE

Each site participating in the off-line evaluation provides a short (text-only) description of systems they will be evaluating to Lincoln. Included at the beginning of each description will be the following paragraph: > {name of site} commits firmly to running this system in the > 1998 DARPA Intrusion Detection Evaluation according to the > guidelines provided by MIT Lincoln Laboratory. {name of site} > will submit results to MIT Lincoln Laboratory by internet > remote file transfer no later than 11:59 pm > EST, Sunday evening, 11/8/98. This system is {or is not} the > primary entry in the evaluation from {name of site} .

Only sites that provide these descriptions and a firm commitment to return test results will receive test data. Each site must submit results from at most one primary system, and sites may submit results from up to three additional systems.

10/23/98: TEST DATA DISTRIBUTED

Test data will be distributed from Lincoln Lab to all sites participating in off-line evaluation, i.e. all sites that have provided the short descriptions and firm commitments provided above. Data will be shipped on CDROM through Federal Express. Evaluation data will not be made available on our web site.

11/8/98: RESULTS DUE BACK

Sites participating in off-line evaluation send evaluation results back to Lincoln Lab.

12/15/98: PI MEETING TO DISCUSS RESULTS

A PI meeting will be held in the Boston area. Lincoln will describe the evaluation procedure and present evaluation results to sites and sponsors, and sites will report on the their research and systems.

 

top of page