Data Organization and Abstraction for Distributed Intrusion Detection

Abstract

Due to the rapid pace of technological development, we find that old systems are 'thrown away' in favor of newer technology. However, we find that data created by these earlier systems is persistent. A Digital Rosetta Stone [16] must be created to allow newer systems to correctly process data created by earlier technology. This document provides a case study of techniques that can be used to create a Digital Rosetta Stone between data formats and within a single evolving format.

The intrusion detection domain provides a solid basis for this study. In a distributed intrusion detection system, many sensors and analyzers must communicate with each other. The Intrusion Detection Message Exchange Format (IDMEF) is a standardized XML format for such communication. To its detriment, the IDMEF specification has been evolving since its inception. Also, the XML parsing during queries can be cumbersome and hinder intrusion detection. Therefore, two Digital Rosetta Stones were created. One migrates information between different versions of the IDMEF standard. The other translates IDMEF XML information into a relational database management system to improve query performance.