Netset - A Software Framework for Automation of Network based Tests

Abstract

Testing network protocols or networking devices for generating performance benchmarks is integral to computer networking research. The general pattern in conducting such tests is configuring a desired topology over a network test-bed, deploying test software and tools onto network nodes, setting up load traffic on the network, executing the deployed software, gathering data and generating reports for analysis and comparison. A typical example of such a process is performance testing of congestion control protocols. Congestion control protocols are designed to satisfy a complex set of goals and their performance is sensitive to network topology, network delays, and router queue sizes and policies. It is thus non-trivial to construct testing procedures for congestion control protocols.

While there exist network emulation test-bed services like Emulab, WanInLab, Deter etc. they limit themselves to merely providing a set of nodes which the developer can use for testing. They provide no means of setting up the test environment, generating test traffic, gathering data and generating reports. As a result developers tend to write their own automation procedures to carry out these steps. Such tendency restricts portability, repeatability and comparison of test procedures and their results. This lack of a generally accepted practice or of a testing tool affects many parts of the networking research community including researchers, students, standardization bodies and developers.

To address this issue, we develop a tool for network based testing called Netset. The tool implements a general model which accommodates a wide range of testing processes and provides programming interface to develop specialized tools for a particular category like congestion control protocols. We then apply this tool to two testing scenarios and compare its benefits versus using a manual approach in the same scenarios.