Efficient Evaluation of Highly Available Services: Fast Simulation and Testing

Abstract

Modern technologies have provided us with highly available services. Systems such as optical backbone networks, robust web servers, and reliable software can provide a service with unavailability probability lower than 10ˆ−6. Although rare, service unavailability can cause serious problems such as significant performance drop, or violation of Service Level Agreements (SLA). Moreover, providers of these services need to know the value of service unavailability probability so they can provide reasonable SLAs and corresponding Quality of Service (QoS). However, due to the extremely low values of the service unavailability probabilities, estimating them using traditional simulation or testing methods can require a vast amount of time to obtain a satisfactory confidence interval. As a result, efficient evaluation techniques are necessary.

In this dissertation, we propose efficient evaluation methods based on importance sampling (IS). For fast simulation, we introduce several types of IS tuning methods: Our static IS method, which is based on asymptotically efficient IS biasing methods for a single queue, is proven to have bounded relative error. Our adaptive IS method, which is based on guidelines of "optimal biasing", is efficient and can be widely employed. Moreover, IS methods that are stochastically optimized by simulated annealing can be used when the system is complicated, or when the knowledge of the system is limited. Finally, for performance evaluation and optimization of a system under various parameter settings, we propose a framework based on IS and metamodeling methodologies. All of these methods provided in this dissertation are verified by either proof or simulation to be both accurate and efficient.