Optimizing Effectiveness and Efficiency of Software Testing: A Hybrid Approach

Show full item record

Title: Optimizing Effectiveness and Efficiency of Software Testing: A Hybrid Approach
Author: Bell, Kera Zakiyah
Advisors: Dr. Winser Alexander, Committee Member
Dr. Donald Bitzer, Committee Member
Dr. Laurie Williams, Committee Member
Dr. Mladen Vouk, Committee Chair
Abstract: The overall goal of software testing is to disclose defects efficiently (i.e. minimal time and cost) and effectively (i.e. maximum faults detected). It takes time to understand what to test, to generate test cases, to execute the test suite and to analyze the results. In a situation where one can parameterize inputs and variables of interest, the cost of generating random operational profile conformant test cases may be acceptable. It is typically O(N*p) where p is the number of parameters and N is the number of test cases. However, this can result in large test suites and may take long to execute. Systematic approaches tend to generate smaller test suites, thus reducing run-time and analysis costs, but may take much longer to generate since they may require a higher-level of initial expertise to develop. Is there a way to maximize benefits of both statistical and systematic approaches together to simultaneously optimize both efficiency and effectiveness? Hybrid approaches combine one or more testing techniques. A particular parameter-based systematic technique of interest is called n-wise testing. It assumes that most of the faults will be found if all or most of the parameter n-tuple values are covered by the tests. The efficiency and effectiveness of a hybrid approach that combines statistical testing with k-wise where 2 <= k < n (k-wise hybrid) testing is explored. Results show that under certain conditions the n-wise hybrid may help maximize efficiency and increase effectiveness beyond statistical testing. Using a hybrid approach seems most effective when the number of values associated with a parameter is very large. The total number of test cases produced by the hybrid technique to detect all n-way defects is approximately vˆ(n-k) times the number of test cases produced by the systematic approach alone where n is the level of testing required to guarantee coverage of all n-way constructs. A potential use of this approach is in testing for failures that may result from a complex combination of interacting parameter values (v), such as those found in security failures, and in testing highly complex network-based systems and workflows in general.
Date: 2006-11-08
Degree: PhD
Discipline: Computer Science
URI: http://www.lib.ncsu.edu/resolver/1840.16/3209


Files in this item

Files Size Format View
etd.pdf 568.3Kb PDF View/Open

This item appears in the following Collection(s)

Show full item record