The Effects of Mode of Administration on Timed Cognitive Ability Tests

Abstract

Although widely used, there exists very little published research on the equivalence of web-based cognitive ability tests used for employee selection to their original paper-and-pencil versions. This issue is even further complicated by the limited research into the effects of proctoring on these types of tests. To investigate this issue, data was analyzed from the Wonderlic Personnel Test (WPT) and the Wonderlic Personnel Test-Quicktest (WPT-Q). Using the Differential Functioning of Items and Test (DFIT) procedure, data from 325 paper-and-pencil WPT administrations were compared to 325 web-based proctored administrations of the test. To check for the effects of proctoring, 108 proctored administrations of the WPT-Q were compared to 104 unproctored administrations again using the DFIT procedure. The results indicate that although the differences in administration produced low levels of differential item functioning (DIF), there is enough DIF to warrant conducting new validation studies when the mode of administration is changed.