Evaluation of Variability in the Assessment of Small Color Differences

Abstract

Several psychophysical methods and the associated intra and inter-observer variability in assessing small color differences of textile samples were investigated in three phases: • Development and validation of a novel perceptually linear gray scale and incorporation into a visual assessment protocol; • Development and determination of a robust visual assessment protocol via testing different psychophysical methods and viewing conditions; • International replication experiment employing a controlled methodology to assess the repeatability of results and ascertain the degree of variability among observer panels from four continents. During the first phase, a perceptually linear gray scale, tested under simulated illuminant D65 and a neutral gray background, was developed and validated for the assessment of magnitude of perceived color differences. The perceptually linear gray scale comprised ten discrete pairs of gray samples mounted onto a suitable support structure in a perceptual order of linearly increasing contrast. This scale was used as a reference in the development of visual assessment protocols for small color differences. During the second phase, five visual assessment methodologies: AATCC standard gray scale, Jumbo scale, Jumbo scale with gap, novel perceptually linear scale, as well as pair comparison were evaluated to determine the method generating the lowest intra- and inter-observer variation. Thirty one polyester knitted samples around 13 color centers were assessed by panels of 25 observers. The visual assessment based on the use of the Jumbo scale, on average, produced the largest visual color differences for all pairs, whereas the assessments based on the standard AATCC scale produced the lowest visual color difference. A comparison of visual data obtained from different methods based on the STRESS function showed that various methodologies produced comparable visual data. However, intra- and inter-observer variability significantly decreased when the perceptually linear gray scale was employed relative to when any of the geometric scales were used. A highly controlled replication study was carried out to test, systematically and for the first time, the range and magnitude of variability among four panels of observers from different regions of the world: Colombia, Czech Republic, USA, and a group of Chinese born and raised observers. Sixty nine polyester knitted samples around 13 color centers were assessed by 25 observers in each location using the same method. On average, all panels demonstrated good agreement considering the perceptual nature of the study. The agreement between visual data from the individual panels and the total observer population increased for all observer panels suggesting that observer panels based on a mixed ethnic makeup may be better suited for investigations that involve human perceptions. Although there were differences in variability for each pair amongst observer panels, the degree of variation (intra-group standard deviation) among responses from any panel compared to the mean response of that panel was not significantly different amongst panels. The performance of color difference equations was tested against the average visual data for each of the observer panels as well as the combined dataset using PF/3 and STRESS functions. Results showed that the ΆE*ab equation performed significantly poorer than CMC(1:1) and CIEDE2000(1:1:1) models. In addition the STRESS function showed no significant difference between CMC(1:1) and CIEDE2000(1:1:1) models for the visual data obtained. Moreover, the inclusion of upper and lower confidence levels in the combined visual data did not change the PF/3 value significantly (<1 unit) while the change in STRESS values was also insignificant (<0.1).