Validation of System Usability Scale From Expert and Ordinary User Perspective

Abstract

This meticulous and comprehensive study aims to provide a thorough overview of the validation results of the ten System Usability Scale (SUS) indicators from the perspectives of experts and ordinary users. The study meticulously compares the validation results of SUS scores in expert and ordinary user groups. The validation results of SUS scores in the expert group are derived from a comprehensive analysis of SUS evaluation results, heuristic evaluation, and interviews with three usability experts. Similarly, the validation results of SUS scores in the ordinary user group are obtained from a rigorous comparison of SUS evaluation results, remote moderated usability testing, and interviews with five ordinary users. While inconclusive, this study's findings shed light on four crucial points. First, there are no universally valid validation results of SUS scores in all SUS indicators from expert and ordinary user perspectives. Second, the ease of use indicator is the only one with a complete and valid evaluation result from the expert perspective. Third, the intuitiveness indicator, while full, yields invalid evaluation results from the standpoint of ordinary users. Fourth, a weak relationship is observed between SUS scores and the results of heuristic evaluation and usability testing. These findings, derived from a robust and meticulous methodology, are expected to significantly contribute to the existing research in application usability, particularly in the context of SUS usage.