Abstract Test fairness has emerged as a prominent topic in the field of language testing since the 1990s. The 18th and 19th Language Testing Research Colloquiums, held in 1996 and 1997, generated a surge of research interest in this research topic. Until now, fairness has been regarded as a central value of assessment, alongside validity and reliability, in the field of language testing. Despite its importance, an accepted definition of test fairness remains elusive within the language testing community. Clarifying the definition of “fairness” is crucial for understanding the concept of “test fairness”—a foundational step for the subsequent theoretical debates and empirical research in language testing. Given the well-developed theoretical underpinnings of “fairness” in various domains, it is imperative to examine different perspectives adopted to define and operationalize fairness. Meanwhile, valuable insights could be extracted to advance research on test fairness in the realm of language testing. Following a delineation of various senses of fairness, this article reviews conceptualizations of test fairness through the lenses of educational measurement, professional standards and guidelines, and philosophy. This review article endeavors to provide insights into the theoretical and empirical inquiry of fairness in language testing.
Fairness is a complex and multifaceted concept that embraces several senses. The first sense of “fair” can be characterized as “relational”, denoting: (1) receiving what an individual merits or demerits; (2) meeting an individual’s legitimate expectation; and (3) treating like cases alike. The second sense is linked to the value of “respect”, denoting “the respect of each individual’s rights and esteem”. The third sense of “fair” can be labelled as “retrospective”, which refers to “rectifying past unfairness”. The fourth sense of “fair” is “formal”, indicating “adherence to procedural rules by players of a leveling field”. The abovementioned senses of fairness align well with the conceptualization of test fairness as proposed by scholars in the field of language testing.
Conceptualizations of fairness differ across different lenses. From the lens of educational measurement, two approaches have been used to conceptualize test fairness: a narrow approach and a broad approach. The former views test fairness as “absence of measurement inaccuracy and bias”. Whereas the latter perceives test fairness as essentially a social concern that extends beyond the psychometric properties of an assessment instrument. In a broad sense, the research scope of test fairness encompasses not only test-takers’ learning opportunities but also test use consequences. Seen through the lens of professional standards and guidelines, a fair test is characterized by the absence of construct irrelevance, as supported by legal documents, rules or guidance authorized by law, authoritative professional guidance, and requirements set by international and regional organizations. Additionally, philosophical thinking has long been concerned with the concept of fairness. Central to Kantian philosophical thought is the notion of “respect for human dignity”. Rawls contends that fairness in decision-making mechanisms is a prerequisite for achieving justice. Sen takes a pragmatic stance, advocating that the recognition and rectification of injustices should be prioritized instead of striving for perfect justice.
By delineating the definitions and theoretical propositions of fairness from the perspectives of educational measurement, professional standards and guidelines, and philosophy, this article suggests that research on test fairness should be contextualized, be supported by transparent empirical evidence, and incorporate the voices of different stakeholder groups. Test fairness is not a black-and-white concept. Instead, it exists along a continuum, and by addressing unfairness, fairer assessment instruments and practices can be expected.
He Lianzhen Zhang Juan. Theoretical Perspectives on Fairness: Implications for Fairness Research in Language Testing[J]. JOURNAL OF ZHEJIANG UNIVERSITY, 0, (): 1-.