|
|
|
| Theoretical Perspectives on Fairness: Implications for Fairness Research in Language Testing |
| He Lianzhen, Zhang Juan |
| School of International Studies, Zhejiang University, Hangzhou 310058, China |
|
|
|
|
Abstract Test fairness has emerged as a prominent topic in the field of language testing since the 1990s. The 18th and 19th Language Testing Research Colloquiums, held in 1996 and 1997 respectively, generated a surge of research interest in this research topic. Until now, fairness has been regarded as a central value of assessment, alongside validity and reliability, in the field of language testing. Despite its importance, an accepted definition of test fairness remains elusive within the language testing community. Clarifying the definition of “fairness” is crucial for understanding the concept of “test fairness”—a foundational step for the subsequent theoretical debates and empirical research in language testing. Given the well-developed theoretical underpinnings of “fairness” in various domains, it is imperative to examine different perspectives adopted to define and operationalize fairness. Meanwhile, valuable insights could be extracted to advance the research on test fairness in the realm of language testing. Following a delineation of various senses of fairness, this article reviews conceptualizations of test fairness through the lenses of educational measurement, professional standards and guidelines, and philosophy. This review article endeavors to provide insights into the theoretical and empirical inquiry of fairness in language testing.Fairness is a complex and multifaceted concept that embraces several senses. The first sense of “fair” can be characterized as “relational”, denoting: (1) receiving what an individual merits or demerits; (2) meeting an individual’s legitimate expectation; and (3) treating like cases alike. The second sense is linked to the value of “respect”, denoting “the respect of each individual’s rights and esteem”. The third sense of “fair” can be labelled as “retrospective”, which refers to “rectifying past unfairness”. The fourth sense of “fair” is “formal”, indicating “adherence to procedural rules by players of a leveling field”. The abovementioned senses of fairness align well with the conceptualization of test fairness as proposed by scholars in the field of language testing.Conceptualizations of fairness differ across different lenses. From the lens of educational measurement, two approaches have been used to conceptualize test fairness: a narrow approach and a broad approach. The former views test fairness as “absence of measurement inaccuracy and bias”. Whereas the latter perceives test fairness as essentially a social concern that extends beyond the psychometric properties of an assessment instrument. In a broad sense, the research scope of test fairness encompasses not only test-takers’ learning opportunities but also test use consequences. Seen through the lens of professional standards and guidelines, a fair test is characterized by the absence of construct irrelevance, as supported by legal documents, rules or guidance authorized by law, authoritative professional guidance, and requirements set by international and regional organizations. Additionally, philosophical thinking has long been concerned with the concept of fairness. Central to Kantian philosophical thought is the notion of “respect for human dignity”. Rawls contends that fairness in decision-making mechanisms is a prerequisite for achieving justice. Sen takes a pragmatic stance, advocating that the recognition and rectification of injustices should be prioritized instead of striving for perfect justice.By delineating the definitions and theoretical propositions of fairness from the perspectives of educational measurement, professional standards and guidelines, and philosophy, this article suggests that research on test fairness should be contextualized, be supported by transparent empirical evidence, and incorporate the voices of different stakeholder groups. Test fairness is not a black-and-white concept. Instead, it exists along a continuum, and by addressing unfairness, fairer assessment instruments and practices can be expected.
|
|
Received: 13 March 2023
|
|
|
|
1 Kunnan A.， “Test fairness and Toulmin’s argument structure，” Language Testing， Vol. 27， No. 2 （2010）， pp. 183-189.
2 Zieky M. J.， “Developing fair tests，” in Lane S.， Raymond M. R. & Haladyna T. M. （eds.）， Handbook of Test Development， New York and London： Routledge， 2015， pp. 81-99.
3 Nisbet I. & Shaw S.， “Fair high-stakes assessment in the long shadow of COVID-19，” Assessment in Education： Principles， Policy & Practice， Vol. 29， No. 5 （2022）， pp. 518-533.
4 万光侠： 《公平范畴的社会哲学审视》，《探索》2001年第1期，第66-69页。
5 Nisbet I. & Shaw S.， Is Assessment Fair？ London： Sage， 2020.
6 Kunnan A.， Evaluating Language Assessments， New York and London： Routledge， 2018.
7 International Language Testing Association， “ILTA Code of Ethics，” 2000-03-01， https：//cdn.ymaws.com/www.iltaonline.com/resource/resmgr/docs/ILTA_2018_CodeOfEthics_Engli.pdf， 2023-03-13.
8 Lawlor S.， Richman S. & Richman C. L.， “The validity of using the SAT as a criterion for black and white students’ admission to college，” College Student Journal， Vol. 31， No. 4 （1997）， pp. 507-515.
9 Alon S. & Tienda M.， “Diversity， opportunity， and the shifting meritocracy in higher education，” American Sociological Review， Vol. 72， No. 4 （2007）， pp. 487-511.
10 Dworkin R.， “Affirmative action： is it fair？” The Journal of Blacks in Higher Education， No. 28 （2000）， pp. 79-88.
11 Dorans N. J. & Cook L. L.， “Introduction，” in Dorans N. J. & Cook L. L. （eds.）， Fairness in Educational Assessment and Measurement， New York： Routledge， 2016， pp. 1-6.
12 McNamara T. & Ryan K.， “Fairness versus justice in language testing： the place of English literacy in the Australian citizenship test，” Language Assessment Quarterly， Vol. 8， No. 2 （2011）， pp. 161-178.
13 McNamara T. & Roever C.， Language Testing： The Social Dimension， Oxford： Blackwell， 2006.
14 Cronbach L. J.， Essentials of Psychological Testing （4th ed.）， New York： Harper & Row， 1984.
15 Deygers B.， “Fairness and social justice in English language assessment，” in Gao X. （ed.）， Second Handbook of English Language Teaching， Cham： Springer Nature， 2019， pp. 541-569.
16 Kunnan A.， “Test fairness，” in Milanovic M. & Weir C. （eds.）， European Language Testing in a Global Context， Cambridge： Cambridge University Press， 2004， pp. 27-48.
17 Lynch B. K.， “In search of the ethical test，” Language Testing， Vol. 14， No. 3 （1997）， pp. 315-327.
18 Zwick R.， “Fairness in measurement and selection： Statistical， philosophical， and public perspectives，” Educational Measurement： Issues and Practice， Vol. 38， No. 4 （2019）， pp. 34-41.
19 Kunnan A.， “Fairness and justice for all，” in Kunnan A. （ed.）， Fairness and Validation in Language Assessment， Cambridge： Cambridge University Press， 2000， pp. 1-13.
20 Kane M.， “Validity and fairness，” Language Testing， Vol. 27， No. 2 （2010）， pp. 177-182.
21 Min S. & He L.， “Test fairness： examining differential functioning of the reading comprehension section of the GSEEE in China，” Studies in Educational Evaluation， Vol. 64 （2020）， https：//doi.org/10.1016/j.stueduc.2019.100811.
22 Xi X.， “How do we go about investigating test fairness？” Language Testing， Vol. 27， No. 2 （2010）， pp. 147-170.
23 刘海峰： 《考试立法应以史为鉴》，《湖北招生考试》2008年第6期，第1页。
24 Ofqual， “Regulatory Framework for National Assessments： National Curriculum and Early Years Foundation Stage Assessments，” 2018-03-01， https：//assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/685610/National_assessment_regulatory_framework_-_March_2018.pdf， 2023-03-13.
25 Qualifications Wales & CCEA Regulation， “Fair Access by Design： Guidance for Awarding Organisations on Designing High-quality and Inclusive Qualifications，” https：//qw-website-prod-master.azurewebsites.net/media/avuda1dk/fair-access-by-design.pdf， 2023-03-13.
26 何莲珍、张娟： 《语言测试的公平性： 内涵、公平观及研究启示》，《外语教学与研究》2022年第1期，第79-89页。
27 Joint Committee on Testing Practices， “Code of Fair Testing Practices in Education，” https：//www.apa.org/science/programs/testing/fair-testing.pdf， 2023-03-13.
28 International Test Commission， “International Guidelines for Test Use，” International Journal of Testing， Vol. 1， No. 2 （2001）， pp. 93-114.
29 Educational Testing Service， “ETS Standards for Quality and Fairness，” https：//www.ets.org/pdfs/about/standards-quality-fairness.pdf， 2023-03-13.
30 德］康德： 《道德形而上学奠基》，杨云飞译，邓晓芒校，北京：人民出版社，2013年。
31 Rawls J.， Justice as Fairness： A Restatement （2nd ed.）， Cambridge： Belknap Press of Harvard University Press， 2001.
32 Sen A.， The Idea of Justice， Cambridge： Belknap Press of Harvard University Press， 2009.
33 AERA， APA & NCME，“Standards for Educational and Psychological Testing，” https：//www.apa.org/science/programs/testing/standards， 2023-03-13.
34 Taylor L.， “Reframing the discourse and rhetoric of language testing and assessment for the public square，” Language Testing， Vol. 40， No. 1 （2023）， pp. 47-53.
35 王后雄、杨季冬： 《国际教育考试质量与公平标准的特点与趋势——基于NVivo 11.0的编码分析》，《现代教育管理》2020年第3期，第45-51页。
36 Stobart G.， “Fairness in multicultural assessment systems，” Assessment in Education， Vol. 12， No. 3 （2005）， pp. 275-287.
37 Kunnan A.， “Language assessment from a wider context，” in Hinkel E. （ed.）， Handbook of Research in Second Language Learning， Mahwah： Lawrence Erlbaum， 2005， pp. 779-794.
38 McArthur J.， Assessment for Social Justice： Perspectives and Practices Within Higher Education， London： Bloomsbury Academic， 2018.
39 Sen A.， “Values and justice，” Journal of Economic Methodology， Vol. 19， No. 2 （2012）， pp. 101-108.
40 Deygers B.， Assessing High-stakes Assumptions： A Longitudinal Mixed-methods Study of University Entrance Language Tests， and of the Policy That Relies on Them， https：//lirias.kuleuven.be/retrieve/456108.
41 Deygers B.， van den Branden K. & van Gorp K.， “University entrance language tests： a matter of justice，” Language Testing， Vol. 35， No. 4 （2018）， pp. 449-476.
42 Sireci S. & Randall J.， “Evolving notions of fairness in testing in the United States，” in Clauser B. E. & Bunch M. B. （eds.）， The History of Educational Measurement： Key Advancements in Theory， Policy， and Practice， New York： Routledge， 2021， pp. 111-135.
43 Rawls J.， Political Liberalism， New York： Columbia University Press， 2005. |
|
|
|
2024, Vol. 54 
