A breakdown of computer-based testing and its algorithmic objectivity
by
Our familiarity with artificial intelligence (AI) and the role of algorithms in our lives is increasing, but the intrigue remains around the validity and fairness of how machines are scoring online assessments—are they accurate, are they fair? The key to the success of AI and computer-based tests is to build the infrastructure from the ground up, according to Dr Rose Clesham, Director of Academic Standards & Measurement (English Assessment) at Pearson. The systems that Pearson’s English language tests are built on are highly sophisticated algorithms, and those algorithms are turning computer information back into a form of human scores, notes Dr Clesham.
“When we train our engines, the human scorers give pieces of work – lots and lots of pieces of work – and they score them in a human way and provide a human score, and that then is used to train or to start the journey of training our systems.”
LISTEN: Episode 8 — The fairness of algorithms
Let’s dive into some of the key points that unpack computer-based testing and its role in modern assessment.
Discover the knowledge and skills you need to build a better future by linking English language learning with assessment and qualifications.
Learn more
So, why is computer-based testing so efficient?
The advantage that computers have over humans is that they can implement more processes and calculations with very few errors. This is often reflected in faster results with computer-based tests returning results in days not weeks, as is the case with traditional, paper-based testing methods.
And does computer-based testing produce better results?
According to research, computer-based testing produces more valid test results for repeated measures, and it increases test-taker motivation, which in turn heightens their willingness to be tested and increases testing participation rates.
One of the key benefits is that it’s fairer and more accurate
Computer-based tests eliminate biases and are a fairer, more scalable solution. And the accuracy and detail of computer-based tests allow for detailed feedback to be delivered, which helps to improve teaching and learning outcomes. It is widely recognised that human marking can be influenced by a range of factors, particularly when only one person scores the test taker’s performance.
The way computer-based tests automatically score has the benefit of removing this effect as it is indifferent to a test taker’s appearance and personality and is not affected by human errors or biases.
Are computer-based tests the future?
To be at the cutting edge of assessment requires investment in AI technology to push the boundaries of English language assessment for the benefit of students, teachers and other professionals who help guide learners to make the right choices and open the door to endless opportunities. Computer-based tests are a significant leap forward in leveraging technology to improve scoring, to enhance feedback and to provide desired outcomes for test takers.