13 papers
Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors
Arya Labroo, Mengjie Qian, Kate Knill
Automatic speaking assessment systems are increasingly deployed in high-stakes settings to mark second language (L2) learners' speaking tests, making it critical to show that their…
Data Augmentation for L2 English Speaking Assessment using TTS
Stefano Bannò, Penny Karanasou, Mengjie Qian +2
Automated assessment of second language (L2) speaking proficiency relies on large-scale annotated speech data, which remains scarce compared to widely available written learner cor…
To Be Multimodal or Not to Be: Query-Adaptive Audio-Visual Person Retrieval via Active Modality Detection
Erfan Loweimi, Mengjie Qian, Kate Knill +7
When retrieving a person from a video archive by voice and face, should the system be multimodal or not? In real-world broadcast archives, unlike curated benchmarks, a target may b…
Who can we trust? LLM-as-a-jury for Comparative Assessment
Mengjie Qian, Guangzhi Sun, Mark J. F. Gales +1
Large language models (LLMs) are increasingly applied as automatic evaluators for natural language generation assessment often using pairwise comparative judgements. Existing appro…
Data Augmentation for Spoken Grammatical Error Correction
Penny Karanasou, Mengjie Qian, Stefano Bannò +2
While there exist strong benchmark datasets for grammatical error correction (GEC), high-quality annotated spoken datasets for Spoken GEC (SGEC) are still under-resourced. In this…
Natural Language-based Assessment of L2 Oral Proficiency using LLMs
Stefano Bannò, Rao Ma, Mengjie Qian +3
Natural language-based assessment (NLA) is an approach to second language assessment that uses instructions - expressed in the form of can-do descriptors - originally intended for…