10 papers
StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams
Jhen-Ke Lin, Chung Chun Wang
Language models increasingly make real-time decisions in applications that apply the latest answer until a newer one arrives. A late answer can prolong an outdated decision, such a…
Balance of Benchmarks: Semantic Density Reweighting for Task-Conditioned Model Comparison
Jhen-Ke Lin, Hong-Yun Lin
Model comparison increasingly relies on large collections of publicly reported benchmark scores, yet common aggregation strategies trade off evidence coverage against control over…
Grading Needs a Rubric, Not Intelligence
Jhen-Ke Lin
Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit rubric. We test this claim as…
ChartGenEval: Corruption-Tested Multi-Dimensional Feedback for Rhythm-Game Chart Generation
Jhen-Ke Lin
A generated rhythm-game chart need not reproduce one official note sequence: many note choices can fit the same song and difficulty. Reference-note agreement therefore measures rec…
BUILD-AND-FIND: An Effort-Aware Protocol for Evaluating Agent-Managed Codebases
Jhen-Ke Lin
Most coding-agent benchmarks ask whether generated code behaves correctly. That remains essential, but repository-level engineering is increasingly agent-managed: one agent writes…
Session-Level Spoken Language Assessment with a Multimodal Foundation Model via Multi-Target Learning
Hong-Yun Lin, Jhen-Ke Lin, Chung-Chun Wang +2
Spoken Language Assessment (SLA) estimates a learner's oral proficiency from spontaneous speech. The growing population of L2 English speakers has intensified the demand for reliab…
Acoustically Precise Hesitation Tagging Is Essential for End-to-End Verbatim Transcription Systems
Jhen-Ke Lin, Hao-Chien Lu, Chung-Chun Wang +2
Verbatim transcription for automatic speaking assessment demands accurate capture of disfluencies, crucial for downstream tasks like error analysis and feedback. However, many ASR…
Advancing Automated Speaking Assessment Leveraging Multifaceted Relevance and Grammar Information
Hao-Chien Lu, Jhen-Ke Lin, Hong-Yun Lin +2
Current automated speaking assessment (ASA) systems for use in multi-aspect evaluations often fail to make full use of content relevance, overlooking image or exemplar cues, and em…
The NTNU System at the S&I Challenge 2025 SLA Open Track
Hong-Yun Lin, Tien-Hong Lo, Yu-Hsuan Fang +4
A recent line of research on spoken language assessment (SLA) employs neural models such as BERT and wav2vec 2.0 (W2V) to evaluate speaking proficiency across linguistic and acoust…
A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions
Chung-Chun Wang, Jhen-Ke Lin, Hao-Chien Lu +2
Automated speaking assessment (ASA) on opinion expressions is often hampered by the scarcity of labeled recordings, which restricts prompt diversity and undermines scoring reliabil…