7 papers
Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuning
Congrui Du, Yang Zhang, Kaizhi Qian +1
Instruction tuning for speech language models (SLMs) is substantially more challenging than for text-based large language models (LLMs), as it requires learning a new modality and…
VISUALSKILL: Multimodal Skills for Computer-Use Agents
Ziyan Jiang, Li An, Yujian Liu +5
Computer-use agents (CUAs) approach human-level performance on standardised benchmarks but still struggle on long-horizon tasks and unseen software. Existing skill libraries addres…
KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing
Xinyi Gao, Qiucheng Wu, Lu Ding +5
Educational Question Generation (EQG) aims to synthesize customized exercise questions that enhance student learning. An effective EQG system should ideally personalize questions f…
ProsodyLM: Uncovering the Emerging Prosody Processing Capabilities in Speech Language Models
Kaizhi Qian, Xulin Fan, Junrui Ni +4
Speech language models refer to language models with speech processing and understanding capabilities. One key desirable capability for speech language models is the ability to cap…
PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play
Wei Fang, Yang Zhang, Kaizhi Qian +2
Large language models (LLMs) are increasingly integrated with specialized external tools, yet many tasks demand zero-shot tool usage with minimal or noisy documentation. Existing s…
A Hierarchical Probabilistic Framework for Incremental Knowledge Tracing in Classroom Settings
Xinyi Gao, Qiucheng Wu, Yang Zhang +4
Knowledge tracing (KT) aims to estimate a student's evolving knowledge state and predict their performance on new exercises based on performance history. Many realistic classroom s…