2 citations · 2 across the 9 of their papers we have counts for
10 papers
FlexComp: One Model for Every Ratio in Context Compression
Kaiyan Zhao, Zhongtao Miao, Akiko Aizawa +1
Soft context compression condenses a context into a few memory tokens that a frozen LLM consumes in place of the raw text, but existing compressors fix the compression ratio at tra…
CNeo-Bench: Diagnosing Large Language Models on Chinese Neologisms
Kaiyan Zhao, Zhongtao Miao, Zheyong Xie +2
Chinese neologisms exploit diverse and unique linguistic mechanisms, such as phonetic substitution (e.g., 886 for ``bye-bye'') and visual character decomposition that are rare in o…
OPDSearch+: On-Policy Distillation with RL Refinement for Search-Augmented Reasoning
Qinglin Ye, Zhiyuan Gu, Jingjie Xia +6
Search-augmented reasoning remains difficult for small language models. On-policy distillation (OPD) from trained teachers offers a promising direction, but suffers from two issues…
RegMix-D: Dynamic Data Mixing via Proxy Training Trajectories
Kaiyan Zhao, Zhongtao Miao, Akiko Aizawa +1
Data mixture selection is critical for Large Language Model pretraining. Existing methods such as RegMix select a single static mixture by fitting a regression model on small-scale…
NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learning
Zhongtao Miao, Kaiyan Zhao, Masaaki Nagata +1
Neologism-aware machine translation aims to translate source sentences containing neologisms into target languages. This field remains underexplored compared with general machine t…
Improving Multimodal Contrastive Learning of Sentence Embeddings with Object-Phrase Alignment
Kaiyan Zhao, Zhongtao Miao, Yoshimasa Tsuruoka
Multimodal sentence embedding models typically leverage image-caption pairs in addition to textual data during training. However, such pairs often contain noise, including redundan…
Direct Quantized Training of Language Models with Stochastic Rounding
Kaiyan Zhao, Tsuguchika Tabaru, Kenichi Kobayashi +3
Although recent quantized Large Language Models (LLMs), such as BitNet, have paved the way for significant reduction in memory usage during deployment with binary or ternary weight…
Improving Arithmetic Reasoning Ability of Large Language Models through Relation Tuples, Verification and Dynamic Feedback
Zhongtao Miao, Kaiyan Zhao, Yoshimasa Tsuruoka
Current representations used in reasoning steps of large language models can mostly be categorized into two main types: (1) natural language, which is difficult to verify; and (2)…
Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment
Zhongtao Miao, Qiyu Wu, Kaiyan Zhao +2
The field of cross-lingual sentence embeddings has recently experienced significant advancements, but research concerning low-resource languages has lagged due to the scarcity of p…
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding
Kaiyan Zhao, Qiyu Wu, Xin-Qiang Cai +1
Learning multi-lingual sentence embeddings is a fundamental task in natural language processing. Recent trends in learning both mono-lingual and multi-lingual sentence embeddings a…