H. Alinejad-Rokny

How to Use Parallel Data for Multilingual Pretraining

Zhihong Huang, Yao Lu, Derek F. Wong +2

High-quality human-translated parallel corpora can improve cross-lingual transfer, but are typically too small for large-scale pretraining. Machine-translated web corpora help over…

SrDetection: A Self-Referential Framework for Data Leakage Detection in Code Large Language Models

Shuaimin Li, Liyang Fan, Zeyang Li +9

Evaluating code large language models (Code LLMs) requires reliable detection of data leakage, where benchmark performance is artificially inflated by exposure to benchmark data du…

PatRe: A Full-Stage Office Action and Rebuttal Generation Benchmark for Patent Examination

Qiyao Wang, Xinyi Chen, Longze Chen +4

Patent examination is a complex, multi-stage process requiring both technical expertise and legal reasoning, increasingly challenged by rising application volumes. Prior benchmarks…

PLOT: Enhancing Preference Learning via Optimal Transport

Liang Zhu, Yuelin Bai, Xiankun Ren +6

Preference learning in Large Language Models (LLMs) has advanced significantly, yet existing methods remain limited by modest performance gains, high computational costs, hyperpara…

RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning

Yukun Chen, Jiaming Li, Longze Chen +10

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a prevailing paradigm for enhancing reasoning in Multimodal Large Language Models (MLLMs). However, relying sol…

Automatic Paper Reviewing with Heterogeneous Graph Reasoning over LLM-Simulated Reviewer-Author Debates

Shuaimin Li, Liyang Fan, Yufang Lin +5

Existing paper review methods often rely on superficial manuscript features or directly on large language models (LLMs), which are prone to hallucinations, biased scoring, and limi…

STORYTELLER: An Enhanced Plot-Planning Framework for Coherent and Cohesive Story Generation

Jiaming Li, Yukun Chen, Ziqiang Liu +10

Stories are central to human culture, serving to share ideas, preserve traditions, and foster connections. Automatic story generation, a key advancement in artificial intelligence…

Breaking the Block: Preserving Data Continuity to Train Superior SAEs for Instruct Models

Jiaming Li, Haoran Ye, Yukun Chen +5

Sparse Autoencoders (SAEs) are a cornerstone of mechanistic interpretability. Existing training methods inherit the Block Training paradigm from LLM pre-training, which introduces…

Expanding before Inferring: Enhancing Factuality in Large Language Models through Premature Layers Interpolation

Dingwei Chen, Ziqiang Liu, Feiteng Fang +6

Large Language Models (LLMs) demonstrate remarkable capabilities in text understanding and generation. However, their tendency to produce factually inconsistent outputs, commonly r…

CLaSp: In-Context Layer Skip for Self-Speculative Decoding

Longze Chen, Renke Shan, Huiming Wang +6

Speculative decoding (SD) is a promising method for accelerating the decoding process of Large Language Models (LLMs). The efficiency of SD primarily hinges on the consistency betw…

Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity

Feiteng Fang, Dingwei Chen, Xiang Huang +10

Currently, most reinforcement learning tasks focus on domains like mathematics and programming, where verification is relatively straightforward. However, in subjective tasks such…

EVADE-Bench: Multimodal Benchmark for Evaluating and Enhancing Evasive Content Detection

Ancheng Xu, Zhihao Yang, Jingpeng Li +9

E-commerce platforms increasingly rely on Large Language Models (LLMs) and Vision Language Models (VLMs) to detect illicit or misleading product content. However, these models rema…

IPBench: Benchmarking the Knowledge of Large Language Models in Intellectual Property

Qiyao Wang, Guhong Chen, Hongbo Wang +20

Intellectual Property (IP) is a highly specialized domain that integrates technical and legal knowledge, making it inherently complex and knowledge-intensive. Recent advancements i…

Quantification of Large Language Model Distillation

Sunbowen Lee, Junting Zhou, Chang Ao +11

Model distillation is a fundamental technique in building large language models (LLMs), transferring knowledge from a teacher model to a student model. However, distillation can le…

xJailbreak: Representation Space Guided Reinforcement Learning for Interpretable LLM Jailbreaking

Sunbowen Lee, Shiwen Ni, Chi Wei +7

Safety alignment mechanism are essential for preventing large language models (LLMs) from generating harmful information or unethical content. However, cleverly crafted prompts can…

OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis

Run Luo, Ting-En Lin, Haonan Zhang +10

Recent advancements in omnimodal learning have significantly improved understanding and generation across images, text, and speech, yet these developments remain predominantly conf…

Small Language Model as Data Prospector for Large Language Model

Shiwen Ni, Haihong Wu, Di Yang +3

The quality of instruction data directly affects the performance of fine-tuned Large Language Models (LLMs). Previously, \cite{li2023one} proposed \texttt{NUGGETS}, which identifie…

AutoPatent: A Multi-Agent Framework for Automatic Patent Generation

Qiyao Wang, Shiwen Ni, Huaren Liu +8

As the capabilities of Large Language Models (LLMs) continue to advance, the field of patent processing has garnered increased attention within the natural language processing comm…

PersonaMath: Boosting Mathematical Reasoning via Persona-Driven Data Augmentation

Jing Luo, Longze Chen, Run Luo +12

While closed-source Large Language Models (LLMs) demonstrate strong mathematical problem-solving abilities, open-source models still face challenges with such tasks. To bridge this…

Lower Layers Matter: Alleviating Hallucination via Multi-Layer Fusion Contrastive Decoding with Truthfulness Refocused

Dingwei Chen, Feiteng Fang, Shiwen Ni +6

Large Language Models (LLMs) have demonstrated exceptional performance across various natural language processing tasks. However, they occasionally generate inaccurate and counterf…

AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents

Guhong Chen, Liyang Fan, Zihan Gong +8

Current research in LLM-based simulation systems lacks comprehensive solutions for modeling real-world court proceedings, while existing legal language models struggle with dynamic…