8 papers
Draft-KV: Learning Useful Latent Communication Between Language Models
Linquan Wu, Shichang Meng, Tianxiang Jiang +7
Latent communication passes internal states between language models instead of decoded text, but higher receiver accuracy does not show that the receiver used the message content.…
FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios
Xiangru Jian, Hao Xu, Wei Pang +13
The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to transition from simple perception to autonomous execution, yet current evaluations fai…
On Temperature-Constrained Non-Deterministic Machine Translation: Potential and Evaluation
Weichuan Wang, Mingyang Liu, Linqi Song +1
In recent years, the non-deterministic properties of language models have garnered considerable attention and have shown a significant influence on real-world applications. However…
LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning
Linquan Wu, Tianxiang Jiang, Yifei Dong +6
Current multimodal latent reasoning often relies on external supervision (e.g., auxiliary images), ignoring intrinsic visual attention dynamics. In this work, we identify a critica…
DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
Guanzhi Deng, Bo Li, Ronghao Chen +5
Mixture-of-Experts (MoE) has become a prominent paradigm for scaling Large Language Models (LLMs). Parameter-efficient fine-tuning methods, such as LoRA, are widely adopted to adap…
REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model
Bo Li, Guanzhi Deng, Ronghao Chen +5
Understanding how Large Language Models (LLMs) perform complex reasoning and their failure mechanisms is a challenge in interpretability research. To provide a measurable geometric…
Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations
Guanzhi Deng, Mingyang Liu, Dapeng Wu +2
Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient fine-tuning method for large language models. However, its linear nature limits expressiveness. We propose LoRAN,…
KLIPA: A Knowledge Graph and LLM-Driven QA Framework for IP Analysis
Guanzhi Deng, Yi Xie, Yu-Keung Ng +6
Effectively managing intellectual property is a significant challenge. Traditional methods for patent analysis depend on labor-intensive manual searches and rigid keyword matching.…