14 papers
Relay, Don't Route: Adaptive Population Handoff for Cost-Efficient LLM-Driven Evolution
Sichun Luo, Yi Huang, Guanzhi Deng +6
Large language model (LLM)-driven evolution has shown promise for program search and algorithm discovery, but relying on strong models throughout long evolutionary runs is costly.…
TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation
Guanzhi Deng, Haibo Wang, Kuan Wu +5
Mixture-of-Experts (MoE) language models route each token through a small subset of experts, making routing patterns useful for identifying task-relevant experts during downstream…
RA-MoE: Routing-Aligned Fine-Tuning for Multilingual Adaptation of Mixture-of-Experts Models
Guanzhi Deng, Kuan Wu, Haibo Wang +6
Mixture-of-Experts (MoE) models enable efficient LLM scaling, yet adapting them to non-English downstream tasks remains challenging. Standard multilingual fine-tuning largely ignor…
SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution
Sichun Luo, Yi Huang, Haochen Luo +7
Large Language Model (LLM)-guided evolutionary search is increasingly used for automated algorithm discovery, yet most current methods track search progress primarily through execu…
FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios
Xiangru Jian, Hao Xu, Wei Pang +13
The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to transition from simple perception to autonomous execution, yet current evaluations fai…
DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
Guanzhi Deng, Bo Li, Ronghao Chen +5
Mixture-of-Experts (MoE) has become a prominent paradigm for scaling Large Language Models (LLMs). Parameter-efficient fine-tuning methods, such as LoRA, are widely adopted to adap…
Towards Improving Interpretability of Language Model Generation through a Structured Knowledge Discovery Approach
Shuqi Liu, Han Wu, Guanzhi Deng +3
Knowledge-enhanced text generation aims to enhance the quality of generated text by utilizing internal or external knowledge sources. While language models have demonstrated impres…
REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model
Bo Li, Guanzhi Deng, Ronghao Chen +5
Understanding how Large Language Models (LLMs) perform complex reasoning and their failure mechanisms is a challenge in interpretability research. To provide a measurable geometric…
Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations
Guanzhi Deng, Mingyang Liu, Dapeng Wu +2
Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient fine-tuning method for large language models. However, its linear nature limits expressiveness. We propose LoRAN,…
KLIPA: A Knowledge Graph and LLM-Driven QA Framework for IP Analysis
Guanzhi Deng, Yi Xie, Yu-Keung Ng +6
Effectively managing intellectual property is a significant challenge. Traditional methods for patent analysis depend on labor-intensive manual searches and rigid keyword matching.…
Reasoning Meets Personalization: Unleashing the Potential of Large Reasoning Model for Personalized Generation
Sichun Luo, Guanzhi Deng, Jian Xu +4
Personalization is a critical task in modern intelligent systems, with applications spanning diverse domains, including interactions with large language models (LLMs). Recent advan…
Privacy in LLM-based Recommendation: Recent Advances and Future Directions
Sichun Luo, Wei Shao, Yuxuan Yao +9
Nowadays, large language models (LLMs) have been integrated with conventional recommendation models to improve recommendation performance. However, while most of the existing works…
Can LLM Substitute Human Labeling? A Case Study of Fine-grained Chinese Address Entity Recognition Dataset for UAV Delivery
Yuxuan Yao, Sichun Luo, Haohan Zhao +2
We present CNER-UAV, a fine-grained \textbf{C}hinese \textbf{N}ame \textbf{E}ntity \textbf{R}ecognition dataset specifically designed for the task of address resolution in \textbf{…
Evaluator for Emotionally Consistent Chatbots
Chenxiao Liu, Guanzhi Deng, Tao Ji +2
One challenge for evaluating current sequence- or dialogue-level chatbots, such as Empathetic Open-domain Conversation Models, is to determine whether the chatbot performs in an em…