6 citations · 18 across the 18 of their papers we have counts for
14 papers · 1 filter
Stochastic Teacher Intervention for Agentic On-Policy Distillation
Junnan Liu, Linhao Luo, Zhijun Chen +3
On-policy distillation (OPD) efficiently transfers capabilities from a stronger teacher to a student language model through dense token-level supervision on student-generated rollo…
Selective Knowledge Edit Reversal via Gated Singular Vector Shrinkage
Weifeng Jiang, Ruirui Chen, Qianren Mao +3
Knowledge editing provides an efficient way to update factual knowledge in large language models. However, malicious edits may introduce safety risks, making it necessary to revers…
Enhancing Multilingual Reasoning via Steerable Model Merging
Zhuoran Li, Rui Xu, Jian Yang +8
Model merging is an effective technique for composing the capabilities of a multilingual model and a reasoning model. It has achieved promising generalization in multilingual reaso…
Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process
Zhijun Chen, Zeyu Ji, Qianren Mao +13
We propose LLM-PeerReview, an unsupervised LLM Ensemble method that selects the most ideal response from multiple LLM-generated candidates for each query, harnessing the collective…
Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation
Qianren Mao, Qili Zhang, Hanwen Hao +11
Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution for enhancing the accuracy and credibility of Large Language Models (LLMs), particularly in Questi…
Harnessing Multiple Large Language Models: A Survey on LLM Ensemble
Zhijun Chen, Xiaodong Lu, Jingzheng Li +12
LLM Ensemble -- which involves the comprehensive use of multiple large language models (LLMs), each aimed at handling user queries during downstream inference, to benefit from thei…
XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
Qili Zhang, Qianren Mao, Yangyifei Luo +15
Retrieval-augmented generation (RAG) synergizes the retrieval of pertinent data with the generative capabilities of Large Language Models (LLMs), ensuring that the generated output…
Lightweight Contenders: Navigating Semi-Supervised Text Mining through Peer Collaboration and Self Transcendence
Qianren Mao, Weifeng Jiang, Junnan Liu +5
The semi-supervised learning (SSL) strategy in lightweight models requires reducing annotated samples and facilitating cost-effective inference. However, the constraint on model pa…
Variational Multi-Modal Hypergraph Attention Network for Multi-Modal Relation Extraction
Qian Li, Cheng Ji, Shu Guo +5
Multi-modal relation extraction (MMRE) is a challenging task that aims to identify relations between entities in text leveraging image information. Existing methods are limited by…
Bipartite Graph Pre-training for Unsupervised Extractive Summarization with Graph Convolutional Auto-Encoders
Qianren Mao, Shaobo Zhao, Jiarui Li +4
Pre-trained sentence representations are crucial for identifying significant sentences in unsupervised document extractive summarization. However, the traditional two-step paradigm…
Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labeler
Zhijun Chen, Hailong Sun, Wanhao Zhang +3
We propose a neuralized undirected graphical model called Neural-Hidden-CRF to solve the weakly-supervised sequence labeling problem. Under the umbrella of probabilistic undirected…
DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining
Weifeng Jiang, Qianren Mao, Chenghua Lin +4
Many text mining models are constructed by fine-tuning a large deep pre-trained language model (PLM) in downstream tasks. However, a significant challenge nowadays is maintaining p…
Attend and select: A segment selective transformer for microblog hashtag generation
Qianren Mao, Xi Li, Bang Liu +4
Hashtag generation aims to generate short and informal topical tags from a microblog post, in which tokens or phrases form the hashtags. These tokens or phrases may originate from…
Noised Consistency Training for Text Summarization
Junnan Liu, Qianren Mao, Bang Liu +3
Neural abstractive summarization methods often require large quantities of labeled training data. However, labeling large amounts of summarization data is often prohibitive due to…