6 citations · 18 across the 18 of their papers we have counts for
23 papers
Stochastic Teacher Intervention for Agentic On-Policy Distillation
Junnan Liu, Linhao Luo, Zhijun Chen +3
On-policy distillation (OPD) efficiently transfers capabilities from a stronger teacher to a student language model through dense token-level supervision on student-generated rollo…
Active Budget Can Kill Sensitivity: Diagnosing and Repairing TopK Sparse Autoencoder Reliability
Zhenting Huang, Junnan Liu, Qianren Mao +2
Sparse autoencoders (SAEs) are increasingly scaled to wider dictionaries to recover fine-grained structure from large language model activations. However, a feature is useful for i…
Learning to Prove, Not Just to Answer: Reinforcement Learning from Formal Verification for Natural-Language Logical Reasoning
Qili Zhang, Qianren Mao, Hanze Cai +11
Large language models (LLMs) are increasingly deployed for natural-language logical reasoning, where the final answer is easy to check but the proof behind it is not. In natural-la…
Selective Knowledge Edit Reversal via Gated Singular Vector Shrinkage
Weifeng Jiang, Ruirui Chen, Qianren Mao +3
Knowledge editing provides an efficient way to update factual knowledge in large language models. However, malicious edits may introduce safety risks, making it necessary to revers…
RiskWorld: Object-Centric Latent World Modeling for Autonomous Driving Risk Identification
Jingzheng Li, Yufei Ge, Qianren Mao +5
Autonomous driving risk identification aims to determine which observed object is likely to become safety-critical to the ego vehicle. Existing approaches typically predict scene-l…
End-to-end Conditional Diffusion for Realistic and Controllable Visual Traffic Scenario Generation
Jingzheng Li, Yufei Ge, Zhijun Chen +8
Generating closed-loop traffic scenarios that are both realistic and controllable is crucial for evaluating autonomous driving systems, especially under rare safety-critical intera…
H3D: Benchmarking Unsupervised Text Hashing for Fine-Grained Document Deduplication
Qianren Mao, Jiaxun Lyu, Junnan Liu +4
Document hashing provides compact representations for efficient similarity search and document deduplication, but existing studies rarely compare hashing pipelines under a unified…
Enhancing Multilingual Reasoning via Steerable Model Merging
Zhuoran Li, Rui Xu, Jian Yang +8
Model merging is an effective technique for composing the capabilities of a multilingual model and a reasoning model. It has achieved promising generalization in multilingual reaso…
NanoNet: Parameter-Efficient Learning with Label-Scarce Supervision for Lightweight Text Mining Model
Qianren Mao, Yashuo Luo, Ziqi Qin +12
The lightweight semi-supervised learning (LSL) strategy provides an effective approach of conserving labeled samples and minimizing model inference costs. Prior research has effect…
Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process
Zhijun Chen, Zeyu Ji, Qianren Mao +13
We propose LLM-PeerReview, an unsupervised LLM Ensemble method that selects the most ideal response from multiple LLM-generated candidates for each query, harnessing the collective…
Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation
Qianren Mao, Qili Zhang, Hanwen Hao +11
Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution for enhancing the accuracy and credibility of Large Language Models (LLMs), particularly in Questi…
Harnessing Multiple Large Language Models: A Survey on LLM Ensemble
Zhijun Chen, Xiaodong Lu, Jingzheng Li +12
LLM Ensemble -- which involves the comprehensive use of multiple large language models (LLMs), each aimed at handling user queries during downstream inference, to benefit from thei…
XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
Qili Zhang, Qianren Mao, Yangyifei Luo +15
Retrieval-augmented generation (RAG) synergizes the retrieval of pertinent data with the generative capabilities of Large Language Models (LLMs), ensuring that the generated output…
Lightweight Contenders: Navigating Semi-Supervised Text Mining through Peer Collaboration and Self Transcendence
Qianren Mao, Weifeng Jiang, Junnan Liu +5
The semi-supervised learning (SSL) strategy in lightweight models requires reducing annotated samples and facilitating cost-effective inference. However, the constraint on model pa…
KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning
Junnan Liu, Qianren Mao, Weifeng Jiang +1
Knowledge graph reasoning plays a vital role in various applications and has garnered considerable attention. Recently, path-based methods have achieved impressive performance. How…
Variational Multi-Modal Hypergraph Attention Network for Multi-Modal Relation Extraction
Qian Li, Cheng Ji, Shu Guo +5
Multi-modal relation extraction (MMRE) is a challenging task that aims to identify relations between entities in text leveraging image information. Existing methods are limited by…
Learning Federated Neural Graph Databases for Answering Complex Queries from Distributed Knowledge Graphs
Qi Hu, Weifeng Jiang, Haoran Li +6
The increasing demand for deep learning-based foundation models has highlighted the importance of efficient data retrieval mechanisms. Neural graph databases (NGDBs) offer a compel…
Bipartite Graph Pre-training for Unsupervised Extractive Summarization with Graph Convolutional Auto-Encoders
Qianren Mao, Shaobo Zhao, Jiarui Li +4
Pre-trained sentence representations are crucial for identifying significant sentences in unsupervised document extractive summarization. However, the traditional two-step paradigm…
Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labeler
Zhijun Chen, Hailong Sun, Wanhao Zhang +3
We propose a neuralized undirected graphical model called Neural-Hidden-CRF to solve the weakly-supervised sequence labeling problem. Under the umbrella of probabilistic undirected…
DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining
Weifeng Jiang, Qianren Mao, Chenghua Lin +4
Many text mining models are constructed by fine-tuning a large deep pre-trained language model (PLM) in downstream tasks. However, a significant challenge nowadays is maintaining p…
Attend and select: A segment selective transformer for microblog hashtag generation
Qianren Mao, Xi Li, Bang Liu +4
Hashtag generation aims to generate short and informal topical tags from a microblog post, in which tokens or phrases form the hashtags. These tokens or phrases may originate from…
CNTLS: A Benchmark Dataset for Abstractive or Extractive Chinese Timeline Summarization
Qianren Mao, Jiazheng Wang, Zheng Wang +3
Timeline summarization (TLS) involves creating summaries of long-running events using dated summaries from numerous news articles. However, limited data availability has significan…
Noised Consistency Training for Text Summarization
Junnan Liu, Qianren Mao, Bang Liu +3
Neural abstractive summarization methods often require large quantities of labeled training data. However, labeling large amounts of summarization data is often prohibitive due to…