184 citations · 863 across the 46 of their papers we have counts for
40 papers · 1 filter
Self-Guided Test-Time Training for Long-Context LLMs
Xinyu Zhu, Zhe Xu, Xiaohan Wei +10
Long-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long…
AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism
Zhepei Wei, Wei-Lin Chen, Xinyu Zhu +1
Large language models (LLMs) are increasingly used for long-content generation (e.g., long Chain-of-Thought reasoning) where decoding efficiency becomes a critical bottleneck: Auto…
The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
Xinyu Zhu, Mengzhou Xia, Zhepei Wei +3
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for training language models (LMs) on reasoning tasks that elicit emergent long chains of thought (CoT…
Do LLM Evaluators Prefer Themselves for a Reason?
Wei-Lin Chen, Zhepei Wei, Xinyu Zhu +2
Large language models (LLMs) are increasingly used as automatic evaluators in applications such as benchmarking, reward modeling, and self-refinement. Prior work highlights a poten…
Establishing Knowledge Preference in Language Models
Sizhe Zhou, Sha Li, Yu Meng +3
Language models are known to encode a great amount of factual knowledge through pretraining. However, such knowledge might be insufficient to cater to user requests, requiring the…
InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales
Zhepei Wei, Wei-Lin Chen, Yu Meng
Retrieval-augmented generation (RAG) has shown promising potential to enhance the accuracy and factuality of language models (LMs). However, imperfect retrievers or noisy corpora c…
SimPO: Simple Preference Optimization with a Reference-Free Reward
Yu Meng, Mengzhou Xia, Danqi Chen
Direct Preference Optimization (DPO) is a widely used offline preference optimization algorithm that reparameterizes reward functions in reinforcement learning from human feedback…
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
Bowen Jin, Chulin Xie, Jiawei Zhang +8
Large language models (LLMs), while exhibiting exceptional performance, suffer from hallucinations, especially on knowledge-intensive tasks. Existing works propose to augment LLMs…
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction
Sizhe Zhou, Yu Meng, Bowen Jin +1
Relation extraction (RE) aims to identify semantic relationships between entities within text. Despite considerable advancements, existing models predominantly require extensive an…
SCStory: Self-supervised and Continual Online Story Discovery
Susik Yoon, Yu Meng, Dongha Lee +1
We present a framework SCStory for online story discovery, that helps people digest rapidly published news article streams in real-time without human annotations. To organize news…
Evaluating Large Language Models at Evaluating Instruction Following
Zhiyuan Zeng, Jiatong Yu, Tianyu Gao +3
As research in large language models (LLMs) continues to accelerate, LLM-based evaluation has emerged as a scalable and cost-effective alternative to human evaluations for comparin…
Learning Multiplex Representations on Text-Attributed Graphs with One Language Model Encoder
Bowen Jin, Wentao Zhang, Yu Zhang +3
In real-world scenarios, texts in a graph are often linked by multiple semantic relations (e.g., papers in an academic graph are referenced by other publications, written by the sa…
Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers
Yu Zhang, Bowen Jin, Xiusi Chen +4
Instead of relying on human-annotated training samples to build a classifier, weakly supervised scientific paper classification aims to classify papers only using category descript…
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Yue Yu, Yuchen Zhuang, Jieyu Zhang +5
Large language models (LLMs) have been recently leveraged as training data generators for various natural language processing (NLP) tasks. While previous research has explored diff…
Patton: Language Model Pretraining on Text-Rich Networks
Bowen Jin, Wentao Zhang, Yu Zhang +4
A real-world text corpus sometimes comprises not only text documents but also semantic links between them (e.g., academic papers in a bibliographic network are linked by citations…
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval
Yue Yu, Yuchen Zhuang, Rongzhi Zhang +3
With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks. Different from prior works that generate training data…
PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training
Yunyi Zhang, Minhao Jiang, Yu Meng +2
Weakly-supervised text classification trains a classifier using the label name of each target class as the only supervision, which largely reduces human annotation efforts. Most ex…
Representation Deficiency in Masked Language Modeling
Yu Meng, Jitin Krishnan, Sinong Wang +6
Masked Language Modeling (MLM) has been one of the most prominent approaches for pretraining bidirectional text encoders due to its simplicity and effectiveness. One notable concer…
Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts
Yu Zhang, Yunyi Zhang, Martin Michalski +3
Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract dis…
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning
Yu Meng, Martin Michalski, Jiaxin Huang +3
Recent studies have revealed the intriguing few-shot learning ability of pretrained language models (PLMs): They can quickly adapt to a new task when fine-tuned on a small amount o…
Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generation
Jiaxin Huang, Yu Meng, Jiawei Han
We study the problem of few-shot Fine-grained Entity Typing (FET), where only a few annotated entity mentions with contexts are given for each entity type. Recently, prompt-based t…
Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds
Yu Zhang, Yu Meng, Xuan Wang +2
Discovering latent topics from text corpora has been studied for decades. Many existing topic models adopt a fully unsupervised setting, and their discovered topics may not cater t…
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
Yu Meng, Chenyan Xiong, Payal Bajaj +4
We present a new framework AMOS that pretrains text encoders with an Adversarial learning curriculum via a Mixture Of Signals from multiple auxiliary generators. Following ELECTRA-…
Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations
Yu Meng, Yunyi Zhang, Jiaxin Huang +2
Topic models have been the prominent tools for automatic topic discovery from text corpora. Despite their effectiveness, topic models suffer from several limitations including the…