69 citations · 248 across the 38 of their papers we have counts for
33 papers · 1 filter
Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting
Cheng Wang, Qin Liu, Wenxuan Zhou +1
Group Relative Policy Optimization (GRPO) has emerged as a promising approach for improving the reasoning capabilities of large language models. However, it struggles to effectivel…
OmniStruct: Universal Text-to-Structure Generation across Diverse Schemas
James Y. Huang, Wenxuan Zhou, Nan Xu +5
The ability of Large Language Models (LLMs) to generate structured outputs that follow arbitrary schemas is crucial to a wide range of downstream tasks that require diverse structu…
Code Execution as Grounded Supervision for LLM Reasoning
Dongwon Jung, Wenxuan Zhou, Muhao Chen
Training large language models (LLMs) with chain-of-thought (CoT) supervision has proven effective for enhancing their reasoning abilities. However, obtaining reliable and accurate…
MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
Qin Liu, Wenxuan Zhou, Nan Xu +5
One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively sel…
ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails
Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo +1
Ensuring the safety of large language models (LLMs) is critical as they are deployed in real-world applications. Existing guardrails rely on rule-based filtering or single-pass cla…
WPO: Enhancing RLHF with Weighted Preference Optimization
Wenxuan Zhou, Ravi Agrawal, Shujian Zhang +5
Reinforcement learning from human feedback (RLHF) is a promising solution to align large language models (LLMs) more closely with human values. Off-policy preference optimization,…
Offset Unlearning for Large Language Models
James Y. Huang, Wenxuan Zhou, Fei Wang +4
Despite the strong capabilities of Large Language Models (LLMs) to acquire knowledge from their training corpora, the memorization of sensitive information in the corpora such as c…
Contrastive Instruction Tuning
Tianyi Lorena Yan, Fei Wang, James Y. Huang +5
Instruction tuning has been used as a promising approach to improve the performance of large language models (LLMs) on unseen tasks. However, current LLMs exhibit limited robustnes…
GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding
Zekun Li, Wenxuan Zhou, Yao-Yi Chiang +1
Humans subconsciously engage in geospatial reasoning when reading articles. We recognize place names and their spatial relations in text and mentally associate them with their phys…
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition
Wenxuan Zhou, Sheng Zhang, Yu Gu +2
Large language models (LLMs) have demonstrated remarkable generalizability, such as understanding arbitrary entities and relations. Instruction tuning has proven effective for dist…
Robust Natural Language Understanding with Residual Attention Debiasing
Fei Wang, James Y. Huang, Tianyi Yan +2
Natural language understanding (NLU) models often suffer from unintended dataset biases. Among bias mitigation methods, ensemble-based debiasing methods, especially product-of-expe…
Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning
Tianqing Fang, Zhaowei Wang, Wenxuan Zhou +3
Event temporal reasoning aims at identifying the temporal relations between two or more events from narratives. However, knowledge conflicts arise when there is a mismatch between…
A Causal View of Entity Bias in (Large) Language Models
Fei Wang, Wenjie Mo, Yiwei Wang +2
Entity bias widely affects pretrained (large) language models, causing them to rely on (biased) parametric knowledge to make unfaithful predictions. Although causality-inspired met…
How Fragile is Relation Extraction under Entity Replacements?
Yiwei Wang, Bryan Hooi, Fei Wang +6
Relation extraction (RE) aims to extract the relations between entity names from the textual context. In principle, textual context determines the ground-truth relation and the RE…
Context-faithful Prompting for Large Language Models
Wenxuan Zhou, Sheng Zhang, Hoifung Poon +1
Large language models (LLMs) encode parametric knowledge about world facts and have shown remarkable performance in knowledge-driven NLP tasks. However, their reliance on parametri…
Continual Contrastive Finetuning Improves Low-Resource Relation Extraction
Wenxuan Zhou, Sheng Zhang, Tristan Naumann +2
Relation extraction (RE), which has relied on structurally annotated corpora for model training, has been particularly challenging in low-resource scenarios and domains. Recent lit…
Multi-hop Evidence Retrieval for Cross-document Relation Extraction
Keming Lu, I-Hung Hsu, Wenxuan Zhou +2
Relation Extraction (RE) has been extended to cross-document scenarios because many relations are not simply described in a single document. This inevitably brings the challenge of…
On-the-fly Denoising for Data Augmentation in Natural Language Understanding
Tianqing Fang, Wenxuan Zhou, Fangyu Liu +3
Data Augmentation (DA) is frequently used to provide additional training data without extra human annotation automatically. However, data augmentation may introduce noisy data that…
Parameter-Efficient Tuning with Special Token Adaptation
Xiaocong Yang, James Y. Huang, Wenxuan Zhou +1
Parameter-efficient tuning aims at updating only a small subset of parameters when adapting a pretrained model to downstream tasks. In this work, we introduce PASTA, in which we on…
GRAPHCACHE: Message Passing as Caching for Sentence-Level Relation Extraction
Yiwei Wang, Muhao Chen, Wenxuan Zhou +3
Entity types and textual context are essential properties for sentence-level relation extraction (RE). Existing work only encodes these properties within individual instances, whic…
Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis
Yiwei Wang, Muhao Chen, Wenxuan Zhou +6
Recent literature focuses on utilizing the entity information in the sentence-level relation extraction (RE), but this risks leaking superficial and spurious clues of relations. As…
Summarization as Indirect Supervision for Relation Extraction
Keming Lu, I-Hung Hsu, Wenxuan Zhou +2
Relation extraction (RE) models have been challenged by their reliance on training data with expensive annotations. Considering that summarization tasks aim at acquiring concise ex…
Answer Consolidation: Formulation and Benchmarking
Wenxuan Zhou, Qiang Ning, Heba Elfardy +2
Current question answering (QA) systems primarily consider the single-answer scenario, where each question is assumed to be paired with one correct answer. However, in many real-wo…
Prix-LM: Pretraining for Multilingual Knowledge Base Construction
Wenxuan Zhou, Fangyu Liu, Ivan Vulić +2
Knowledge bases (KBs) contain plenty of structured world and commonsense knowledge. As such, they often complement distributional text-based information and facilitate various down…