49 citations · 109 across the 25 of their papers we have counts for
25 papers
TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
Jinho Choo, JunSeung Lee, Jimyeong Kim +3
Large language models (LLMs) demonstrate strong multilingual capabilities, yet often fail to consistently generate responses in the intended language, exhibiting a phenomenon known…
Adam's Law: Textual Frequency Law on Large Language Models
Hongyuan Adam Lu, Z. L., Victor Wei +5
While textual frequency has been validated as relevant to human cognition in reading speed, its relatedness to Large Language Models (LLMs) is seldom studied. We propose a novel re…
Towards a Unified View of Preference Learning for Large Language Models: A Survey
Bofei Gao, Feifan Song, Yibo Miao +22
Large Language Models (LLMs) exhibit remarkably powerful capabilities. One of the crucial factors to achieve success is aligning the LLM's output with human preferences. This align…
Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
Bofei Gao, Feifan Song, Zhe Yang +17
Recent advancements in large language models (LLMs) have led to significant breakthroughs in mathematical reasoning capabilities. However, existing benchmarks like GSM8K or MATH ar…
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
DeepSeek-AI, Qihao Zhu, Daya Guo +37
We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, D…
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback
Bofei Gao, Zefan Cai, Runxin Xu +10
In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models. However, exi…
Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models
Lei Li, Yuqi Wang, Runxin Xu +4
Large vision-language models (LVLMs) excel across diverse tasks involving concrete images from natural scenes. However, their ability to interpret abstract figures, such as geometr…
A Double-Graph Based Framework for Frame Semantic Parsing
Ce Zheng, Xudong Chen, Runxin Xu +1
Frame semantic parsing is a fundamental NLP task, which consists of three subtasks: frame identification, argument identification and role classification. Most previous studies ten…
A Two-Stream AMR-enhanced Model for Document-level Event Argument Extraction
Runxin Xu, Peiyi Wang, Tianyu Liu +3
Most previous studies aim at extracting events from a single sentence, while document-level event extraction still remains under-explored. In this paper, we focus on extracting eve…
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs
Liang Chen, Peiyi Wang, Runxin Xu +3
As Abstract Meaning Representation (AMR) implicitly involves compound semantic annotations, we hypothesize auxiliary tasks which are semantically or formally related can better enh…
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency
Yanyang Li, Fuli Luo, Runxin Xu +3
Structured pruning has been extensively studied on monolingual pre-trained language models and is yet to be fully evaluated on their multilingual counterparts. This work investigat…
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base
Cunxiang Wang, Fuli Luo, Yanyang Li +3
Pre-trained language models (PLMs) like BERT have made significant progress in various downstream NLP tasks. However, by asking models to do cloze-style tests, recent work finds th…
Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuning
Ziyun Xu, Chengyu Wang, Minghui Qiu +4
Pre-trained Language Models (PLMs) have achieved remarkable performance for various language understanding tasks in IR systems, which require the fine-tuning process based on label…
Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation
Liang Chen, Runxin Xu, Baobao Chang
Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models. However, we argue that simply applying both techniques can be conflictin…
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression
Runxin Xu, Fuli Luo, Chengyu Wang +4
Pre-trained Language Models (PLMs) have achieved great success in various Natural Language Processing (NLP) tasks under the pre-training and fine-tuning paradigm. With large quanti…
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
Runxin Xu, Fuli Luo, Zhiyuan Zhang +4
Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus aris…
An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling
Peiyi Wang, Runxin Xu, Tianyu Liu +4
Few-Shot Sequence Labeling (FSSL) is a canonical paradigm for the tagging models, e.g., named entity recognition and slot filling, to generalize on an emerging, resource-scarce dom…
Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification
Peiyi Wang, Runxin Xu, Tianyu Liu +3
Few-Shot Event Classification (FSEC) aims at developing a model for event prediction, which can generalize to new event types with a limited number of annotated data. Existing FSEC…
Explicit Interaction Network for Aspect Sentiment Triplet Extraction
Peiyi Wang, Tianyu Liu, Damai Dai +3
Aspect Sentiment Triplet Extraction (ASTE) aims to recognize targets, their sentiment polarities and opinions explaining the sentiment from a sentence. ASTE could be naturally divi…
Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker
Runxin Xu, Tianyu Liu, Lei Li +1
Document-level event extraction aims to recognize event information from a whole piece of article. Existing methods are not effective due to two challenges of this task: a) the tar…
Volctrans Parallel Corpus Filtering System for WMT 2020
Runxin Xu, Zhuo Zhi, Jun Cao +2
In this paper, we describe our submissions to the WMT20 shared task on parallel corpus filtering and alignment for low-resource conditions. The task requires the participants to al…
Double Graph Based Reasoning for Document-level Relation Extraction
Shuang Zeng, Runxin Xu, Baobao Chang +1
Document-level relation extraction aims to extract relations among entities within a document. Different from sentence-level relation extraction, it requires reasoning over multipl…
Xiaomingbot: A Multilingual Robot News Reporter
Runxin Xu, Jun Cao, Mingxuan Wang +10
This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translat…
ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization
Xunpeng Huang, Runxin Xu, Hao Zhou +3
Due to its simplicity and outstanding ability to generalize, stochastic gradient descent (SGD) is still the most widely used optimization method despite its slow convergence. Meanw…