416 citations · 651 across the 17 of their papers we have counts for
17 papers
Rho-1: Not All Tokens Are What You Need
Zhenghao Lin, Zhibin Gou, Yeyun Gong +8
Previous language model pre-training methods have uniformly applied a next-token prediction loss to all training tokens. Challenging this norm, we posit that "9l training". Our ini…
Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
Yiming Huang, Xiao Liu, Yeyun Gong +4
Large language models (LLMs) have shown great potential in complex reasoning tasks, yet their performance is often hampered by the scarcity of high-quality and reasoning-focused tr…
Using Left and Right Brains Together: Towards Vision and Language Planning
Jun Cen, Chenfei Wu, Xiao Liu +6
Large Language Models (LLMs) and Large Multi-modality Models (LMMs) have demonstrated remarkable decision masking capabilities on a variety of tasks. However, they inherently opera…
Competition-Level Problems are Effective LLM Evaluators
Yiming Huang, Zhenghao Lin, Xiao Liu +8
Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about these abilities and the potential data contamination problem rec…
Boosting Event Extraction with Denoised Structure-to-Text Augmentation
bo wang, Heyan Huang, Xiaochi Wei +6
Event extraction aims to recognize pre-defined event triggers and arguments from texts, which suffer from the lack of high-quality annotations. In most NLP applications, involving…
Allies: Prompting Large Language Model with Beam Search
Hao Sun, Xiao Liu, Yeyun Gong +4
With the advance of large language models (LLMs), the research field of LLM applications becomes more and more popular and the idea of constructing pipelines to accomplish complex…
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu +9
Diffusion models have gained significant attention in the realm of image generation due to their exceptional performance. Their success has been recently expanded to text generatio…
MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrievers
Kun Zhou, Xiao Liu, Yeyun Gong +4
Pre-trained Transformers (\eg BERT) have been commonly used in existing dense retrieval methods for parameter initialization, and recent studies are exploring more effective pre-tr…
LEAD: Liberal Feature-based Distillation for Dense Retrieval
Hao Sun, Xiao Liu, Yeyun Gong +6
Knowledge distillation is often used to transfer knowledge from a strong teacher model to a relatively weak student model. Traditional methods include response-based methods and fe…
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Kun Zhou, Yeyun Gong, Xiao Liu +8
Sampling proper negatives from a large document pool is vital to effectively train a dense retrieval model. However, existing negative sampling strategies suffer from the uninforma…
PROD: Progressive Distillation for Dense Retrieval
Zhenghao Lin, Yeyun Gong, Xiao Liu +8
Knowledge distillation is an effective way to transfer knowledge from a strong teacher to an efficient student model. Ideally, we expect the better the teacher is, the better the s…
Dynamic Prefix-Tuning for Generative Template-based Event Extraction
Xiao Liu, Heyan Huang, Ge Shi +1
We consider event extraction in a generative manner with template-based conditional generation. Although there is a rising trend of casting the task of event extraction as a sequen…
Dialogue State Induction Using Neural Latent Variable Models
Qingkai Min, Libo Qin, Zhiyang Teng +2
Dialogue state modules are a useful component in a task-oriented dialogue system. Traditional methods find dialogue states by manually labeling training corpora, upon which neural…
News-Driven Stock Prediction With Attention-Based Noisy Recurrent State Transition
Xiao Liu, Heyan Huang, Yue Zhang +1
We consider direct modeling of underlying stock value movement sequences over time in the news-driven stock movement prediction. A recurrent state transition model is constructed,…
Open Domain Event Extraction Using Neural Latent Variable Models
Xiao Liu, Heyan Huang, Yue Zhang
We consider open domain event extraction, the task of extracting unconstraint types of events from news clusters. A novel latent variable neural model is constructed, which is scal…
Distant Supervision for Relation Extraction with Linear Attenuation Simulation and Non-IID Relevance Embedding
Changsen Yuan, Heyan Huang, Chong Feng +2
Distant supervision for relation extraction is an efficient method to reduce labor costs and has been widely used to seek novel relational facts in large corpora, which can be iden…
Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation
Xiao Liu, Zhunchen Luo, Heyan Huang
Event extraction is of practical utility in natural language processing. In the real world, it is a common phenomenon that multiple events existing in the same sentence, where extr…