9 citations · 26 across the 19 of their papers we have counts for
19 papers
Muse Spark Safety & Preparedness Report
Cristina Menghini, Peter Ney, Hamza Kwisaba +117
Muse Spark is the latest large language model developed by Meta. In this report, we first present evaluations for catastrophic risk domains under Meta's Advanced AI Scaling Framewo…
Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training
Junkai Zhang, Zihao Wang, Lin Gui +7
Reinforcement fine-tuning (RFT) often suffers from reward over-optimization, where a policy model hacks the reward signals to achieve high scores while producing low-quality output…
MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs
Ved Sirdeshmukh, Kaustubh Deshpande, Johannes Mols +7
We present MultiChallenge, a pioneering benchmark evaluating large language models (LLMs) on conducting multi-turn conversations with human users, a crucial yet underexamined capab…
Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models
Souvik Das, Lifeng Jin, Linfeng Song +3
Large language models (LLMs) exhibit impressive natural language capabilities but suffer from hallucination -- generating content ungrounded in the realities of training data. Rece…
Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
Ye Tian, Baolin Peng, Linfeng Song +4
Despite the impressive capabilities of Large Language Models (LLMs) on various tasks, they still struggle with scenarios that involves complex reasoning and planning. Recent work p…
Self-Consistency Boosts Calibration for Math Reasoning
Ante Wang, Linfeng Song, Ye Tian +5
Calibration, which establishes the correlation between accuracy and model confidence, is important for LLM development. We design three off-the-shelf calibration methods based on s…
A Knowledge Plug-and-Play Test Bed for Open-domain Dialogue Generation
Xiangci Li, Linfeng Song, Lifeng Jin +3
Knowledge-based, open-domain dialogue generation aims to build chit-chat systems that talk to humans using mined support knowledge. Many types and sources of knowledge have previou…
Collaborative decoding of critical tokens for boosting factuality of large language models
Lifeng Jin, Baolin Peng, Linfeng Song +3
The most common training pipeline for large language models includes pretraining, finetuning and aligning phases, with their respective resulting models, such as the pretrained mod…
Fine-Grained Self-Endorsement Improves Factuality and Reasoning
Ante Wang, Linfeng Song, Baolin Peng +5
This work studies improving large language model (LLM) generations at inference time by mitigating fact-conflicting hallucinations. Particularly, we propose a self-endorsement fram…
Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation
Xiaoying Zhang, Baolin Peng, Ye Tian +5
Despite showing increasingly human-like abilities, large language models (LLMs) often struggle with factual inaccuracies, i.e. "hallucinations", even when they hold relevant knowle…
Inconsistent dialogue responses and how to recover from them
Mian Zhang, Lifeng Jin, Linfeng Song +2
One critical issue for chat systems is to stay consistent about preferences, opinions, beliefs and facts of itself, which has been shown a difficult problem. In this work, we study…
TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMs
Shuyi Xie, Wenlin Yao, Yong Dai +11
Large language models (LLMs) have shown impressive capabilities across various natural language tasks. However, evaluating their alignment with human preferences remains a challeng…
Friend-training: Learning from Models of Different but Related Tasks
Mian Zhang, Lifeng Jin, Linfeng Song +3
Current self-training methods such as standard self-training, co-training, tri-training, and others often focus on improving model performance on a single task, utilizing differenc…
Learning a Grammar Inducer from Massive Uncurated Instructional Videos
Songyang Zhang, Linfeng Song, Lifeng Jin +4
Video-aided grammar induction aims to leverage video information for finding more accurate syntactic grammars for accompanying text. While previous work focuses on building systems…
Hierarchical Context Tagging for Utterance Rewriting
Lisa Jin, Linfeng Song, Lifeng Jin +2
Utterance rewriting aims to recover coreferences and omitted information from the latest turn of a multi-turn dialogue. Recently, methods that tag rather than linearly generate seq…
Distant finetuning with discourse relations for stance classification
Lifeng Jin, Kun Xu, Linfeng Song +1
Approaches for the stance classification task, an important task for understanding argumentation in debates and detecting fake news, have been relying on models which deal with ind…
Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories
Wenlin Yao, Xiaoman Pan, Lifeng Jin +3
Word Sense Disambiguation (WSD) aims to automatically identify the exact meaning of one word according to its context. Existing supervised models struggle to make correct predictio…
Domain-Adaptive Pretraining Methods for Dialogue Understanding
Han Wu, Kun Xu, Linfeng Song +3
Language models like BERT and SpanBERT pretrained on open-domain data have obtained impressive gains on various NLP tasks. In this paper, we probe the effectiveness of domain-adapt…
Video-aided Unsupervised Grammar Induction
Songyang Zhang, Linfeng Song, Lifeng Jin +3
We investigate video-aided grammar induction, which learns a constituency parser from both unlabeled text and its corresponding video. Existing methods of multi-modal grammar induc…