2 citations · 3 across the 14 of their papers we have counts for
12 papers · 1 filter
When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models
Jiaqi Wei, Xiang Zhang, Yuejin Yang +10
As pretraining scaling laws approach saturation, Test-Time Scaling (TTS) has emerged as an important direction for improving reasoning by allocating inference-time compute to a fix…
LatentMT: Machine Translation with Latent Reasoning
Wei-Rui Chen, Samar M. Magdy, Chiyu Zhang +3
Latent-reasoning looped language models (LoopLMs) offer a different scaling path for machine translation (MT): instead of increasing parameter count or emitting explicit chain-of-t…
Reflection Pretraining Enables Token-Level Self-Correction in Biological Sequence Models
Xiang Zhang, Jiaqi Wei, Yuejin Yang +8
Chain-of-Thought (CoT) prompting has significantly advanced task-solving capabilities in natural language processing with large language models. Unlike standard prompting, CoT enco…
Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5
Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data. However, knowledge disti…
Unifying Tree Search Algorithm and Reward Design for LLM Reasoning: A Survey
Jiaqi Wei, Xiang Zhang, Yuejin Yang +10
Deliberative tree search is a cornerstone of modern Large Language Model (LLM) research, driving the pivot from brute-force scaling toward algorithmic efficiency. This single parad…
Cross-Modal Consistency in Multimodal Large Language Models
Xiang Zhang, Senyu Li, Ning Shi +5
Recent developments in multimodal methodologies have marked the beginning of an exciting era for models adept at processing diverse data types, encompassing text, audio, and visual…
Autoregressive + Chain of Thought = Recurrent: Recurrence's Role in Language Models' Computability and a Revisit of Recurrent Transformer
Xiang Zhang, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan
The Transformer architecture excels in a variety of language modeling tasks, outperforming traditional neural architectures such as RNN and LSTM. This is partially due to its elimi…
LLM Performance Predictors are good initializers for Architecture Search
Ganesh Jawahar, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan +1
In this work, we utilize Large Language Models (LLMs) for a novel use case: constructing Performance Predictors (PP) that estimate the performance of specific deep neural network a…
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
Ganesh Jawahar, Haichuan Yang, Yunyang Xiong +10
Weight-sharing supernets are crucial for performance estimation in cutting-edge neural architecture search (NAS) frameworks. Despite their ability to generate diverse subnetworks w…
Cross-Platform and Cross-Domain Abusive Language Detection with Supervised Contrastive Learning
Md Tawkat Islam Khondaker, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan
The prevalence of abusive language on different online platforms has been a major concern that raises the need for automated cross-platform abusive language detection. However, pri…
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu +6
Mixture-of-Expert (MoE) models have obtained state-of-the-art performance in Neural Machine Translation (NMT) tasks. Existing works in MoE mostly consider a homogeneous design wher…
Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints
Ganesh Jawahar, Subhabrata Mukherjee, Debadeepta Dey +5
Autocomplete is a task where the user inputs a piece of text, termed prompt, which is conditioned by the model to generate semantically coherent continuation. Existing works for th…