6 citations · 6 across the 19 of their papers we have counts for
9 papers · 1 filter
Entropy-Constrained Adaptive Stochastic Quantization
Ran Ben Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher +1
Adaptive stochastic quantization (ASQ) is a recently introduced quantization approach that optimizes the Mean Squared Error (MSE) for a given input while preserving unbiasedness. I…
DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reduce
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +1
Multi-hop all-reduce is the de facto backbone of large model training. As the training scale increases, the network often becomes a bottleneck, motivating the reduction of the volu…
From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
Rana Shahout, Colin Cai, Yilun Du +2
Mixture-of-Experts (MoE) models can scale parameter capacity by routing each token to a subset of experts through a learned gate function. While conditional routing reduces trainin…
Intra-request branch orchestration for efficient LLM reasoning
Weifan Jiang, Rana Shahout, Yilun Du +2
Large Language Models (LLMs) increasingly rely on inference-time reasoning algorithms such as chain-of-thought and multi-branch reasoning to improve accuracy on complex tasks. Thes…
Fast Inference for Augmented Large Language Models
Rana Shahout, Cong Liang, Shiji Xin +4
Augmented Large Language Models (LLMs) enhance the capabilities of standalone LLMs by integrating external data sources through API calls. In interactive LLM applications, efficien…
Don't Stop Me Now: Embedding Based Scheduling for LLMs
Rana Shahout, Eran Malach, Chunwei Liu +3
Efficient scheduling is crucial for interactive Large Language Model (LLM) applications, where low request completion time directly impacts user engagement. Size-based scheduling a…