1 citations · 2 across the 14 of their papers we have counts for
14 papers
Almost Free State Prediction Separation
John Langford, Nathan Godey, Giovanni Monea +5
State--prediction separation (SPS) relieves a language model's hidden state of two competing burdens---summarizing the context and predicting the next token---by splitting the forw…
Full-bandwidth transformer
Xi Wang, Ziyang Cai, Zheng Zhan +5
Autoregressive transformers compute along two axes: horizontally across generated tokens, and vertically through model depth. Dense attention gives each token broad horizontal acce…
Multimodal Diffusion to Mutually Enhance Polarized Light and Low Resolution EBSD Data
Harry Dong, Timofey Efimov, Megna Shah +4
In spite of the utility of 3-D electron back-scattered diffraction (EBSD) microscopy, the data collection process can be time-consuming with serial-sectioning. Hence, it is natural…
STEM: Scaling Transformers with Embedding Modules
Ranajoy Sadhukhan, Sheng Cao, Harry Dong +5
Fine-grained sparsity promises higher parametric capacity without proportional per-token compute, but often suffers from training instability, load balancing, and communication ove…
Generalized Parallel Scaling with Interdependent Generations
Harry Dong, David Brandfonbrener, Eryk Helenowski +5
Parallel LLM inference scaling involves sampling a set of responses for a single input prompt. However, these parallel responses tend to be generated independently from e…
Scalable LLM Reasoning Acceleration with Low-rank Distillation
Harry Dong, Bilge Acun, Beidi Chen +1
Due to long generations, large language model (LLM) math reasoning demands significant computational resources and time. While many existing efficient inference methods have been d…
Towards Low-bit Communication for Tensor Parallel LLM Inference
Harry Dong, Tyler Johnson, Minsik Cho +1
Tensor parallelism provides an effective way to increase server large language model (LLM) inference efficiency despite adding an additional communication cost. However, as server…
Leveraging Multimodal Diffusion Models to Accelerate Imaging with Side Information
Timofey Efimov, Harry Dong, Megna Shah +3
Diffusion models have found phenomenal success as expressive priors for solving inverse problems, but their extension beyond natural images to more structured scientific domains re…
ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
Hanshi Sun, Li-Wen Chang, Wenlei Bao +6
With the widespread deployment of long-context large language models (LLMs), there has been a growing demand for efficient support of high-throughput inference. However, as the key…
Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation
Harry Dong, Beidi Chen, Yuejie Chi
With the development of transformer-based large language models (LLMs), they have been applied to many fields due to their remarkable utility, but this comes at a considerable comp…
Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
Harry Dong, Xinyu Yang, Zhenyu Zhang +3
Many computational factors limit broader deployment of large language models. In this paper, we focus on a memory bottleneck imposed by the key-value (KV) cache, a computational sh…
A Lightweight Transformer for Faster and Robust EBSD Data Collection
Harry Dong, Sean Donegan, Megna Shah +1
Three dimensional electron back-scattered diffraction (EBSD) microscopy is a critical tool in many applications in materials science, yet its data quality can fluctuate greatly dur…
Deep Unfolded Tensor Robust PCA with Self-supervised Learning
Harry Dong, Megna Shah, Sean Donegan +1
Tensor robust principal component analysis (RPCA), which seeks to separate a low-rank tensor from its sparse corruptions, has been crucial in data science and machine learning wher…
Fast and Provable Tensor Robust Principal Component Analysis via Scaled Gradient Descent
Harry Dong, Tian Tong, Cong Ma +1
An increasing number of data science and machine learning problems rely on computation with tensors, which better capture the multi-way relationships and interactions of data than…