22 citations · 40 across the 25 of their papers we have counts for
11 papers · 1 filter
P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formats
Yuzong Chen, Chao Fang, Xilai Dai +4
The substantial memory bandwidth and computational demands of large language models (LLMs) present critical challenges for efficient inference. To tackle this, the literature has e…
On the Role of Temperature Sampling in Test-Time Scaling
Yuheng Wu, Azalia Mirhoseini, Thierry Tambe
Large language models (LLMs) can improve reasoning at inference time through test-time scaling (TTS), where multiple reasoning traces are generated and the best one is selected. Pr…
AttAnchor: Guiding Cross-Modal Token Alignment in VLMs with Attention Anchors
Junyang Zhang, Tianyi Zhu, Thierry Tambe
A fundamental reason for the dominance of attention over RNNs and LSTMs in LLMs is its ability to capture long-range dependencies by modeling direct interactions between all tokens…
The Future of Memory: Limits and Opportunities
Samuel Dayo, Shuhan Liu, Peijing Li +5
Memory latency, bandwidth, capacity, and energy increasingly limit performance. In this paper, we reconsider proposed system architectures that consist of huge (many-terabyte to pe…
GaussianVision: Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting
Yasmine Omri, Connor Ding, Tsachy Weissman +1
Modern vision language pipelines are driven by RGB vision encoders trained on massive image text corpora. While these pipelines have enabled impressive zero-shot capabilities and s…
Towards Memory Specialization: A Case for Long-Term and Short-Term RAM
Peijing Li, Muhammad Shahir Abdurraman, Rachel Cleaveland +6
Both SRAM and DRAM have stopped scaling: there is no technical roadmap to reduce their cost (per byte/GB). As a result, memory now dominates system cost. This paper argues for a pa…
OpenGCRAM: An Open-Source Gain Cell Compiler Enabling Design-Space Exploration for AI Workloads
Xinxin Wang, Lixian Yan, Shuhan Liu +10
Gain Cell memory (GCRAM) offers higher density and lower power than SRAM, making it a promising candidate for on-chip memory in domain-specific accelerators. To support workloads w…
Token Sequence Compression for Efficient Multimodal Computing
Yasmine Omri, Parth Shroff, Thierry Tambe
The exponential growth of Large Multimodal Models (LMMs) has driven advancements in cross-modal reasoning but at significant computational costs. In this work, we focus on visual l…
GainSight: A Unified Framework for Data Lifetime Profiling and Heterogeneous Memory Composition
Peijing Li, Matthew Hung, Yiming Tan +8
As AI workloads drive increasing memory requirements, domain-specific accelerators need higher-density on-chip memory beyond what current SRAM scaling trends can provide. Simultane…
RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping
Yuzong Chen, Xilai Dai, Jake Hyun +6
The recently introduced NVFP4 format demonstrates remarkable performance and memory benefits for quantized large language model (LLM) inference. However, we observe two types of re…
BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference
Wonsuk Jang, Thierry Tambe
The rapidly increasing size of large language models (LLMs) presents significant challenges in memory usage and computational costs. Quantizing both weights and activations can add…