7 papers
An Information-Theoretic Perspective on LLM Tokenizers
Mete Erdogan, Abhiram Gorle, Shubham Chandak +2
Large language model (LLM) tokenizers act as structured compressors: by mapping text to discrete token sequences, they determine token count (and thus compute and context usage) an…
On the Fundamental Limits of LLMs at Scale
Muhammad Ahmed Mohsin, Muhammad Umer, Ahsan Bilal +13
Large Language Models (LLMs) have benefited enormously from scaling, yet these gains are bounded by five fundamental limitations: (1) hallucination, (2) context compression, (3) re…
minPIC: Towards Optimal Power Allocation in Multi-User Interference Channels
Sagnik Bhattacharya, Abhiram Rao Gorle, John M. Cioffi
6G envisions massive cell-free networks with spatially nested multiple access (MAC) and broadcast (BC) channels without centralized coordination. This makes optimal resource alloca…
AI Enabled 6G for Semantic Metaverse: Prospects, Challenges and Solutions for Future Wireless VR
Muhammad Ahmed Mohsin, Sagnik Bhattacharya, Abhiram Gorle +2
Wireless support of virtual reality (VR) has challenges when a network has multiple users, particularly for 3D VR gaming, digital AI avatars, and remote team collaboration. This wo…
Information-computation trade-offs in non-linear transforms
Connor Ding, Abhiram Rao Gorle, Jiwon Jeong +2
In this work, we explore the interplay between information and computation in non-linear transform-based compression for broad classes of modern information-processing tasks. We fi…
LZMidi: Compression-Based Symbolic Music Generation
Connor Ding, Abhiram Gorle, Sagnik Bhattacharya +3
Recent advances in symbolic music generation primarily rely on deep learning models such as Transformers, GANs, and diffusion models. While these approaches achieve high-quality re…