10 papers
Small Models Scout Bottleneck Order for Large-Model Data Control
Seungmin Choi, Jiwon Sung, Muhammad Umer +4
Small proxy models are commonly used to identify data mixtures for larger-scale training. We ask whether their training trajectories reveal another transferable structure: the orde…
ItDPDM: Information-Theoretic Discrete Poisson Diffusion Model
Sagnik Bhattacharya, Abhiram Gorle, Ahsan Bilal +3
Generative modeling of non-negative, discrete data, such as symbolic music, remains challenging due to two persistent limitations in existing methods. Firstly, many approaches rely…
On the Fundamental Limits of LLMs at Scale
Muhammad Ahmed Mohsin, Muhammad Umer, Ahsan Bilal +13
Large Language Models (LLMs) have benefited enormously from scaling, yet these gains are bounded by five fundamental limitations: (1) hallucination, (2) context compression, (3) re…
An Information-Theoretic Perspective on LLM Tokenizers
Mete Erdogan, Abhiram Gorle, Shubham Chandak +2
Large language model (LLM) tokenizers act as structured compressors: by mapping text to discrete token sequences, they determine token count (and thus compute and context usage) an…
minPIC: Towards Optimal Power Allocation in Multi-User Interference Channels
Sagnik Bhattacharya, Abhiram Rao Gorle, John M. Cioffi
6G envisions massive cell-free networks with spatially nested multiple access (MAC) and broadcast (BC) channels without centralized coordination. This makes optimal resource alloca…
AI Enabled 6G for Semantic Metaverse: Prospects, Challenges and Solutions for Future Wireless VR
Muhammad Ahmed Mohsin, Sagnik Bhattacharya, Abhiram Gorle +2
Wireless support of virtual reality (VR) has challenges when a network has multiple users, particularly for 3D VR gaming, digital AI avatars, and remote team collaboration. This wo…