14 papers
Forecasting the Growth of Social Media Information Cascades: Towards Human-in-the-Loop Misinformation Triage
Ansh Gupta, Abhiram Gorle, Aayush Rajesh +1
Limited review teams must identify which emerging claims are likely to keep growing before their eventual reach is known. We center early misinformation triage on this continuation…
Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads
Yasmine Omri, Ziyu Gan, Zachary Broveak +6
LLM agents are increasingly deployed on long-horizon tasks requiring sustained reasoning over extended interaction histories. Realizing this at scale requires agents to persistentl…
An Information-Theoretic Perspective on LLM Tokenizers
Mete Erdogan, Abhiram Gorle, Shubham Chandak +2
Large language model (LLM) tokenizers act as structured compressors: by mapping text to discrete token sequences, they determine token count (and thus compute and context usage) an…
The Performance of Compression-Based Denoisers
Dan Song, Ayfer Özgür, Tsachy Weissman
We consider a denoiser that reconstructs a stationary ergodic source by lossily compressing samples of the source observed through a memoryless noisy channel. Prior work on compres…
GaussianVision: Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting
Yasmine Omri, Connor Ding, Tsachy Weissman +1
Modern vision language pipelines are driven by RGB vision encoders trained on massive image text corpora. While these pipelines have enabled impressive zero-shot capabilities and s…
Quantum sensing with a spin ensemble in a two-dimensional material
Souvik Biswas, Giovanni Scuri, Noah Huffman +12
Quantum sensing with solid-state spin defects has transformed nanoscale metrology, offering sub-wavelength spatial resolution with exceptional sensitivity to multiple signal types.…
Information-computation trade-offs in non-linear transforms
Connor Ding, Abhiram Rao Gorle, Jiwon Jeong +2
In this work, we explore the interplay between information and computation in non-linear transform-based compression for broad classes of modern information-processing tasks. We fi…
Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMs
Hao Kang, Qingru Zhang, Han Cai +4
Large language models (LLMs) have shown remarkable performance across diverse reasoning and generation tasks, and are increasingly deployed as agents in dynamic environments such a…
ItDPDM: Information-Theoretic Discrete Poisson Diffusion Model
Sagnik Bhattacharya, Abhiram Gorle, Ahsan Bilal +3
Generative modeling of non-negative, discrete data, such as symbolic music, remains challenging due to two persistent limitations in existing methods. Firstly, many approaches rely…
LZMidi: Compression-Based Symbolic Music Generation
Connor Ding, Abhiram Gorle, Sagnik Bhattacharya +3
Recent advances in symbolic music generation primarily rely on deep learning models such as Transformers, GANs, and diffusion models. While these approaches achieve high-quality re…
The LZ78 Source
Naomi Sagan, Amir Dembo, Matthew Ho +1
We study a family of processes generated according to sequential probability assignments induced by the LZ78 universal compressor. We characterize entropic and distributional prope…
Universal Discrete Filtering with Lookahead or Delay
Pumiao Yan, Jiwon Jeong, Naomi Sagan +1
We consider the universal discrete filtering problem, where an input sequence generated by an unknown source passes through a discrete memoryless channel, and the goal is to estima…
A Family of LZ78-based Universal Sequential Probability Assignments
Naomi Sagan, Tsachy Weissman
We propose and study a family of universal sequential probability assignments on individual sequences, based on the incremental parsing procedure of the Lempel-Ziv (LZ78) compressi…
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs
Ashwinee Panda, Berivan Isik, Xiangyu Qi +3
Existing methods for adapting large language models (LLMs) to new tasks are not suited to multi-task adaptation because they modify all the model weights -- causing destructive int…