8 papers
Tree-Structured Orthonormal Decomposition of the Aitchison Simplex
Daisuke Yamada, Qijun Zhang, Travis Pence +3
Compositional data -- vectors encoding relative proportions -- arise across scientific domains, including ecology, geochemistry, and genomics. The features in these data often come…
Recursive Binding on a Budget: Subspace Carving in Order-p Tensor Memories
Travis Pence, Daisuke Yamada, Vikas Singh
Tensor Product Representations provide the structural fidelity required for symbolic reasoning in models but suffer from exponential dimensionality growth when encoding deep recurs…
Concept Attractors in LLMs and their Applications
Sotirios Panagiotis Chytas, Vikas Singh
Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show tha…
CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding
Ziteng Sun, Adrian Benton, Samuel Kushnir +4
Post-training quantization is an effective method for reducing the serving cost of large language models, where the standard approach is to use a round-to-nearest quantization leve…
Composing Linear Layers from Irreducibles
Travis Pence, Daisuke Yamada, Vikas Singh
Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental buildin…
FoGE: Fock Space inspired encoding for graph prompting
Sotirios Panagiotis Chytas, Rudrasis Chakraborty, Vikas Singh
Recent results show that modern Large Language Models (LLM) are indeed capable of understanding and answering questions about structured data such as graphs. This new paradigm can…
ReCo: Reminder Composition Mitigates Hallucinations in Vision-Language Models
Sotirios Panagiotis Chytas, Miso Choi, Hyunwoo J. Kim +1
Vision Language Models (VLMs) show impressive capabilities in integrating and reasoning with both visual and language data. But these models make mistakes. A common finding -- simi…
Pooling Image Datasets With Multiple Covariate Shift and Imbalance
Sotirios Panagiotis Chytas, Vishnu Suresh Lokhande, Peiran Li +1
Small sample sizes are common in many disciplines, which necessitates pooling roughly similar datasets across multiple institutions to study weak but relevant associations between…