7 papers
Tree-Structured Orthonormal Decomposition of the Aitchison Simplex
Daisuke Yamada, Qijun Zhang, Travis Pence +3
Compositional data -- vectors encoding relative proportions -- arise across scientific domains, including ecology, geochemistry, and genomics. The features in these data often come…
Recursive Binding on a Budget: Subspace Carving in Order-p Tensor Memories
Travis Pence, Daisuke Yamada, Vikas Singh
Tensor Product Representations provide the structural fidelity required for symbolic reasoning in models but suffer from exponential dimensionality growth when encoding deep recurs…
Composing Linear Layers from Irreducibles
Travis Pence, Daisuke Yamada, Vikas Singh
Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental buildin…
Concept Attractors in LLMs and their Applications
Sotirios Panagiotis Chytas, Vikas Singh
Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show tha…
CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding
Ziteng Sun, Adrian Benton, Samuel Kushnir +4
Post-training quantization is an effective method for reducing the serving cost of large language models, where the standard approach is to use a round-to-nearest quantization leve…
FoGE: Fock Space inspired encoding for graph prompting
Sotirios Panagiotis Chytas, Rudrasis Chakraborty, Vikas Singh
Recent results show that modern Large Language Models (LLM) are indeed capable of understanding and answering questions about structured data such as graphs. This new paradigm can…