9 papers
End-to-End Context Compression at Scale
Ang Li, Sean McLeish, Haozhe Chen +12
Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degra…
Learning Reasoning World Models for Parallel Code
Gautam Singh, Arjun Guha, Bhavya Kailkhura +1
Large language models have shown remarkable ability in serial code generation, but they still struggle with parallel code for which training data is comparatively scarce. A common…
The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
Eric Onyame, Runtao Zhou, Kowshik Thopalli +2
Chain-of-thought (CoT) monitoring has been proposed as a promising safety mechanism for detecting misaligned behavior in large language models. However, its reliability remains lar…
LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning
Sumeet Ramesh Motwani, Daniel Nichols, Charles London +17
As language models are increasingly deployed for complex autonomous tasks, their ability to reason accurately over longer horizons becomes critical. An essential component of this…
Improving Robustness In Sparse Autoencoders via Masked Regularization
Vivek Narayanaswamy, Kowshik Thopalli, Bhavya Kailkhura +1
Sparse autoencoders (SAEs) are widely used in mechanistic interpretability to project LLM activations onto sparse latent spaces. However, sparsity alone is an imperfect proxy for i…
ProtAlign: Contrastive learning paradigm for Sequence and structure alignment
Aditya Ranganath, Hasin Us Sami, Kowshik Thopalli +2
Protein language models often take into consideration the alignment between a protein sequence and its textual description. However, they do not take structural information into co…