6 papers
PersonalAI 2.0: Enhancing knowledge graph traversal/retrieval with planning mechanism for Personalized LLM Agents
Mikhail Menschikov, Matvey Iskornev, Alexander Kharitonov +8
We introduce PersonalAI 2.0 (PAI-2), a novel framework, designed to enhance large language model (LLM) based systems through integration of external knowledge graphs (KG). The prop…
Contextual Linear Activation Steering of Language Models
Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan +1
Linear activation steering is a powerful approach for eliciting the capabilities of large language models and specializing their behavior using limited labeled data. While effectiv…
Convergent Evolution: How Different Language Models Learn Similar Number Representations
Deqing Fu, Tianyi Zhou, Mikhail Belkin +2
Language models trained on natural text learn to represent numbers using periodic features with dominant periods at . In this paper, we identify a two-tiered hierarchy…
Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing
Parsa Mirtaheri, Mikhail Belkin
Large language models (LLMs) can produce chains of thought (CoT) that do not accurately reflect the actual factors driving their answers. In multiple-choice settings with an inject…
Task Generalization With AutoRegressive Compositional Structure: Can Learning From Tasks Generalize to Tasks?
Amirhesam Abedsoltan, Huaqing Zhang, Kaiyue Wen +3
Large language models (LLMs) exhibit remarkable task generalization, solving tasks they were never explicitly trained on with only a few demonstrations. This raises a fundamental q…
UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models
Yijiang River Dong, Hongzhou Lin, Mikhail Belkin +2
Mitigating the retention of sensitive or private information in large language models is essential for enhancing privacy and safety. Existing unlearning methods, like Gradient Asce…