2.5k citations · 4.7k across the 60 of their papers we have counts for
4 papers · 1 filter
Physics of Language Models: Part 3.2, Knowledge Manipulation
Zeyuan Allen-Zhu, Yuanzhi Li
Language models can store vast factual knowledge, yet their ability to flexibly use this knowledge for downstream tasks (e.g., via instruction finetuning) remains questionable. Thi…
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Zeyuan Allen-Zhu, Yuanzhi Li
Large language models (LLMs) can store a vast amount of world knowledge, often extractable via question-answering (e.g., "What is Abraham Lincoln's birthday?"). However, do they an…
SALSA VERDE: a machine learning attack on Learning With Errors with sparse small secrets
Cathy Yuanchen Li, Emily Wenger, Zeyuan Allen-Zhu +2
Learning with Errors (LWE) is a hard math problem used in post-quantum cryptography. Homomorphic Encryption (HE) schemes rely on the hardness of the LWE problem for their security,…
Physics of Language Models: Part 1, Learning Hierarchical Language Structures
Zeyuan Allen-Zhu, Yuanzhi Li
Transformer-based language models are effective but complex, and understanding their inner workings and reasoning mechanisms is a significant challenge. Previous research has prima…