7 papers
Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models
Liyi Zhang, Akshay K. Jagadish, Brenden M. Lake +1
Post-training Large Language Models (LLMs) for reasoning typically focuses on deductive tasks such as mathematics and coding where correctness is verifiable. Yet, many real-world r…
Meta-Learning at Scale for Large Language Models via Low-Rank Amortized Bayesian Meta-Learning
Liyi Zhang, Jake Snell, Thomas L. Griffiths
Fine-tuning large language models (LLMs) with low-rank adaptation (LoRA) is a cost-effective way to incorporate information from a specific dataset. However, when a problem require…
Levels of Analysis for Large Language Models
Alexander Y. Ku, Declan Campbell, Xuechunzi Bai +10
Modern artificial intelligence systems, such as large language models, are increasingly powerful but also increasingly hard to understand. Recognizing this problem as analogous to…
What Should Embeddings Embed? Autoregressive Models Represent Latent Generating Distributions
Liyi Zhang, Michael Y. Li, R. Thomas McCoy +3
Autoregressive language models have demonstrated a remarkable ability to extract latent structure from text. The embeddings from large language models have been shown to capture as…
Identifying and Mitigating the Influence of the Prior Distribution in Large Language Models
Liyi Zhang, Veniamin Veselovsky, R. Thomas McCoy +1
Large language models (LLMs) sometimes fail to respond appropriately to deterministic tasks -- such as counting or forming acronyms -- because the implicit prior distribution they…
Figame: A Family Digital Game Based on JME for Shaping Parent-Child Healthy Gaming Relationship
Liyi Zhang, Yujie Peng, Yi Lian +1
With the development of technology, digital games have permeated into family and parent-child relationships, leading to cognitive deficiencies and inter-generational conflicts that…