7 papers
On the Emergence and Test-Time Use of Structural Information in Large Language Models
Michelle Chao Chen, Moritz Miller, Bernhard Schölkopf +1
Learning structural information from observational data is central to producing new knowledge outside the training corpus. This holds for mechanistic understanding in scientific di…
Do-PFN: In-Context Learning for Causal Effect Estimation
Jake Robertson, Arik Reuter, Siyuan Guo +3
Estimation of causal effects is critical to a range of scientific disciplines. Existing methods for this task either require interventional data, knowledge about the ground truth c…
Counterfactual reasoning: an analysis of in-context emergence
Moritz Miller, Bernhard Schölkopf, Siyuan Guo
Large-scale neural language models exhibit remarkable performance in in-context learning: the ability to learn and reason about the input context on the fly. This work studies in-c…
Hausdorff measure of cartesian product of Cantor sets
Siyuan Guo, Taylor Jones
Hausdorff measure and Hausdorff dimension are useful tools to describe fractals. This paper investigates the bounds on the -dimensional Hausdorff measure of the -fold…
Physics of Learning: A Lagrangian perspective to different learning paradigms
Siyuan Guo, Bernhard Schölkopf
We study the problem of building an efficient learning system. Efficient learning processes information in the least time, i.e., building a system that reaches a desired error thre…
Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning
Patrik Reizinger, Bálint Mucsányi, Siyuan Guo +3
Self-supervised feature learning and pretraining methods in reinforcement learning (RL) often rely on information-theoretic principles, termed mutual information skill learning (MI…