6 papers
On the Emergence and Test-Time Use of Structural Information in Large Language Models
Michelle Chao Chen, Moritz Miller, Bernhard Schölkopf +1
Learning structural information from observational data is central to producing new knowledge outside the training corpus. This holds for mechanistic understanding in scientific di…
Hausdorff measure of cartesian product of Cantor sets
Siyuan Guo, Taylor Jones
Hausdorff measure and Hausdorff dimension are useful tools to describe fractals. This paper investigates the bounds on the -dimensional Hausdorff measure of the -fold…
Physics of Learning: A Lagrangian perspective to different learning paradigms
Siyuan Guo, Bernhard Schölkopf
We study the problem of building an efficient learning system. Efficient learning processes information in the least time, i.e., building a system that reaches a desired error thre…
Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning
Patrik Reizinger, Bálint Mucsányi, Siyuan Guo +3
Self-supervised feature learning and pretraining methods in reinforcement learning (RL) often rely on information-theoretic principles, termed mutual information skill learning (MI…
Do-PFN: In-Context Learning for Causal Effect Estimation
Jake Robertson, Arik Reuter, Siyuan Guo +3
Estimation of causal effects is critical to a range of scientific disciplines. Existing methods for this task either require interventional data, knowledge about the ground truth c…
Counterfactual reasoning: an analysis of in-context emergence
Moritz Miller, Bernhard Schölkopf, Siyuan Guo
Large-scale neural language models exhibit remarkable performance in in-context learning: the ability to learn and reason about the input context on the fly. This work studies in-c…