19 citations · 69 across the 25 of their papers we have counts for
14 papers · 1 filter
From Plato to Aristotle: Toward a General Theory of Representation Hypotheses
Dai Shi, Andi Han, Xiaoyu Li +2
The Platonic Representation Hypothesis (PRH) holds that increasingly capable models converge toward a common representation. However, recent studies have shown different views on a…
Self-Confirming Superposition Traps in Reinforcement Learning
Dai Shi, Andi Han, Feng Chen +3
Reinforcement learning (RL) trains representations on data selected by the agent's policy, which then uses the resulting returns to guide its next choices. We show that this loop c…
SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroboration
Linhan Luo, Lequan Lin, Dai Shi +3
Post-training often improves task performance but can degrade confidence calibration, leaving post-trained language models (PoLMs) more overconfident than their corresponding pretr…
Mitigating Forgetting in Low Rank Adaptation
Joanna Sliwa, Frank Schneider, Philipp Hennig +1
Parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), enable fast specialization of large pre-trained models to different downstream applications. However, t…
Training-Free Vector Quantization via Gaussian VAEs
Tongda Xu, Wendi Zheng, Jiajun He +4
Vector-quantized variational autoencoders (VQ-VAEs) are discrete autoencoders that compress images into discrete tokens. However, they are difficult to train due to discretization.…
Weighted Conditional Flow Matching
Sergio Calvo-Ordonez, Matthieu Meunier, Alvaro Cartea +3
Conditional flow matching (CFM) has emerged as a powerful framework for training continuous normalizing flows due to its computational efficiency and effectiveness. However, standa…
A Gaussian Process View on Observation Noise and Initialization in Wide Neural Networks
Sergio Calvo-Ordoñez, Jonathan Plenk, Richard Bergna +4
Performing gradient descent in a wide neural network is equivalent to computing the posterior mean of a Gaussian Process with the Neural Tangent Kernel (NTK-GP), for a specific pri…
Uncertainty Modeling in Graph Neural Networks via Stochastic Differential Equations
Richard Bergna, Sergio Calvo-Ordoñez, Felix L. Opolka +2
We propose a novel Stochastic Differential Equation (SDE) framework to address the problem of learning uncertainty-aware representations for graph-structured data. While Graph Neur…
Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanation
Xuexin Chen, Ruichu Cai, Zhengting Huang +5
We investigate the problem of explainability for machine learning models, focusing on Feature Attribution Methods (FAMs) that evaluate feature importance through perturbation tests…
Graph Neural Stochastic Differential Equations
Richard Bergna, Felix Opolka, Pietro Liò +1
We present a novel model Graph Neural Stochastic Differential Equations (Graph Neural SDEs). This technique enhances the Graph Neural Ordinary Differential Equations (Graph Neural…
Addressing Bias in Active Learning with Depth Uncertainty Networks... or Not
Chelsea Murray, James U. Allingham, Javier Antorán +1
Farquhar et al. [2021] show that correcting for active learning bias with underparameterised models leads to improved downstream performance. For overparameterised models such as N…
Depth Uncertainty Networks for Active Learning
Chelsea Murray, James U. Allingham, Javier Antorán +1
In active learning, the size and complexity of the training dataset changes over time. Simple models that are well specified by the amount of data available at the start of active…
Bootstrap Your Flow
Laurence Illing Midgley, Vincent Stimper, Gregor N. C. Simm +1
Normalizing flows are flexible, parameterized distributions that can be used to approximate expectations from intractable distributions via importance sampling. However, current fl…
BSODA: A Bipartite Scalable Framework for Online Disease Diagnosis
Weijie He, Xiaohao Mao, Chao Ma +3
A growing number of people are seeking healthcare advice online. Usually, they diagnose their medical conditions based on the symptoms they are experiencing, which is also known as…