7 citations · 29 across the 33 of their papers we have counts for
38 papers
From Plato to Aristotle: Toward a General Theory of Representation Hypotheses
Dai Shi, Andi Han, Xiaoyu Li +2
The Platonic Representation Hypothesis (PRH) holds that increasingly capable models converge toward a common representation. However, recent studies have shown different views on a…
A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and Gauges
Xiaoyu Li, Zhizhou Sha, Chiwun Yang +1
Parameter-efficient fine-tuning has hundreds of methods, usually compared by parameter count and benchmark score. We compare them as maps. A pretrained checkpoint is a base point i…
Self-Confirming Superposition Traps in Reinforcement Learning
Dai Shi, Andi Han, Feng Chen +3
Reinforcement learning (RL) trains representations on data selected by the agent's policy, which then uses the resulting returns to guide its next choices. We show that this loop c…
SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroboration
Linhan Luo, Lequan Lin, Dai Shi +3
Post-training often improves task performance but can degrade confidence calibration, leaving post-trained language models (PoLMs) more overconfident than their corresponding pretr…
Feature Superposition in Neural Networks: From Theory to Practice
Dai Shi, Xiaoyu Li, Andi Han +1
Superposition refers to neural networks representing more features than they have dimensions. It was proposed to explain polysemantic neurons, which respond to several features, an…
When the Canonical Completion Is Wrong: Formalizing and Measuring the Jump in Large Language Models
Dai Shi, Xiaoyu Li, José Miguel Hernández-Lobato
Whether large language models (LLMs) can perform the abductive leap from evidence to a new system of axioms, commonly referred to as a jump, has recently attracted considerable deb…
On Computational Hardness of Mistake-Bounded Language Generation: A Random-Oracle Query Separation
Xiaoyu Li, Andi Han, Dai Shi +2
Generation in the limit guarantees eventual generation for every countable collection of infinite languages in the model of Kleinberg and Mullainathan [KM24], while closure dimensi…
Flood and Harvest: The Provable Necessity of Trivia for Generating Valuable Mathematics via the Lens of Language Generation in the Limit
Xiaoyu Li, Andi Han, Dai Shi +3
AI systems coupled to proof assistants now generate formal mathematics at scale, and the gap between what a checker can verify and what a mathematician would value has become the b…
Learning Manifold and Itô Dynamics with Branched Neural Rough Differential Equations
Luke Thompson, Dai Shi, Lequan Lin +2
Neural rough differential equations (NRDEs) stay accurate under irregular sampling while taking far fewer integration steps than standard neural differential equations, summarising…
SGNN: Efficient Global Mixing and Local Message Passing for Long-Range Graph Learning
Dai Shi, Luke Thompson, Linhan Luo +4
Message-passing neural networks (MPNNs) often suffer from an information bottleneck when capturing long-range dependencies, leading to the oversquashing (OSQ) phenomenon. Alongside…
Wiener Chaos Expansion based Neural Operator for Singular Stochastic Partial Differential Equations
Dai Shi, Luke Thompson, Andi Han +3
In this paper, we explore how our recently developed Wiener Chaos Expansion (WCE)-based neural operator (NO) can be applied to singular stochastic partial differential equations, e…
Expanding the Chaos: Neural Operator for Stochastic (Partial) Differential Equations
Dai Shi, Lequan Lin, Andi Han +4
Stochastic differential equations (SDEs) and stochastic partial differential equations (SPDEs) are fundamental for modeling stochastic dynamics across the natural sciences and mode…
ACT as Human: Multimodal Large Language Model Data Annotation with Critical Thinking
Lequan Lin, Dai Shi, Andi Han +7
Supervised learning relies on high-quality labeled data, but obtaining such data through human annotation is both expensive and time-consuming. Recent work explores using large lan…
ATOM: A Pretrained Neural Operator for Multitask Molecular Dynamics
Luke Thompson, Davy Guan, Dai Shi +3
Molecular dynamics (MD) simulations underpin modern computational drug discovery, materials science, and biochemistry. Recent machine learning models provide high-fidelity MD predi…
SPDEBench: An Extensive Benchmark for Learning Stochastic PDEs
Yuantu Zhu, Zheyan Li, Dai Shi +8
Stochastic Partial Differential Equations (SPDEs) driven by random noise play a central role in modeling physical processes with rough spatio-temporal dynamics, such as turbulence…
Graph Pseudotime Analysis and Neural Stochastic Differential Equations for Analyzing Retinal Degeneration Dynamics and Beyond
Dai Shi, Kuan Yan, Lequan Lin +6
Understanding disease progression at the molecular pathway level usually requires capturing both structural dependencies between pathways and the temporal dynamics of disease evolu…
Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning
Lequan Lin, Dai Shi, Andi Han +2
Graph Neural Networks (GNNs) are proficient in graph representation learning and achieve promising performance on versatile tasks such as node classification and link prediction. U…
When Graph Neural Networks Meet Dynamic Mode Decomposition
Dai Shi, Lequan Lin, Andi Han +3
Graph Neural Networks (GNNs) have emerged as fundamental tools for a wide range of prediction tasks on graph-structured data. Recent studies have drawn analogies between GNN featur…
A Riemannian Approach to Ground Metric Learning for Optimal Transport
Pratik Jawanpuria, Dai Shi, Bamdev Mishra +1
Optimal transport (OT) theory has attracted much attention in machine learning and signal processing applications. OT defines a notion of distance between probability distributions…
Machine Learning-Based Prediction of Key Genes Correlated to the Subretinal Lesion Severity in a Mouse Model of Age-Related Macular Degeneration
Kuan Yan, Yue Zeng, Dai Shi +5
Age-related macular degeneration (AMD) is a major cause of blindness in older adults, severely affecting vision and quality of life. Despite advances in understanding AMD, the mole…
Unleash Graph Neural Networks from Heavy Tuning
Lequan Lin, Dai Shi, Andi Han +2
Graph Neural Networks (GNNs) are deep-learning architectures designed for graph-type data, where understanding relationships among individual observations is crucial. However, achi…
Design Your Own Universe: A Physics-Informed Agnostic Method for Enhancing Graph Neural Networks
Dai Shi, Andi Han, Lequan Lin +3
Physics-informed Graph Neural Networks have achieved remarkable performance in learning through graph-structured data by mitigating common GNN challenges such as over-smoothing, ov…
SpecSTG: A Fast Spectral Diffusion Framework for Probabilistic Spatio-Temporal Traffic Forecasting
Lequan Lin, Dai Shi, Andi Han +1
Traffic forecasting, a crucial application of spatio-temporal graph (STG) learning, has traditionally relied on deterministic models for accurate point estimations. Yet, these mode…
Exposition on over-squashing problem on GNNs: Current Methods, Benchmarks and Challenges
Dai Shi, Andi Han, Lequan Lin +2
Graph-based message-passing neural networks (MPNNs) have achieved remarkable success in both node and graph-level learning tasks. However, several identified problems, including ov…