9 papers
Your Probabilistic JEPA Is Secretly a Hidden Markov Model: A State-Space Interpretation of Joint-Embedding Predictive Learning
Yongchao Huang
A hidden Markov model (HMM) combines three roles: inference of a hidden-state belief from observations, propagation through a Markov transition, and emission back to observation sp…
SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors
Yongchao Huang
Joint-embedding predictive architectures learn abstract states by predicting target embeddings from context embeddings, but their transition models are typically opaque neural maps…
Gaussian Mixture Attention: Linear-Time Sequence Mixing via Probabilistic Latent Routing
Yongchao Huang, Hassan Raza
The dense token-to-token interaction pattern of standard dot-product attention remains a central bottleneck in scaling Transformer architectures to long contexts. We introduce \tex…
Knowledge, Rules and Their Embeddings: Two Paths towards Neuro-Symbolic JEPA
Yongchao Huang, Hassan Raza
Modern self-supervised predictive architectures excel at capturing complex statistical correlations from high-dimensional data but lack mechanisms to internalize verifiable human l…
VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models
Yongchao Huang
Joint Embedding Predictive Architectures (JEPA) offer a scalable paradigm for self-supervised learning by predicting latent representations rather than reconstructing high-entropy…
Sampling via Gaussian Mixture Approximations
Yongchao Huang
We present a family of \textit{Gaussian Mixture Approximation} (GMA) samplers for sampling unnormalised target densities, encompassing \textit{weights-only GMA} (W-GMA), \textit{La…