5 papers
S-JEPA : Soft Clustering Anchors for Self-Supervised Speech Representation Learning
Georgios Ioannides, Adrian Kieback, Judah Goldfeder +5
Self-supervised speech encoders are predominantly trained by predicting discrete hard cluster IDs at masked positions, a recipe that collapses acoustic ambiguity at category bounda…
Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures
Georgios Ioannides, Adrian Kieback, Judah Goldfeder +5
Joint Embedding Predictive Architectures (JEPA) offer a promising approach to self-supervised speech representation learning, but suffer from representation collapse without explic…
JEPA as a Neural Tokenizer: Learning Robust Speech Representations with Density Adaptive Attention
Georgios Ioannides, Christos Constantinou, Aman Chadha +4
We introduce a two-stage self-supervised framework that combines the Joint-Embedding Predictive Architecture (JEPA) with a Density Adaptive Attention Mechanism (DAAM) for learning…
MOD-X: A Modular Open Decentralized eXchange Framework proposal for Heterogeneous Interoperable Artificial Intelligence Agents
Georgios Ioannides, Christos Constantinou, Vinija Jain +2
As Artificial Intelligence systems evolve from monolithic models to ecosystems of specialized agents, the need for standardized communication protocols becomes increasingly critica…
Density Adaptive Attention is All You Need: Robust Parameter-Efficient Fine-Tuning Across Multiple Modalities
Georgios Ioannides, Aman Chadha, Aaron Elkins
We propose the Multi-Head Density Adaptive Attention Mechanism (DAAM), a novel probabilistic attention framework that can be used for Parameter-Efficient Fine-tuning (PEFT), and th…