9 papers · 1 filter
Adaptive Head Budgeting for Efficient Multi-Head Attention
Bilal Faye, Abdoulaye Mbaye, Hanane Azzag +1
Multi-head attention enables Transformers to capture diverse representations, but all attention heads are typically activated for every input, regardless of task complexity. For co…
Value-Free Policy Optimization via Reward Partitioning
Bilal Faye, Hanane Azzag, Mustapha Lebbah
Single-trajectory preference optimization methods learn from datasets of ((prompt, response, reward)) tuples, offering a practical alternative to pairwise preference learning by di…
Prototype-Guided Diffusion: Visual Conditioning without External Memory
Bilal Faye, Hanane Azzag, Mustapha Lebbah
Diffusion models achieve state-of-the-art image generation but remain computationally costly due to iterative denoising. Latent-space models like Stable Diffusion reduce overhead y…
Enhancing Neural Network Representations with Prior Knowledge-Based Normalization
Bilal Faye, Hanane Azzag, Mustapha Lebbah +1
Deep learning models face persistent challenges in training, particularly due to internal covariate shift and label shift. While single-mode normalization methods like Batch Normal…
Game Theory Meets Statistical Mechanics in Deep Learning Design
Djamel Bouchaffra, Fayçal Ykhlef, Bilal Faye +2
We present a novel deep graphical representation that seamlessly merges principles of game theory with laws of statistical mechanics. It performs feature extraction, dimensionality…
Evaluating the Efficacy of Instance Incremental vs. Batch Learning in Delayed Label Environments: An Empirical Study on Tabular Data Streaming for Fraud Detection
Kodjo Mawuena Amekoe, Mustapha Lebbah, Gregoire Jaffre +2
Real-world tabular learning production scenarios typically involve evolving data streams, where data arrives continuously and its distribution may change over time. In such a setti…