1 citations · 1 across the 14 of their papers we have counts for
8 papers · 1 filter
Adaptive Head Budgeting for Efficient Multi-Head Attention
Bilal Faye, Abdoulaye Mbaye, Hanane Azzag +1
Multi-head attention enables Transformers to capture diverse representations, but all attention heads are typically activated for every input, regardless of task complexity. For co…
Prototype-Guided Diffusion: Visual Conditioning without External Memory
Bilal Faye, Hanane Azzag, Mustapha Lebbah
Diffusion models achieve state-of-the-art image generation but remain computationally costly due to iterative denoising. Latent-space models like Stable Diffusion reduce overhead y…
Value-Free Policy Optimization via Reward Partitioning
Bilal Faye, Hanane Azzag, Mustapha Lebbah
Single-trajectory preference optimization methods learn from datasets of ((prompt, response, reward)) tuples, offering a practical alternative to pairwise preference learning by di…
Game Theory Meets Statistical Mechanics in Deep Learning Design
Djamel Bouchaffra, Fayçal Ykhlef, Bilal Faye +2
We present a novel deep graphical representation that seamlessly merges principles of game theory with laws of statistical mechanics. It performs feature extraction, dimensionality…
Unsupervised Adaptive Normalization
Bilal Faye, Hanane Azzag, Mustapha Lebbah +1
Deep neural networks have become a staple in solving intricate problems, proving their mettle in a wide array of applications. However, their training process is often hampered by…
Supervised Batch Normalization
Bilal Faye, Mustapha Lebbah, Hanane Azzag
Batch Normalization (BN), a widely-used technique in neural networks, enhances generalization and expedites training by normalizing each mini-batch to the same mean and variance. H…
Enhancing Neural Network Representations with Prior Knowledge-Based Normalization
Bilal Faye, Hanane Azzag, Mustapha Lebbah +1
Deep learning models face persistent challenges in training, particularly due to internal covariate shift and label shift. While single-mode normalization methods like Batch Normal…
Lightweight Cross-Modal Representation Learning
Bilal Faye, Hanane Azzag, Mustapha Lebbah +1
Low-cost cross-modal representation learning is crucial for deriving semantic representations across diverse modalities such as text, audio, images, and video. Traditional approach…