Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Breaking the Bubble: Asynchronous Pipeline Parallel Training with Bounded Weight Inconsistency
Itay Elam, Eliron Rahimi, Avi Mendelson +1
Pipeline parallelism is essential for training large neural networks, but existing schedules trade off throughput, memory, and optimization consistency. Synchronous pipelines prese…
cs.LG2026
Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models
Eliron Rahimi, Elad Hirshel, Rom Himelstein +3
Diffusion language models (DLMs) have recently emerged as a competitive alternative to autoregressive (AR) models, offering parallel decoding, competitive generation quality, and i…
cs.LG2025
Hysteresis Activation Function for Efficient Inference
Moshe Kimhi, Idan Kashani, Avi Mendelson +1
The widely used ReLU is favored for its hardware efficiency, {as the implementation at inference is a one bit sign case,} yet suffers from issues such as the ``dying ReLU'' problem…