4 citations · 10 across the 6 of their papers we have counts for
6 papers
Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs
Anshumann, Mohd Abbas Zaidi, Akhil Kedia +5
Knowledge distillation can be a cost-effective technique to distill knowledge in Large Language Models, if the teacher output logits can be pre-computed and cached. However, succes…
Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models
Akhil Kedia, Mohd Abbas Zaidi, Sushil Khyalia +3
In spite of their huge success, transformer models remain difficult to scale in depth. In this work, we develop a unified signal propagation theory and provide formulae that govern…
FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for Open-Domain Question Answering
Akhil Kedia, Mohd Abbas Zaidi, Haejun Lee
Generative models have recently started to outperform extractive models in Open Domain Question Answering, largely by leveraging their decoder to attend over multiple encoded passa…
Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems
Mohd Abbas Zaidi, Beomseok Lee, Sangha Kim +1
Simultaneous translation systems start producing the output while processing the partial source sentence in the incoming input stream. These systems need to decide when to read mor…
Infusing Future Information into Monotonic Attention Through Language Models
Mohd Abbas Zaidi, Sathish Indurthi, Beomseok Lee +2
Simultaneous neural machine translation(SNMT) models start emitting the target sequence before they have processed the source sequence. The recent adaptive policies for SNMT use mo…
Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation
Hyojung Han, Sathish Indurthi, Mohd Abbas Zaidi +5
Recently, simultaneous translation has gathered a lot of attention since it enables compelling applications such as subtitle translation for a live event or real-time video-call tr…