10 citations · 19 across the 15 of their papers we have counts for
17 papers
Gemma 4 Technical Report
Gemma Team, Sherif El Abd, Vaibhav Aggarwal +320
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemm…
Generalised Probabilistic Modelling and Improved Uncertainty Estimation in Comparative LLM-as-a-judge
Yassir Fathullah, Mark J. F. Gales
This paper explores generalised probabilistic modelling and uncertainty estimation in comparative LLM-as-a-judge frameworks. We show that existing Product-of-Experts methods are sp…
Cross-Lingual Transfer Learning for Speech Translation
Rao Ma, Mengjie Qian, Yassir Fathullah +3
There has been increasing interest in building multilingual foundation models for NLP and speech research. This paper examines how to expand the speech translation capability of th…
Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisons
Adian Liusie, Vatsal Raina, Yassir Fathullah +1
LLM-as-a-judge approaches are a practical and effective way of assessing a range of text tasks. However, when using pairwise comparisons to rank a set of candidates, the computatio…
Efficient Sample-Specific Encoder Perturbations
Yassir Fathullah, Mark J. F. Gales
Encoder-decoder foundation models have displayed state-of-the-art performance on a range of autoregressive sequence tasks. This paper proposes a simple and lightweight modification…
Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models
Adian Liusie, Yassir Fathullah, Mark J. F. Gales
Large Language Models (LLMs) have demonstrated impressive zero-shot capabilities and versatility in NLP tasks, however they sometimes fail to maintain crucial invariances for speci…
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs
Yassir Fathullah, Chunyang Wu, Egor Lakomkin +7
In this work, we extend the instruction-tuned Llama-2 model with end-to-end general-purpose speech processing and reasoning abilities while maintaining the wide range of original L…
End-to-End Speech Recognition Contextualization with Large Language Models
Egor Lakomkin, Chunyang Wu, Yassir Fathullah +3
In recent years, Large Language Models (LLMs) have garnered significant attention from the research community due to their exceptional performance and generalization capabilities.…
TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models
Yuan Shangguan, Haichuan Yang, Danni Li +11
Automatic Speech Recognition (ASR) models need to be optimized for specific hardware before they can be deployed on devices. This can be done by tuning the model's hyperparameters…
Prompting Large Language Models with Speech Recognition Abilities
Yassir Fathullah, Chunyang Wu, Egor Lakomkin +9
Large language models have proven themselves highly flexible, able to solve a wide range of generative tasks, such as abstractive summarization and open-ended question answering. I…
CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models
Potsawee Manakul, Yassir Fathullah, Adian Liusie +3
In this paper, we consider the challenge of summarizing patients' medical progress notes in a limited data setting. For the Problem List Summarization (shared task 1A) at the BioNL…
Multi-Head State Space Model for Speech Recognition
Yassir Fathullah, Chunyang Wu, Yuan Shangguan +8
State space models (SSMs) have recently shown promising results on small-scale sequence and language modelling tasks, rivalling and outperforming many attention-based approaches. I…
Logit-Based Ensemble Distribution Distillation for Robust Autoregressive Sequence Uncertainties
Yassir Fathullah, Guoxuan Xia, Mark Gales
Efficiently and reliably estimating uncertainty is an important objective in deep learning. It is especially pertinent to autoregressive sequence tasks, where training and inferenc…
Who Needs Decoders? Efficient Estimation of Sequence-level Attributes
Yassir Fathullah, Puria Radmard, Adian Liusie +1
State-of-the-art sequence-to-sequence models often require autoregressive decoding, which can be highly expensive. However, for some downstream tasks such as out-of-distribution (O…
Self-Distribution Distillation: Efficient Uncertainty Estimation
Yassir Fathullah, Mark J. F. Gales
Deep learning is increasingly being applied in safety-critical domains. For these scenarios it is important to know the level of uncertainty in a model's prediction to ensure appro…
Ensemble Distillation Approaches for Grammatical Error Correction
Yassir Fathullah, Mark Gales, Andrey Malinin
Ensemble approaches are commonly used techniques to improving a system by combining multiple model predictions. Additionally these schemes allow the uncertainty, as well as the sou…
Improved Large-margin Softmax Loss for Speaker Diarisation
Yassir Fathullah, Chao Zhang, Philip C. Woodland
Speaker diarisation systems nowadays use embeddings generated from speech segments in a bottleneck layer, which are needed to be discriminative for unseen speakers. It is well-know…