4 papers
Improving Recursive Transformers with Mixture of LoRAs
Mohammadmahdi Nouriborji, Morteza Rohanian, Omid Rohanian
Parameter sharing in recursive transformers reduces model size but collapses layer-wise expressivity. We propose Mixture of LoRAs (MoL), a lightweight conditional-computation mecha…
Efficient Interleaved Speech Modeling through Knowledge Distillation
Mohammadmahdi Nouriborji, Morteza Rohanian
Current speech language models exceed the size and latency constraints of many deployment environments. We build compact, expressive speech generation models through layer-aligned…
Rapid Biomedical Research Classification: The Pandemic PACT Advanced Categorisation Engine
Omid Rohanian, Mohammadmahdi Nouriborji, Olena Seminog +11
This paper introduces the Pandemic PACT Advanced Categorisation Engine (PPACE) along with its associated dataset. PPACE is a fine-tuned model developed to automatically classify re…
Efficiency at Scale: Investigating the Performance of Diminutive Language Models in Clinical Tasks
Niall Taylor, Upamanyu Ghose, Omid Rohanian +4
The entry of large language models (LLMs) into research and commercial spaces has led to a trend of ever-larger models, with initial promises of generalisability, followed by a wid…