34 citations · 42 across the 10 of their papers we have counts for
10 papers
Improving Recursive Transformers with Mixture of LoRAs
Mohammadmahdi Nouriborji, Morteza Rohanian, Omid Rohanian
Parameter sharing in recursive transformers reduces model size but collapses layer-wise expressivity. We propose Mixture of LoRAs (MoL), a lightweight conditional-computation mecha…
Efficient Interleaved Speech Modeling through Knowledge Distillation
Mohammadmahdi Nouriborji, Morteza Rohanian
Current speech language models exceed the size and latency constraints of many deployment environments. We build compact, expressive speech generation models through layer-aligned…
Rapid Biomedical Research Classification: The Pandemic PACT Advanced Categorisation Engine
Omid Rohanian, Mohammadmahdi Nouriborji, Olena Seminog +11
This paper introduces the Pandemic PACT Advanced Categorisation Engine (PPACE) along with its associated dataset. PPACE is a fine-tuned model developed to automatically classify re…
Efficiency at Scale: Investigating the Performance of Diminutive Language Models in Clinical Tasks
Niall Taylor, Upamanyu Ghose, Omid Rohanian +4
The entry of large language models (LLMs) into research and commercial spaces has led to a trend of ever-larger models, with initial promises of generalisability, followed by a wid…
Exploring the Effectiveness of Instruction Tuning in Biomedical Language Processing
Omid Rohanian, Mohammadmahdi Nouriborji, David A. Clifton
Large Language Models (LLMs), particularly those similar to ChatGPT, have significantly influenced the field of Natural Language Processing (NLP). While these models excel in gener…
Lightweight Transformers for Clinical Natural Language Processing
Omid Rohanian, Mohammadmahdi Nouriborji, Hannah Jauncey +5
Specialised pre-trained language models are becoming more frequent in NLP since they can potentially outperform models trained on generic texts. BioBERT and BioClinicalBERT are two…
Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints
Omid Rohanian, Hannah Jauncey, Mohammadmahdi Nouriborji +6
Processing information locked within clinical health records is a challenging task that remains an active area of research in biomedical NLP. In this work, we evaluate a broad set…
MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers
Mohammadmahdi Nouriborji, Omid Rohanian, Samaneh Kouchaki +1
Pre-trained Language Models (LMs) have become an integral part of Natural Language Processing (NLP) in recent years, due to their superior performance in downstream applications. I…
On the Effectiveness of Compact Biomedical Transformers
Omid Rohanian, Mohammadmahdi Nouriborji, Samaneh Kouchaki +1
Language models pre-trained on biomedical corpora, such as BioBERT, have recently shown promising results on downstream biomedical tasks. Many existing pre-trained models, on the o…
Nowruz at SemEval-2022 Task 7: Tackling Cloze Tests with Transformers and Ordinal Regression
Mohammadmahdi Nouriborji, Omid Rohanian, David Clifton
This paper outlines the system using which team Nowruz participated in SemEval 2022 Task 7 Identifying Plausible Clarifications of Implicit and Underspecified Phrases for both subt…