Roy Miles

activity

20242026

most cited

V_{k} D :

Improving Knowledge Distillation using Orthogonal Projections

1 citations · 1 across the 8 of their papers we have counts for

collaborators

Showing 2024Show all

2 papers · 1 filter

cs.CV2024

VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections

Roy Miles, Pradyumna Reddy, Ismail Elezi +1

Large language models (LLMs) have recently emerged as powerful tools for tackling many language-processing tasks. Despite their success, training and fine-tuning these models is st…

cs.CV2024★ 1 cited

$V_{k} D :$ Improving Knowledge Distillation using Orthogonal Projections

Roy Miles, Ismail Elezi, Jiankang Deng

Knowledge distillation is an effective method for training small and efficient deep learning models. However, the efficacy of a single method can degenerate when transferring to ot…

VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections

Vk​D: Improving Knowledge Distillation using Orthogonal Projections

$V_{k} D :$ Improving Knowledge Distillation using Orthogonal Projections