4 citations · 14 across the 21 of their papers we have counts for
22 papers
Metro-WM: Long-Horizon Latent Planning with Realisable Sub-Goals
Royson Lee, Fady Rezk, Titouan Parcollet +2
Model-predictive control with Joint-Embedding Predictive Architectures (JEPAs) provides a strong zero-shot goal-reaching planner, but it is only effective over short planning horiz…
Learning to Reason About Implicit Multimodal Harm: From Diagnosis to Generalization
Anisha Saha, Varsha Suresh, Teodora Kamova +3
A benign image paired with a benign caption can become harmful only in combination, making detection fundamentally a cross-modal reasoning problem. Existing multimodal harm benchma…
From Misclassifications to Outliers: Joint Reliability Assessment in Classification
Yang Li, Youyang Sha, Yinzhi Wang +4
Building reliable classifiers is a fundamental challenge for deploying machine learning in real-world applications. A reliable system should not only detect out-of-distribution (OO…
Dynamic Expert Sharing: Decoupling Memory from Parallelism in Mixture-of-Experts Diffusion LLMs
Hao Mark Chen, Zhiwen Mo, Royson Lee +6
Among parallel decoding paradigms, diffusion large language models (dLLMs) have emerged as a promising candidate that balances generation quality and throughput. However, their int…
The Reward Model Selection Crisis in Personalized Alignment
Fady Rezk, Yuangang Pan, Chuan-Sheng Foo +4
Personalized alignment from preference data has focused primarily on improving personal reward model (RM) accuracy, with the implicit assumption that better preference ranking tran…
Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation
Taehoon Kim, Henry Gouk, Timothy Hospedales
Test-time alignment (TTA) aims to adapt models to specific rewards during inference. However, existing methods tend to either under-optimise or over-optimise (reward hack) the targ…
Model Merging is Secretly Certifiable: Non-Vacuous Generalisation Bounds for Low-Shot Learning
Taehoon Kim, Henry Gouk, Minyoung Kim +1
Certifying the IID generalisation ability of deep networks is the first of many requirements for trusting AI in high-stakes applications from medicine to security. However, when in…
CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographs
Raman Dutt, Pedro Sanchez, Yongchen Yao +3
Structured benchmarks have advanced text-conditional image generation for real-world imagery, however, no such benchmark exists for synthetic radiograph generation. Despite being a…
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization
Hao Mark Chen, Shell Xu Hu, Wayne Luk +2
Model merging has emerged as a promising approach for multi-task learning (MTL), offering a data-efficient alternative to conventional fine-tuning. However, with the rapid developm…
Weight Space Learning for Certifiable Few-shot Transfer Learning
Fady Rezk, Royson Lee, Henry Gouk +2
In contemporary deep learning, a prevalent and effective workflow for solving low-data problems is adapting powerful pre-trained foundation models (FMs) to new tasks via parameter-…
FedPEFT: Federated Learning to Personalize PEFT for Multilingual LLMs
Royson Lee, Minyoung Kim, Fady Rezk +3
Federated learning (FL) has enabled the training of multilingual large language models (LLMs) on diverse and decentralized multilingual data, especially on low-resource languages.…
A Bayesian Approach to Data Point Selection
Xinnuo Xu, Minyoung Kim, Royson Lee +2
Data point selection (DPS) is becoming a critical topic in deep learning due to the ease of acquiring uncurated training data compared to the difficulty of obtaining curated or pro…
Capacity Control is an Effective Memorization Mitigation Mechanism in Text-Conditional Diffusion Models
Raman Dutt, Pedro Sanchez, Ondrej Bohdal +2
In this work, we present compelling evidence that controlling model capacity during fine-tuning can effectively mitigate memorization in diffusion models. Specifically, we demonstr…
MobileQuant: Mobile-friendly Quantization for On-device Language Models
Fuwen Tan, Royson Lee, Łukasz Dudziak +5
Large language models (LLMs) have revolutionized language processing, delivering outstanding results across multiple applications. However, deploying LLMs on edge devices poses sev…
Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning
Bingchen Zhao, Yongshuo Zong, Letian Zhang +1
The advancement of large language models (LLMs) has significantly broadened the scope of applications in natural language processing, with multi-modal LLMs extending these capabili…
Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted
Ruchika Chavhan, Ondrej Bohdal, Yongshuo Zong +2
Large-scale text-to-image diffusion models excel in generating high-quality images from textual inputs, yet concerns arise as research indicates their tendency to memorize and repl…
ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning
Ruchika Chavhan, Da Li, Timothy Hospedales
While large-scale text-to-image diffusion models have demonstrated impressive image-generation capabilities, there are significant concerns about their potential misuse for generat…
Recurrent Early Exits for Federated Learning with Heterogeneous Clients
Royson Lee, Javier Fernandez-Marques, Shell Xu Hu +6
Federated learning (FL) has enabled distributed learning of a model across multiple clients in a privacy-preserving manner. One of the main challenges of FL is to accommodate clien…
MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selection
Raman Dutt, Ondrej Bohdal, Pedro Sanchez +2
Diffusion models excel in generating images that closely resemble their training data but are also susceptible to data memorization, raising privacy, ethical, and legal concerns, p…
VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning
Yongshuo Zong, Ondrej Bohdal, Timothy Hospedales
Large language models (LLMs) famously exhibit emergent in-context learning (ICL) -- the ability to rapidly adapt to new tasks using few-shot examples provided as a prompt, without…
Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models
Yongshuo Zong, Ondrej Bohdal, Tingyang Yu +2
Current vision large language models (VLLMs) exhibit remarkable capabilities yet are prone to generate harmful content and are vulnerable to even the simplest jailbreaking attacks.…
Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations
Yongshuo Zong, Tingyang Yu, Ruchika Chavhan +2
Large language and vision-language models are rapidly being deployed in practice thanks to their impressive capabilities in instruction following, in-context learning, and so on. T…