16 citations · 21 across the 12 of their papers we have counts for
13 papers
Rethinking LoRA Memory Through the Lens of KV Cache Compression
Chunsheng Zuo, Liaoyaqi Wang, William Jurayj +2
Parametric retrieval augmentation encodes document information into lightweight, document-specific modules such as LoRA adapters, reducing the need to include all evidence as input…
SEQR: Secure and Efficient QR-based LoRA Routing
William Fleshman, Benjamin Van Durme
Low-Rank Adaptation (LoRA) has become a standard technique for parameter-efficient fine-tuning of large language models, enabling large libraries of LoRAs, each for a specific task…
mmBERT: A Modern Multilingual Encoder with Annealed Language Learning
Marc Marone, Orion Weller, William Fleshman +3
Encoder-only languages models are frequently used for a variety of standard machine learning tasks, including classification and retrieval. However, there has been a lack of recent…
LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks
William Fleshman, Benjamin Van Durme
The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Ge…
SpectR: Dynamically Composing LM Experts with Spectral Routing
William Fleshman, Benjamin Van Durme
Training large, general-purpose language models poses significant challenges. The growing availability of specialized expert models, fine-tuned from pretrained models for specific…
RE-AdaptIR: Improving Information Retrieval through Reverse Engineered Adaptation
William Fleshman, Benjamin Van Durme
Large language models (LLMs) fine-tuned for text-retrieval have demonstrated state-of-the-art results across several information retrieval (IR) benchmarks. However, supervised trai…
RE-Adapt: Reverse Engineered Adaptation of Large Language Models
William Fleshman, Benjamin Van Durme
We introduce RE-Adapt, an approach to fine-tuning large language models on new domains without degrading any pre-existing instruction-tuning. We reverse engineer an adapter which i…
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees
William Fleshman, Aleem Khan, Marc Marone +1
Large language models (LLMs) are increasingly capable of completing knowledge intensive tasks by recalling information from a static pretraining corpus. Here we are concerned with…
Toucan: Token-Aware Character Level Language Modeling
William Fleshman, Benjamin Van Durme
Character-level language models obviate the need for separately trained tokenizers, but efficiency suffers from longer sequence lengths. Learning to combine character representatio…
Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection
Edward Raff, William Fleshman, Richard Zak +3
Recent works within machine learning have been tackling inputs of ever-increasing size, with cybersecurity presenting sequence classification problems of particularly extreme lengt…
Deception and the Strategy of Influence
Brian B., William Fleshman, Kevin H. +2
Organizations have long used deception as a means to exert influence in pursuit of their agendas. In particular, information operations such as propaganda distribution, support of…
Static Malware Detection & Subterfuge: Quantifying the Robustness of Machine Learning and Current Anti-Virus
William Fleshman, Edward Raff, Richard Zak +2
As machine-learning (ML) based systems for malware detection become more prevalent, it becomes necessary to quantify the benefits compared to the more traditional anti-virus (AV) s…
Non-Negative Networks Against Adversarial Attacks
William Fleshman, Edward Raff, Jared Sylvester +2
Adversarial attacks against neural networks are a problem of considerable importance, for which effective defenses are not yet readily available. We make progress toward this probl…