46 citations · 75 across the 10 of their papers we have counts for
10 papers
Scaling Laws for Behavioral Foundation Models over User Event Sequences
Rickard Brüel Gabrielsson
Foundation models are increasingly trained on sequences of user actions in recommendation, payments, fraud, and commerce, but these models still lack the kind of compute calibratio…
Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead
Rickard Brüel-Gabrielsson, Jiacheng Zhu, Onkar Bhardwaj +4
Fine-tuning large language models (LLMs) with low-rank adaptations (LoRAs) has become common practice, often yielding numerous copies of the same LLM differing only in their LoRA u…
Slicing Mutual Information Generalization Bounds for Neural Networks
Kimia Nadjahi, Kristjan Greenewald, Rickard Brüel Gabrielsson +1
The ability of machine learning (ML) algorithms to generalize well to unseen data has been studied through the lens of information theory, by bounding the generalization error with…
Asymmetry in Low-Rank Adapters of Foundation Models
Jiacheng Zhu, Kristjan Greenewald, Kimia Nadjahi +6
Parameter-efficient fine-tuning optimizes large, pre-trained foundation models by updating a subset of parameters; in this class, Low-Rank Adaptation (LoRA) is particularly effecti…
Deep Augmentation: Dropout as Augmentation for Self-Supervised Learning
Rickard Brüel-Gabrielsson, Tongzhou Wang, Manel Baradad +1
Despite dropout's ubiquity in machine learning, its effectiveness as a form of data augmentation remains under-explored. We address two key questions: (i) When is dropout effective…
Relative Position Prediction as Pre-training for Text Encoders
Rickard Brüel-Gabrielsson, Chris Scarvelis
Meaning is defined by the company it keeps. However, company is two-fold: It's based on the identity of tokens and also on their position (topology). We argue that a position-centr…
Rewiring with Positional Encodings for Graph Neural Networks
Rickard Brüel-Gabrielsson, Mikhail Yurochkin, Justin Solomon
Several recent works use positional encodings to extend the receptive fields of graph neural network (GNN) layers equipped with attention mechanisms. These techniques, however, ext…
Universal Function Approximation on Graphs
Rickard Brüel-Gabrielsson
In this work we produce a framework for constructing universal function approximators on graph isomorphism classes. We prove how this framework comes with a collection of theoretic…
A Topology Layer for Machine Learning
Rickard Brüel-Gabrielsson, Bradley J. Nelson, Anjan Dwaraknath +3
Topology applied to real world data using persistent homology has started to find applications within machine learning, including deep learning. We present a differentiable topolog…
Topological Approaches to Deep Learning
Gunnar Carlsson, Rickard Brüel Gabrielsson
We perform topological data analysis on the internal states of convolutional deep neural networks to develop an understanding of the computations that they perform. We apply this u…