113 citations · 397 across the 33 of their papers we have counts for
16 papers · 1 filter
MLKV: Efficiently Scaling up Large Embedding Model Training with Disk-based Key-Value Storage
Yongjun He, Roger Waleffe, Zhichao Han +8
Many modern machine learning (ML) methods rely on embedding models to learn vector representations (embeddings) for a set of entities (embedding tables). As increasingly diverse ML…
Armada: Memory-Efficient Distributed Training of Large-Scale Graph Neural Networks
Roger Waleffe, Devesh Sarda, Jason Mohoney +3
We study distributed training of Graph Neural Networks (GNNs) on billion-scale graphs that are partitioned across machines. Efficient training in this setting relies on min-edge-cu…
Fundamental Challenges in Evaluating Text2SQL Solutions and Detecting Their Limitations
Cedric Renggli, Ihab F. Ilyas, Theodoros Rekatsinas
In this work, we dive into the fundamental challenges of evaluating Text2SQL solutions and highlight potential failure causes and the potential risks of relying on aggregate metric…
TSDS: Data Selection for Task-Specific Model Finetuning
Zifan Liu, Amin Karbasi, Theodoros Rekatsinas
Finetuning foundation models for specific tasks is an emerging paradigm in modern machine learning. The efficacy of task-specific finetuning largely depends on the selection of app…
Co-design Hardware and Algorithm for Vector Search
Wenqi Jiang, Shigang Li, Yu Zhu +8
Vector search has emerged as the foundation for large-scale information retrieval and machine learning systems, with search engines like Google and Bing processing tens of thousand…
Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning
Patrik Okanovic, Roger Waleffe, Vasilis Mageirakos +5
Methods for carefully selecting or generating a small set of training data to learn from, i.e., data pruning, coreset selection, and data distillation, have been shown to be effect…
MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks
Roger Waleffe, Jason Mohoney, Theodoros Rekatsinas +1
We study training of Graph Neural Networks (GNNs) for large-scale graphs. We revisit the premise of using distributed training for billion-scale graphs and show that for graphs tha…
Marius: Learning Massive Graph Embeddings on a Single Machine
Jason Mohoney, Roger Waleffe, Yiheng Xu +2
We propose a new framework for computing the embeddings of large-scale graphs on a single machine. A graph embedding is a fixed length vector representation for each node (and/or e…
Principal Component Networks: Parameter Reduction Early in Training
Roger Waleffe, Theodoros Rekatsinas
Recent works show that overparameterized networks contain small subnetworks that exhibit comparable accuracy to the full model when trained in isolation. These results highlight th…
Record fusion: A learning approach
Alireza Heidari, George Michalopoulos, Shrinu Kushagra +2
Record fusion is the task of aggregating multiple records that correspond to the same real-world entity in a database. We can view record fusion as a machine learning problem where…
Picket: Guarding Against Corrupted Data in Tabular Data during Learning and Inference
Zifan Liu, Zhechun Zhou, Theodoros Rekatsinas
Data corruption is an impediment to modern machine learning deployments. Corrupted data can severely bias the learned model and can also lead to invalid inferences. We present, Pic…
On Robust Mean Estimation under Coordinate-level Corruption
Zifan Liu, Jongho Park, Theodoros Rekatsinas +1
We study the problem of robust mean estimation and introduce a novel Hamming distance-based measure of distribution shift for coordinate-level corruptions. We show that this measur…
Approximate Inference in Structured Instances with Noisy Categorical Observations
Alireza Heidari, Ihab F. Ilyas, Theodoros Rekatsinas
We study the problem of recovering the latent ground truth labeling of a structured instance with categorical random variables in the presence of noisy observations. We present a n…
Data-Dependent Differentially Private Parameter Learning for Directed Graphical Models
Amrita Roy Chowdhury, Theodoros Rekatsinas, Somesh Jha
Directed graphical models (DGMs) are a class of probabilistic models that are widely used for predictive analysis in sensitive domains, such as medical diagnostics. In this paper w…
MLSys: The New Frontier of Machine Learning Systems
Alexander Ratner, Dan Alistarh, Gustavo Alonso +66
Machine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and implementing the systems that support ML models in real-world deployments remains…
Multi-relational Learning Using Weighted Tensor Decomposition with Modular Loss
Ben London, Theodoros Rekatsinas, Bert Huang +1
We propose a modular framework for multi-relational learning via tensor decomposition. In our learning setting, the training data contains multiple types of relationships among a s…