285 citations · 338 across the 49 of their papers we have counts for
33 papers · 1 filter
Memorilla: Latent Semantic Memory for LLMs
Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli +4
Memory enables large language models (LLMs) to use information beyond their parameters, such as past user conversations, long documents, or an agent's earlier actions. For a frozen…
RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
Rishabh Ranjan, Vignesh Kothapalli, Harshvardhan Agarwal +5
Foundation models for tables and relational databases are increasingly used for predictive tasks across various scientific and industrial domains. However, adapting existing models…
Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistry
Yiming Huang, Yujie Zeng, Vijay Prakash Dwivedi +4
Molecular learning models are strongly shaped by their underlying representations. Yet standard sequential and graph formalisms struggle to explicitly encode higher-order topology,…
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Atindra Jha, Margaret Li, Jure Leskovec +2
As the supply of human-written text is exhausted, it has become standard practice to repeat language model training data. Prior work has studied data repetition for densely activat…
Advancing Open and Reproducible Relational Learning: RelArena-, TabPFN-Rel and RPI
Adrian Hayler, Klemens Flöge, Alan Arazi +44
This first release of Prior Labs in relational learning shows our continued commitment to open science. We open-source three pieces of software that we expect to accelerate researc…
Deciphering Fingerprints of 3D Molecular Surfaces for Accurate Epitope Prediction
Fang Wu, Weihao Xuan, Jure Leskovec +2
Molecular surfaces encode the geometric and physicochemical patterns that determine antibody-antigen recognition, central to epitope prediction. However, existing methods rely on s…
M*: A Modular, Extensible, Serving System for Multimodal Models
Atindra Jha, Naomi Sagan, Keisuke Kamahori +9
We are entering a new era of composite model architectures that integrate diverse components such as vision encoders, language backbones, diffusion and flow heads, audio codecs, ac…
Learning over Positive and Negative Edges with Contrastive Message Passing
Peter Pao-Huang, Charilaos I. Kanatsoulis, Michael Bereket +1
Conventional approaches to learning on graphs involve message passing along existing (i.e., positive) edges to update node features. However, these approaches often disregard the p…
NoiseRater: Meta-Learned Noise Valuation for Diffusion Model Training
Fang Wu, Haokai Zhao, Da Xing +18
Training a diffusion model involves two sources of randomness for each data sample: the timestep and the Gaussian noise realization. The timestep has been studied extensively throu…
Proteo-R1: Reasoning Foundation Models for De Novo Protein Design
Fang Wu, Weihao Xuan, Heli Qi +26
Deep learning in de novo protein design has achieved atomic-level fidelity. However, existing models remain largely non-deliberative: they directly synthesize molecular geometries…
KumoRFM-2: Scaling Foundation Models for Relational Learning
Valter Hudovernik, Federico López, Vid Kocijan +4
We introduce KumoRFM-2, the next iteration of a pre-trained foundation model for relational data. KumoRFM-2 supports in-context learning as well as fine-tuning and is applicable to…
RelBench v2: A Large-Scale Benchmark and Repository for Relational Data
Justin Gu, Rishabh Ranjan, Charilaos Kanatsoulis +8
Relational deep learning (RDL) has emerged as a powerful paradigm for learning directly on relational databases by modeling entities and their relationships across multiple interco…
End-to-End Test-Time Training for Long Context
Arnuv Tandon, Karan Dalal, Xinhao Li +11
We formulate long-context language modeling as a problem in continual learning rather than architecture design. Under this formulation, we only use a standard architecture -- a Tra…
TGM: a Modular and Efficient Library for Machine Learning on Temporal Graphs
Jacob Chmura, Shenyang Huang, Tran Gia Bao Ngo +7
Well-designed open-source software drives progress in Machine Learning (ML) research. While static graph ML enjoys mature frameworks like PyTorch Geometric and DGL, ML for temporal…
Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data
Rishabh Ranjan, Valter Hudovernik, Mark Znidar +7
Pretrained transformers readily adapt to new sequence modeling tasks via zero-shot prompting, but relational domains still lack architectures that transfer across datasets and task…
A deep reinforcement learning platform for antibiotic discovery
Hanqun Cao, Marcelo D. T. Torres, Jingjie Zhang +8
Antimicrobial resistance (AMR) is projected to cause up to 10 million deaths annually by 2050, underscoring the urgent need for new antibiotics. Here we present ApexAmphion, a deep…
Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
Fang Wu, Aaron Tu, Weihao Xuan +21
Reinforcement learning with verifiable rewards (RLVR) is a practical, scalable way to improve large language models on math, code, and other structured tasks. However, we argue tha…
PyG 2.0: Scalable Learning on Real World Graphs
Matthias Fey, Jinu Sunil, Akihiro Nitta +10
PyG (PyTorch Geometric) has evolved significantly since its initial release, establishing itself as a leading framework for Graph Neural Networks. In this paper, we present Pyg 2.0…
Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards
Shirley Wu, Parth Sarthi, Shiyu Zhao +10
Compound AI systems integrating multiple components, such as Large Language Models, specialized tools, and traditional machine learning models, are increasingly deployed to solve c…
Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures
Vijay Prakash Dwivedi, Charilaos Kanatsoulis, Shenyang Huang +1
Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data and has been applied to molecules, social netwo…
RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models
Valter Hudovernik, Minkai Xu, Juntong Shi +4
Real-world databases are predominantly relational, comprising multiple interlinked tables that contain complex structural and statistical dependencies. Learning generative models o…
Relational Graph Transformer
Vijay Prakash Dwivedi, Sri Jaladi, Yangyi Shen +5
Relational Deep Learning (RDL) is a promising approach for building state-of-the-art predictive models on multi-table relational data by representing it as a heterogeneous temporal…
Automated Hypothesis Validation with Agentic Sequential Falsifications
Kexin Huang, Ying Jin, Ryan Li +3
Hypotheses are central to information acquisition, decision-making, and discovery. However, many real-world hypotheses are abstract, high-level statements that are difficult to val…
Learning Efficient Positional Encodings with Graph Neural Networks
Charilaos I. Kanatsoulis, Evelyn Choi, Stephanie Jegelka +2
Positional encodings (PEs) are essential for effective graph representation learning because they provide position awareness in inherently position-agnostic transformer architectur…