49 citations · 135 across the 24 of their papers we have counts for
25 papers
Gemma 4 Technical Report
Gemma Team, Sherif El Abd, Vaibhav Aggarwal +320
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemm…
Multi-Turn Puzzles: Evaluating Interactive Reasoning and Strategic Dialogue in LLMs
Kartikeya Badola, Jonathan Simon, Arian Hosseini +7
Large language models (LLMs) excel at solving problems with clear and complete statements, but often struggle with nuanced environments or interactive tasks which are common in mos…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
BIG-Bench Extra Hard
Mehran Kazemi, Bahare Fatemi, Hritik Bansal +17
Large language models (LLMs) are increasingly deployed in everyday applications, demanding robust general reasoning capabilities and diverse reasoning skillset. However, current LL…
CausalGraph2LLM: Evaluating LLMs for Causal Queries
Ivaxi Sheth, Bahare Fatemi, Mario Fritz
Causality is essential in scientific research, enabling researchers to interpret true relationships between variables. These causal relationships are often represented by causal gr…
Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries
Kiran Vodrahalli, Santiago Ontanon, Nilesh Tripuraneni +21
We introduce Michelangelo: a minimal, synthetic, and unleaked long-context reasoning evaluation for large language models which is also easy to automatically score. This evaluation…
Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights
Zhikai Chen, Haitao Mao, Jingzhe Liu +8
Given the ubiquity of graph data and its applications in diverse domains, building a Graph Foundation Model (GFM) that can work well across different graphs and tasks with a unifie…
ReMI: A Dataset for Reasoning with Multiple Images
Mehran Kazemi, Nishanth Dikkala, Ankit Anand +8
With the continuous advancement of large language models (LLMs), it is essential to create new benchmarks to effectively evaluate their expanding capabilities and identify areas fo…
Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
Bahare Fatemi, Mehran Kazemi, Anton Tsitsulin +6
Large language models (LLMs) have showcased remarkable reasoning capabilities, yet they remain susceptible to errors, particularly in temporal reasoning tasks involving complex tem…
Understanding Transformer Reasoning Capabilities via Graph Algorithms
Clayton Sanford, Bahare Fatemi, Ethan Hall +5
Which transformer scaling regimes are able to perfectly solve different classes of algorithmic problems? While tremendous empirical advances have been attained by transformer-based…
Don't Forget to Connect! Improving RAG with Graph-based Reranking
Jialin Dong, Bahare Fatemi, Bryan Perozzi +2
Retrieval Augmented Generation (RAG) has greatly improved the performance of Large Language Model (LLM) responses by grounding generation with context from existing documents. Thes…
Let Your Graph Do the Talking: Encoding Structured Data for LLMs
Bryan Perozzi, Bahare Fatemi, Dustin Zelle +4
How can we best encode structured data into sequential form for use in large language models (LLMs)? In this work, we introduce a parameter-efficient method to explicitly represent…
Talk like a Graph: Encoding Graphs for Large Language Models
Bahare Fatemi, Jonathan Halcrow, Bryan Perozzi
Graphs are a powerful tool for representing and analyzing complex relationships in real-world applications such as social networks, recommender systems, and computational finance.…
UGSL: A Unified Framework for Benchmarking Graph Structure Learning
Bahare Fatemi, Sami Abu-El-Haija, Anton Tsitsulin +5
Graph neural networks (GNNs) demonstrate outstanding performance in a broad range of applications. While the majority of GNN applications assume that a graph structure is given, so…
TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs
Phitchaya Mangpo Phothilimthana, Sami Abu-El-Haija, Kaidi Cao +4
Precise hardware performance models play a crucial role in code optimizations. They can assist compilers in making heuristic decisions or aid autotuners in identifying the optimal…
Learning to Substitute Ingredients in Recipes
Bahare Fatemi, Quentin Duval, Rohit Girdhar +2
Recipe personalization through ingredient substitution has the potential to help people meet their dietary needs and preferences, avoid potential allergens, and ease culinary explo…
Using Graph Algorithms to Pretrain Graph Completion Transformers
Jonathan Pilault, Michael Galkin, Bahare Fatemi +3
Recent work on Graph Neural Networks has demonstrated that self-supervised pretraining can further enhance performance on downstream graph, link, and node classification tasks. How…
Knowledge Hypergraph Embedding Meets Relational Algebra
Bahare Fatemi, Perouz Taslakian, David Vazquez +1
Embedding-based methods for reasoning in knowledge hypergraphs learn a representation for each entity and relation. Current methods do not capture the procedural rules underlying t…
SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
Bahare Fatemi, Layla El Asri, Seyed Mehran Kazemi
Graph neural networks (GNNs) work well when the graph structure is provided. However, this structure may not always be available in real-world applications. One solution to this pr…
Knowledge Hypergraphs: Prediction Beyond Binary Relations
Bahare Fatemi, Perouz Taslakian, David Vazquez +1
Knowledge graphs store facts using relations between two entities. In this work, we address the question of link prediction in knowledge hypergraphs where relations are defined on…
Improved Knowledge Graph Embedding using Background Taxonomic Information
Bahare Fatemi, Siamak Ravanbakhsh, David Poole
Knowledge graphs are used to represent relational information in terms of triples. To enable learning about domains, embedding models, such as tensor factorization models, can be u…
Structure Learning for Relational Logistic Regression: An Ensemble Approach
Nandini Ramanan, Gautam Kunapuli, Tushar Khot +5
We consider the problem of learning Relational Logistic Regression (RLR). Unlike standard logistic regression, the features of RLRs are first-order formulae with associated weight…
Record Linkage to Match Customer Names: A Probabilistic Approach
Bahare Fatemi, Seyed Mehran Kazemi, David Poole
Consider the following problem: given a database of records indexed by names (e.g., name of companies, restaurants, businesses, or universities) and a new name, determine whether t…
Comparing Aggregators for Relational Probabilistic Models
Seyed Mehran Kazemi, Bahare Fatemi, Alexandra Kim +5
Relational probabilistic models have the challenge of aggregation, where one variable depends on a population of other variables. Consider the problem of predicting gender from mov…