49 citations · 135 across the 24 of their papers we have counts for
8 papers · 1 filter
CausalGraph2LLM: Evaluating LLMs for Causal Queries
Ivaxi Sheth, Bahare Fatemi, Mario Fritz
Causality is essential in scientific research, enabling researchers to interpret true relationships between variables. These causal relationships are often represented by causal gr…
Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries
Kiran Vodrahalli, Santiago Ontanon, Nilesh Tripuraneni +21
We introduce Michelangelo: a minimal, synthetic, and unleaked long-context reasoning evaluation for large language models which is also easy to automatically score. This evaluation…
Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights
Zhikai Chen, Haitao Mao, Jingzhe Liu +8
Given the ubiquity of graph data and its applications in diverse domains, building a Graph Foundation Model (GFM) that can work well across different graphs and tasks with a unifie…
ReMI: A Dataset for Reasoning with Multiple Images
Mehran Kazemi, Nishanth Dikkala, Ankit Anand +8
With the continuous advancement of large language models (LLMs), it is essential to create new benchmarks to effectively evaluate their expanding capabilities and identify areas fo…
Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
Bahare Fatemi, Mehran Kazemi, Anton Tsitsulin +6
Large language models (LLMs) have showcased remarkable reasoning capabilities, yet they remain susceptible to errors, particularly in temporal reasoning tasks involving complex tem…
Understanding Transformer Reasoning Capabilities via Graph Algorithms
Clayton Sanford, Bahare Fatemi, Ethan Hall +5
Which transformer scaling regimes are able to perfectly solve different classes of algorithmic problems? While tremendous empirical advances have been attained by transformer-based…
Don't Forget to Connect! Improving RAG with Graph-based Reranking
Jialin Dong, Bahare Fatemi, Bryan Perozzi +2
Retrieval Augmented Generation (RAG) has greatly improved the performance of Large Language Model (LLM) responses by grounding generation with context from existing documents. Thes…
Let Your Graph Do the Talking: Encoding Structured Data for LLMs
Bryan Perozzi, Bahare Fatemi, Dustin Zelle +4
How can we best encode structured data into sequential form for use in large language models (LLMs)? In this work, we introduce a parameter-efficient method to explicitly represent…