6 citations · 13 across the 12 of their papers we have counts for
13 papers
Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval
Nickil Maveli, Antonio Vergari, Shay B. Cohen
Large language models (LLMs) have demonstrated strong performance in code generation, where success depends on both recalling relevant algorithmic knowledge and reasoning about how…
Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information
Antonio Valerio Miceli-Barone, Vaishak Belle, Shay B. Cohen
In this work we study agents in simulated bargaining scenarios, where a buyer and a seller communicate through a text channel and attempt to negotiate mutually beneficial trades, u…
Can LLMs Compress (and Decompress)? Evaluating Code Understanding and Execution via Invertibility
Nickil Maveli, Antonio Vergari, Shay B. Cohen
LLMs demonstrate strong performance on code benchmarks, yet consistent reasoning across forward and backward execution remains elusive. We present RoundTripCodeEval (RTCE), a bench…
One More Question is Enough, Expert Question Decomposition (EQD) Model for Domain Quantitative Reasoning
Mengyu Wang, Sotirios Sabanis, Miguel de Carvalho +2
Domain-specific quantitative reasoning remains a major challenge for large language models (LLMs), especially in fields requiring expert knowledge and complex question answering (Q…
Pre-training Time Series Models with Stock Data Customization
Mengyu Wang, Tiejun Ma, Shay B. Cohen
Stock selection, which aims to predict stock prices and identify the most profitable ones, is a crucial task in finance. While existing methods primarily focus on developing model…
Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour
Bálint Gyevnár, Christopher G. Lucas, Stefano V. Albrecht +1
Autonomous multi-agent systems (MAS) are useful for automating complex tasks but raise trust concerns due to risks such as miscoordination or goal misalignment. Explainability is v…
Theorem Prover as a Judge for Synthetic Data Generation
Joshua Ong Jun Leang, Giwon Hong, Wenda Li +1
The demand for synthetic data in mathematical reasoning has increased due to its potential to enhance the mathematical capabilities of large language models (LLMs). However, ensuri…
Modeling News Interactions and Influence for Financial Market Prediction
Mengyu Wang, Shay B. Cohen, Tiejun Ma
The diffusion of financial news into market prices is a complex process, making it challenging to evaluate the connections between news events and market movements. This paper intr…
CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning
Joshua Ong Jun Leang, Aryo Pradipta Gema, Shay B. Cohen
Mathematical reasoning remains a significant challenge for large language models (LLMs), despite progress in prompting techniques such as Chain-of-Thought (CoT). We present **Chain…
What can Large Language Models Capture about Code Functional Equivalence?
Nickil Maveli, Antonio Vergari, Shay B. Cohen
Code-LLMs, LLMs pre-trained on large code corpora, have shown great progress in learning rich representations of the structure and syntax of code, successfully using it to generate…
Evaluating Automatic Metrics with Incremental Machine Translation Systems
Guojun Wu, Shay B. Cohen, Rico Sennrich
We introduce a dataset comprising commercial machine translations, gathered weekly over six years across 12 translation directions. Since human A/B testing is commonly used, we ass…
People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Explainable AI
Balint Gyevnar, Stephanie Droop, Tadeg Quillien +4
It is often argued that effective human-centered explainable artificial intelligence (XAI) should resemble human reasoning. However, empirical investigations of how concepts from c…
On the Trade-off between Redundancy and Local Coherence in Summarization
Ronald Cardenas, Matthias Galle, Shay B. Cohen
Extractive summaries are usually presented as lists of sentences with no expected cohesion between them and with plenty of redundant information if not accounted for. In this paper…