6 citations · 26 across the 27 of their papers we have counts for
30 papers
Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agents
Liqin Ye, Haorui Wang, Fardin Ahmed +8
We introduce Forecast-Dojo, a replayable environment for benchmarking and training LLM forecasting agents. It combines resolved prediction-market questions with dated news, allowin…
Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation of LLM Nowcasts on Macroeconomic Indicators
Xinyue Zhao, Ruiyi Zhang, Liqin Ye +3
Nowcasting headline macroeconomic indicators, i.e., estimating an indicator's value for the current reference period before its official release, is critical for monetary policy an…
Credit Cards, Confusion, Computation, and Consequences: What Can We Uncover About Language Model Reasoning?
Arnav Hiray, Agam Shah, Caleb Lu +3
We introduce CreditCardQA, the first financial literacy benchmark for numerical reasoning derived from real credit card agreements. The dataset contains 1,800 questions, including…
IPO-Mine: A Toolkit and Dataset for Section-Structured Analysis of Long, Multimodal IPO Documents
Michael Galarnyk, Siddharth Lohani, Vidhyakshaya Kannan +8
An Initial Public Offering (IPO) filing is a document released when a private firm goes public, allowing individual (retail) investors to purchase its shares. These filings describ…
Evolutionary Task Discovery: Advancing Reasoning Frontiers via Skill Composition and Complexity Scaling
Liqin Ye, Yanbin Yin, Michael Galarnyk +3
The reasoning frontier of Large Language Models (LLMs) has advanced significantly through modern post-training paradigms (e.g., Reinforcement Learning from Verifiable Rewards (RLVR…
FinForge: Semi-Synthetic Financial Benchmark Generation
Glenn Matlin, Akhil Theerthala, Anant Gupta +4
Evaluating Language Models (LMs) in specialized, high-stakes domains such as finance remains a significant challenge due to the scarcity of open, high-quality, and domain-specific…
Financial Instruction Following Evaluation (FIFE)
Glenn Matlin, Siddharth, Anirudh JM +3
Language Models (LMs) struggle with complex, interdependent instructions, particularly in high-stakes domains like finance where precision is critical. We introduce FIFE, a novel,…
Precise Attribute Intensity Control in Large Language Models via Targeted Representation Editing
Rongzhi Zhang, Liqin Ye, Yuzhao Heng +5
Precise attribute intensity control--generating Large Language Model (LLM) outputs with specific, user-defined attribute intensities--is crucial for AI systems adaptable to diverse…
FinCap: Topic-Aligned Captions for Short-Form Financial YouTube Videos
Siddhant Sukhani, Yash Bhardwaj, Riya Bhadani +3
We evaluate multimodal large language models (MLLMs) for topic-aligned captioning in financial short-form videos (SVs) by testing joint reasoning over transcripts (T), audio (A), a…
Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement
Liqin Ye, Agam Shah, Chao Zhang +1
The traditional process of creating labeled datasets is labor-intensive and expensive. Recent breakthroughs in open-source large language models (LLMs) have opened up a new avenue…
Finance Language Model Evaluation (FLaME)
Glenn Matlin, Mika Okamoto, Huzaifa Pardawala +2
Language Models (LMs) have demonstrated impressive capabilities with core Natural Language Processing (NLP) tasks. The effectiveness of LMs for highly specialized knowledge-intensi…
VideoConviction: A Multimodal Benchmark for Human Conviction and Stock Market Recommendations
Michael Galarnyk, Veer Kejriwal, Agam Shah +4
Social media has amplified the reach of financial influencers known as "finfluencers," who share stock recommendations on platforms like YouTube. Understanding their influence requ…
Words That Unite The World: A Unified Framework for Deciphering Central Bank Communications Globally
Agam Shah, Siddhant Sukhani, Huzaifa Pardawala +24
Central banks around the world play a crucial role in maintaining economic stability. Deciphering policy implications in their communications is essential, especially as misinterpr…
KG-MuLQA: A Framework for KG-based Multi-Level QA Extraction and Long-Context LLM Evaluation
Nikita Tatarinov, Vidhyakshaya Kannan, Haricharana Srinivasa +7
We introduce KG-MuLQA (Knowledge-Graph-based Multi-Level Question-Answer Extraction): a framework that (1) extracts QA pairs at multiple complexity levels (2) along three key dimen…
Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunities
Nikita Tatarinov, Siddhant Sukhani, Agam Shah +1
Recent advances in language modeling have led to a growing number of papers related to finance in top-tier Natural Language Processing (NLP) venues. To systematically examine this…
Beyond the Reported Cutoff: Where Large Language Models Fall Short on Financial Knowledge
Agam Shah, Liqin Ye, Sebastian Jaskowski +2
Large Language Models (LLMs) are frequently utilized as sources of knowledge for question-answering. While it is known that LLMs may lack access to real-time data or newer data pro…
How Inclusively do LMs Perceive Social and Moral Norms?
Michael Galarnyk, Agam Shah, Dipanwita Guhathakurta +2
This paper discusses and contains offensive content. Language models (LMs) are used in decision-making systems and as interactive assistants. However, how well do these models maki…
CoCoHD: Congress Committee Hearing Dataset
Arnav Hiray, Yunsong Liu, Mingxiao Song +2
U.S. congressional hearings significantly influence the national economy and social fabric, impacting individual lives. Despite their importance, there is a lack of comprehensive d…
SubjECTive-QA: Measuring Subjectivity in Earnings Call Transcripts' QA Through Six-Dimensional Feature Analysis
Huzaifa Pardawala, Siddhant Sukhani, Agam Shah +7
Fact-checking is extensively studied in the context of misinformation and disinformation, addressing objective inaccuracies. However, a softer form of misinformation involves respo…
ConfReady: A RAG based Assistant and Dataset for Conference Checklist Responses
Michael Galarnyk, Rutwik Routu, Vidhyakshaya Kannan +4
The ARR Responsible NLP Research checklist website states that the "checklist is designed to encourage best practices for responsible research, addressing issues of research ethics…
Numerical Claim Detection in Finance: A New Financial Dataset, Weak-Supervision Model, and Market Analysis
Agam Shah, Arnav Hiray, Pratvi Shah +6
In this paper, we investigate the influence of claims in analyst reports and earnings calls on financial market returns, considering them as significant quarterly events for public…
Abnormal Trading Detection in the NFT Market
Mingxiao Song, Yunsong Liu, Agam Shah +1
The Non-Fungible-Token (NFT) market has experienced explosive growth in recent years. According to DappRadar, the total transaction volume on OpenSea, the largest NFT marketplace,…
Shifting Cryptocurrency Influence: A High-Resolution Network Analysis of Market Leaders
Arnav Hiray, Pratvi Shah, Vishwa Shah +3
Over the last decade, the cryptocurrency market has experienced unprecedented growth, emerging as a prominent financial market. As this market rapidly evolves, it necessitates re-e…
Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks
Agam Shah, Sudheer Chava
Recently large language models (LLMs) like ChatGPT have shown impressive performance on many natural language processing tasks with zero-shot. In this paper, we investigate the eff…