3 citations · 9 across the 19 of their papers we have counts for
21 papers
Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
Reetu Raj Harsh, Bhaskarjit Sarmah, Stefano Pasquali
As Large Language Models (LLMs) are increasingly deployed in safety-critical applications, robust content moderation becomes essential. We present a comprehensive evaluation of 14…
FinReflectKG -- HalluBench: GraphRAG Hallucination Benchmark for Financial Question Answering Systems
Mahesh Kumar, Bhaskarjit Sarmah, Stefano Pasquali
As organizations increasingly integrate AI-powered question-answering systems into financial information systems for compliance, risk assessment, and decision support, ensuring the…
Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services
Fabrizio Dimino, Bhaskarjit Sarmah, Stefano Pasquali
Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Services, and Insurance (BFSI) deployments…
Deep Reinforcement Learning for Optimum Order Execution: Mitigating Risk and Maximizing Returns
Khabbab Zakaria, Jayapaulraj Jerinsh, Andreas Maier +3
Optimal Order Execution is a well-established problem in finance that pertains to the flawless execution of a trade (buy or sell) for a given volume within a specified time frame.…
Uncovering Representation Bias for Investment Decisions in Open-Source Large Language Models
Fabrizio Dimino, Krati Saxena, Bhaskarjit Sarmah +1
Large Language Models are increasingly adopted in financial applications to support investment workflows. However, prior studies have seldom examined how these models reflect biase…
FinCARE: Financial Causal Analysis with Reasoning and Evidence
Alejandro Michel, Abhinav Arun, Bhaskarjit Sarmah +1
Portfolio managers rely on correlation-based analysis and heuristic methods that fail to capture true causal relationships driving performance. We present a hybrid framework that i…
FinReflectKG -- MultiHop: Financial QA Benchmark for Reasoning with Knowledge Graph Evidence
Abhinav Arun, Reetu Raj Harsh, Bhaskarjit Sarmah +1
Multi-hop reasoning over financial disclosures is often a retrieval problem before it becomes a reasoning or generation problem: relevant facts are dispersed across sections, filin…
FINCH: Financial Intelligence using Natural language for Contextualized SQL Handling
Avinash Kumar Singh, Bhaskarjit Sarmah, Stefano Pasquali
Text-to-SQL, the task of translating natural language questions into SQL queries, has long been a central challenge in NLP. While progress has been significant, applying it to the…
FinReflectKG -- EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluation
Fabrizio Dimino, Abhinav Arun, Bhaskarjit Sarmah +1
Large language models (LLMs) are increasingly being used to extract structured knowledge from unstructured financial text. Although prior studies have explored various extraction m…
AlphaAgents: Large Language Model based Multi-Agents for Equity Portfolio Constructions
Tianjiao Zhao, Jingrao Lyu, Stokes Jones +3
The field of artificial intelligence (AI) agents is evolving rapidly, driven by the capabilities of Large Language Models (LLMs) to autonomously perform and refine tasks with human…
Tracing Positional Bias in Financial Decision-Making: Mechanistic Insights from Qwen2.5
Fabrizio Dimino, Krati Saxena, Bhaskarjit Sarmah +1
The growing adoption of large language models (LLMs) in finance exposes high-stakes decision-making to subtle, underexamined positional biases. The complexity and opacity of modern…
FinReflectKG: Agentic Construction and Evaluation of Financial Knowledge Graphs
Abhinav Arun, Fabrizio Dimino, Tejas Prakash Agarwal +2
The financial domain poses unique challenges for knowledge graph (KG) construction at scale due to the complexity and regulatory nature of financial documents. Despite the critical…
Supervised Similarity for High-Yield Corporate Bonds with Quantum Cognition Machine Learning
Joshua Rosaler, Luca Candelori, Vahagn Kirakosyan +5
We investigate the application of quantum cognition machine learning (QCML), a novel paradigm for both supervised and unsupervised learning tasks rooted in the mathematical formali…
A Comparative Study of DSPy Teleprompter Algorithms for Aligning Large Language Models Evaluation Metrics to Human Evaluation
Bhaskarjit Sarmah, Kriti Dutta, Anna Grigoryan +3
We argue that the Declarative Self-improving Python (DSPy) optimizers are a way to align the large language model (LLM) prompts and their evaluations to the human annotations. We p…
How to Choose a Threshold for an Evaluation Metric for Large Language Models
Bhaskarjit Sarmah, Mingshu Li, Jingrao Lyu +4
To ensure and monitor large language models (LLMs) reliably, various evaluation metrics have been proposed in the literature. However, there is little research on prescribing a met…
Can an unsupervised clustering algorithm reproduce a categorization system?
Nathalia Castellanos, Dhruv Desai, Sebastian Frank +2
Peer analysis is a critical component of investment management, often relying on expert-provided categorization systems. These systems' consistency is questioned when they do not a…
Case-based Explainability for Random Forest: Prototypes, Critics, Counter-factuals and Semi-factuals
Gregory Yampolsky, Dhruv Desai, Mingshu Li +2
The explainability of black-box machine learning algorithms, commonly known as Explainable Artificial Intelligence (XAI), has become crucial for financial and other regulated indus…
HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction
Bhaskarjit Sarmah, Benika Hall, Rohan Rao +3
Extraction and interpretation of intricate information from unstructured text data arising in financial applications, such as earnings call transcripts, present substantial challen…
Open Set Recognition for Random Forest
Guanchao Feng, Dhruv Desai, Stefano Pasquali +1
In many real-world classification or recognition tasks, it is often difficult to collect training examples that exhaust all possible classes due to, for example, incomplete knowled…
Towards reducing hallucination in extracting information from financial reports using Large Language Models
Bhaskarjit Sarmah, Tianjie Zhu, Dhagash Mehta +1
For a financial analyst, the question and answer (Q\&A) segment of the company financial report is a crucial piece of information for various analysis and investment decisions. How…
Enhanced Local Explainability and Trust Scores with Random Forest Proximities
Joshua Rosaler, Dhruv Desai, Bhaskarjit Sarmah +4
We initiate a novel approach to explain the predictions and out of sample performance of random forest (RF) regression and classification models by exploiting the fact that any RF…