3 citations · 10 across the 17 of their papers we have counts for
18 papers
Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
Reetu Raj Harsh, Bhaskarjit Sarmah, Stefano Pasquali
As Large Language Models (LLMs) are increasingly deployed in safety-critical applications, robust content moderation becomes essential. We present a comprehensive evaluation of 14…
FinReflectKG -- HalluBench: GraphRAG Hallucination Benchmark for Financial Question Answering Systems
Mahesh Kumar, Bhaskarjit Sarmah, Stefano Pasquali
As organizations increasingly integrate AI-powered question-answering systems into financial information systems for compliance, risk assessment, and decision support, ensuring the…
Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services
Fabrizio Dimino, Bhaskarjit Sarmah, Stefano Pasquali
Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Services, and Insurance (BFSI) deployments…
Memoria: A Scalable Agentic Memory Framework for Personalized Conversational AI
Samarth Sarin, Lovepreet Singh, Bhaskarjit Sarmah +1
Agentic memory is emerging as a key enabler for large language models (LLM) to maintain continuity, personalization, and long-term context in extended user interactions, critical c…
Uncovering Representation Bias for Investment Decisions in Open-Source Large Language Models
Fabrizio Dimino, Krati Saxena, Bhaskarjit Sarmah +1
Large Language Models are increasingly adopted in financial applications to support investment workflows. However, prior studies have seldom examined how these models reflect biase…
FinCARE: Financial Causal Analysis with Reasoning and Evidence
Alejandro Michel, Abhinav Arun, Bhaskarjit Sarmah +1
Portfolio managers rely on correlation-based analysis and heuristic methods that fail to capture true causal relationships driving performance. We present a hybrid framework that i…
FinReflectKG -- MultiHop: Financial QA Benchmark for Reasoning with Knowledge Graph Evidence
Abhinav Arun, Reetu Raj Harsh, Bhaskarjit Sarmah +1
Multi-hop reasoning over financial disclosures is often a retrieval problem before it becomes a reasoning or generation problem: relevant facts are dispersed across sections, filin…
FINCH: Financial Intelligence using Natural language for Contextualized SQL Handling
Avinash Kumar Singh, Bhaskarjit Sarmah, Stefano Pasquali
Text-to-SQL, the task of translating natural language questions into SQL queries, has long been a central challenge in NLP. While progress has been significant, applying it to the…
FinReflectKG -- EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluation
Fabrizio Dimino, Abhinav Arun, Bhaskarjit Sarmah +1
Large language models (LLMs) are increasingly being used to extract structured knowledge from unstructured financial text. Although prior studies have explored various extraction m…
Tracing Positional Bias in Financial Decision-Making: Mechanistic Insights from Qwen2.5
Fabrizio Dimino, Krati Saxena, Bhaskarjit Sarmah +1
The growing adoption of large language models (LLMs) in finance exposes high-stakes decision-making to subtle, underexamined positional biases. The complexity and opacity of modern…
FinReflectKG: Agentic Construction and Evaluation of Financial Knowledge Graphs
Abhinav Arun, Fabrizio Dimino, Tejas Prakash Agarwal +2
The financial domain poses unique challenges for knowledge graph (KG) construction at scale due to the complexity and regulatory nature of financial documents. Despite the critical…
A Comparative Study of DSPy Teleprompter Algorithms for Aligning Large Language Models Evaluation Metrics to Human Evaluation
Bhaskarjit Sarmah, Kriti Dutta, Anna Grigoryan +3
We argue that the Declarative Self-improving Python (DSPy) optimizers are a way to align the large language model (LLM) prompts and their evaluations to the human annotations. We p…
How to Choose a Threshold for an Evaluation Metric for Large Language Models
Bhaskarjit Sarmah, Mingshu Li, Jingrao Lyu +4
To ensure and monitor large language models (LLMs) reliably, various evaluation metrics have been proposed in the literature. However, there is little research on prescribing a met…
HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction
Bhaskarjit Sarmah, Benika Hall, Rohan Rao +3
Extraction and interpretation of intricate information from unstructured text data arising in financial applications, such as earnings call transcripts, present substantial challen…
Quantile Regression using Random Forest Proximities
Mingshu Li, Bhaskarjit Sarmah, Dhruv Desai +4
Due to the dynamic nature of financial markets, maintaining models that produce precise predictions over time is difficult. Often the goal isn't just point prediction but determini…
Towards reducing hallucination in extracting information from financial reports using Large Language Models
Bhaskarjit Sarmah, Tianjie Zhu, Dhagash Mehta +1
For a financial analyst, the question and answer (Q\&A) segment of the company financial report is a crucial piece of information for various analysis and investment decisions. How…
Enhanced Local Explainability and Trust Scores with Random Forest Proximities
Joshua Rosaler, Dhruv Desai, Bhaskarjit Sarmah +4
We initiate a novel approach to explain the predictions and out of sample performance of random forest (RF) regression and classification models by exploiting the fact that any RF…
Learning Embedded Representation of the Stock Correlation Matrix using Graph Machine Learning
Bhaskarjit Sarmah, Nayana Nair, Dhagash Mehta +1
Understanding non-linear relationships among financial instruments has various applications in investment processes ranging from risk management, portfolio construction and trading…