4 citations · 6 across the 13 of their papers we have counts for
14 papers
Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions
Oluwanifemi Bamgbose, Simon Rosen, Jash Shah +6
Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Model (Audio-LLM) judges) are expected to reflect human perception,…
EVA-Bench: A New End-to-end Framework for Evaluating Voice Agents
Tara Bogavelli, Gabrielle Gauthier Melançon, Katrina Stankiewicz +10
Voice agents are increasingly deployed across enterprise applications. However, no existing benchmark jointly addresses realistic conversation simulation and comprehensive voice-sp…
Evaluating Robustness of Large Language Models in Enterprise Applications: Benchmarks for Perturbation Consistency Across Formats and Languages
Tara Bogavelli, Oluwanifemi Bamgbose, Gabrielle Gauthier Melançon +2
Enterprise LLM applications require consistently high quality and reliable performance across diverse scenarios, demanding robustness to minor variations. Existing research shows t…
Apriel-1.5-15b-Thinker
Shruthan Radhakrishna, Aman Tiwari, Aanjaneya Shukla +21
We present Apriel-1.5-15B-Thinker, a 15-billion parameter open-weights multimodal reasoning model that achieves frontier-level performance through training design rather than sheer…
AU-Harness: An Open-Source Toolkit for Efficient and Unified Evaluation of AUdio-LLMs
Hoang Nguyen, Sidharth Surapaneni, Akshay Kalkunte +9
Large Audio Language Models (LALMs) are rapidly advancing, but evaluating them remains challenging due to inefficient and nonstandardized toolkits that limit fair comparison and sy…
Apriel-Nemotron-15B-Thinker
Shruthan Radhakrishna, Soham Parikh, Gopal Sarda +32
While large language models (LLMs) have achieved remarkable reasoning capabilities across domains like code, math and other enterprise tasks, their significant memory and computati…
DNR Bench: Benchmarking Over-Reasoning in Reasoning LLMs
Masoud Hashemi, Oluwanifemi Bamgbose, Sathwik Tejaswi Madhusudhan +3
Test-time scaling has significantly improved large language model performance, enabling deeper reasoning to solve complex problems. However, this increased reasoning capability als…
Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models
Meghana Rajeev, Rajkumar Ramamurthy, Prapti Trivedi +5
We investigate the robustness of reasoning models trained for step-by-step problem solving by introducing query-agnostic adversarial triggers - short, irrelevant text that, when ap…
Practical Guide for Causal Pathways and Sub-group Disparity Analysis
Farnaz Kohankhaki, Shaina Raza, Oluwanifemi Bamgbose +2
In this study, we introduce the application of causal disparity analysis to unveil intricate relationships and causal pathways between sensitive attributes and the targeted outcome…
Developing Safe and Responsible Large Language Model : Can We Balance Bias Reduction and Language Understanding in Large Language Models?
Shaina Raza, Oluwanifemi Bamgbose, Shardul Ghuge +3
Large Language Models (LLMs) have advanced various Natural Language Processing (NLP) tasks, such as text generation and translation, among others. However, these models often gener…
FakeWatch: A Framework for Detecting Fake News to Ensure Credible Elections
Shaina Raza, Tahniat Khan, Veronica Chatrath +3
In today's technologically driven world, the rapid spread of fake news, particularly during critical events like elections, poses a growing threat to the integrity of information.…
FakeWatch ElectionShield: A Benchmarking Framework to Detect Fake News for Credible US Elections
Tahniat Khan, Mizanur Rahman, Veronica Chatrath +2
In today's technologically driven world, the spread of fake news, particularly during crucial events such as elections, presents an increasing challenge to the integrity of informa…
She had Cobalt Blue Eyes: Prompt Testing to Create Aligned and Sustainable Language Models
Veronica Chatrath, Oluwanifemi Bamgbose, Shaina Raza
As the use of large language models (LLMs) increases within society, as does the risk of their misuse. Appropriate safeguards must be in place to ensure LLM outputs uphold the ethi…
Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysis
Shaina Raza, Oluwanifemi Bamgbose, Veronica Chatrath +3
Bias detection in text is crucial for combating the spread of negative stereotypes, misinformation, and biased decision-making. Traditional language models frequently face challeng…