1 citations · 1 across the 7 of their papers we have counts for
8 papers
A Human-Centric Evaluation of a Retrieval-Augmented Generation System for Explaining Quebec Insurance Contracts
David Beauchemin, Richard Khoury
With the rise of online insurance sales, consumers face a significant \enquote{advice gap}, requiring them to navigate complex legal contracts without expert guidance. This paper p…
Benchmarking Large Language Models for Quebec Insurance: From Closed-Book to Retrieval-Augmented Generation
David Beauchemin, Richard Khoury
The digitization of insurance distribution in the Canadian province of Quebec, accelerated by legislative changes such as Bill 141, has created a significant "advice gap", leaving…
COLE: a Comprehensive Benchmark for French Language Understanding Evaluation
David Beauchemin, Yan Tremblay, Mohamed Amine Youssef +1
To address the need for a more comprehensive evaluation of French Natural Language Understanding (NLU), we introduce COLE, a new benchmark composed of 23 diverse task covering a br…
Idiom Understanding as a Tool to Measure the Dialect Gap
David Beauchemin, Yan Tremblay, Mohamed Amine Youssef +1
The tasks of idiom understanding and dialect understanding are both well-established benchmarks in natural language processing. In this paper, we propose combining them, and using…
QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs
David Beauchemin, Pier-Luc Veilleux, Johanna-Pascale Roy +1
In this paper, we introduce the Quebec-French Benchmark of Linguistic Minimal Pairs (QFrBLiMP), a corpus designed to evaluate LLMs' linguistic knowledge of prominent grammatical ph…
JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences
David Beauchemin, Michelle Albert-Rochette, Richard Khoury +1
Simplifying text while preserving its meaning is a complex yet essential task, especially in sensitive domain applications like legal texts. When applied to a specialized field, li…
QFrCoLA: a Quebec-French Corpus of Linguistic Acceptability Judgments
David Beauchemin, Richard Khoury
Large and Transformer-based language models perform outstandingly in various downstream tasks. However, there is limited understanding regarding how these models internalize lingui…
Quebec Automobile Insurance Question-Answering With Retrieval-Augmented Generation
David Beauchemin, Zachary Gagnon, Ricahrd Khoury
Large Language Models (LLMs) perform outstandingly in various downstream tasks, and the use of the Retrieval-Augmented Generation (RAG) architecture has been shown to improve perfo…