2 citations · 2 across the 9 of their papers we have counts for
9 papers · 1 filter
PRISM: Transfer and Limits of Activation Interventions in Tool-Calling Language Models
Giovana Kerche Bonás, Thales Sales Almeida, Marcelo da Silva Reis +2
Recent literature shows that unsafe tool calls can often accompany textual refusals in language models. To investigate this hidden vulnerability, we introduce PRISM, a framework fo…
LLM-Based Persuasion Enables Guardrail Override in Frontier LLMs
Rodrigo Nogueira, Thales Sales Almeida, Giovana Kerche Bonás +7
Frontier assistant LLMs ship with strong guardrails: asked directly to write a persuasive essay denying the Holocaust, denying vaccine safety, defending flat-earth cosmology, argui…
Magis-Bench: Evaluating LLMs on Magistrate-Level Legal Tasks
Ramon Pires, Thales Sales Almeida, Celio Larcher Junior +6
Existing benchmarks for legal AI focus primarily on tasks where LLMs must produce legal arguments or documents, yet the capacity to \emph{judge} such arguments -- weighing competin…
Prosa: Rubric-Based Evaluation of LLMs on Real User Chats in Brazilian Portuguese
Roseval Malaquias Junior, Giovana Kerche Bonás, Thales Sales Almeida +6
Rankings produced by holistic LLM-as-a-judge scoring are sensitive to the bias of the chosen judge model. We show that switching to binary rubric scoring with multi-judge filtering…
Measuring Opinion Bias and Sycophancy via LLM-based Persuasion
Rodrigo Nogueira, Giovana Kerche Bonás, Thales Sales Almeida +7
Large language models increasingly shape the information people consume: they are embedded in search, consulted for professional advice, deployed as agents, and used as a first sto…
MARCA: A Checklist-Based Benchmark for Multilingual Web Search
Thales Sales Almeida, Giovana Kerche Bonás, Ramon Pires +6
Large language models (LLMs) are increasingly used as sources of information, yet their reliability depends on the ability to search the web, select relevant evidence, and synthesi…
Automatic Legal Writing Evaluation of LLMs
Ramon Pires, Roseval Malaquias Junior, Rodrigo Nogueira
Despite the recent advances in Large Language Models, benchmarks for evaluating legal writing remain scarce due to the inherent complexity of assessing open-ended responses in this…
The interplay between domain specialization and model size
Roseval Malaquias Junior, Ramon Pires, Thales Sales Almeida +3
Scaling laws for language models have often focused on finding the optimal model size and token count for training from scratch. However, achieving this optimal balance requires si…
Juru: Legal Brazilian Large Language Model from Reputable Sources
Roseval Malaquias Junior, Ramon Pires, Roseli Romero +1
The high compute cost associated with pretraining large language models limits their research. Two strategies have emerged to address this issue: domain specialization and pretrain…