15 citations · 28 across the 19 of their papers we have counts for
22 papers
Few Trials, Many Metrics: Beta Mixtures for Low-Budget AI Evaluation
Gustavo Tironi, Felipe Maia Polo, Diego Mesquita
Evaluating a stochastic AI system often requires many repeated samples per instance, especially for metrics defined over long horizons. Recent work has shown that modeling instance…
Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization
Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti +2
Moving beyond evaluations that collapse performance across heterogeneous prompts toward fine-grained evaluation at the prompt level, or within relatively homogeneous subsets, is ne…
A Latent Variable Framework for Scaling Laws in Large Language Models
Peiyao Cai, Chengyu Cui, Felipe Maia Polo +6
We propose a statistical framework built on latent variable modeling for scaling laws of large language models (LLMs). Our work is motivated by the rapid emergence of numerous new…
COMET-poly: Machine Translation Metric Grounded in Other Candidates
Maike Züfle, Vilém Zouhar, Tu Anh Dinh +3
Automated metrics for machine translation attempt to replicate human judgment. Unlike humans, who often assess a translation in the context of multiple alternatives, these metrics…
Bridging Human and LLM Judgments: Understanding and Narrowing the Gap
Felipe Maia Polo, Xinhe Wang, Mikhail Yurochkin +3
Large language models are increasingly used as judges (LLM-as-a-judge) to evaluate model outputs at scale, but their assessments often diverge systematically from human judgments.…
Personalized Image Generation for Recommendations Beyond Catalogs
Gabriel Patron, Zhiwei Xu, Ishan Kapnadak +1
Personalization is central to human-AI interaction, yet current diffusion-based image generation systems remain largely insensitive to user diversity. Existing attempts to address…
CARROT: A Cost Aware Rate Optimal Router
Seamus Somerstep, Felipe Maia Polo, Allysson Flavio Melo de Oliveira +5
With the rapid growth in the number of Large Language Models (LLMs), there has been a recent interest in LLM routing, or directing queries to the cheapest LLM that can deliver a su…
Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families
Felipe Maia Polo, Seamus Somerstep, Leshem Choshen +2
Scaling laws for large language models (LLMs) predict model performance based on parameters like size and training data. However, differences in training configurations and data pr…
Microfoundation Inference for Strategic Prediction
Daniele Bracale, Subha Maity, Felipe Maia Polo +3
Often in prediction tasks, the predictive model itself can influence the distribution of the target variable, a phenomenon termed performative prediction. Generally, this influence…
LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Content
Nimrod Shabtay, Felipe Maia Polo, Sivan Doveh +8
The large-scale training of multi-modal models on data scraped from the web has shown outstanding utility in infusing these models with the required world knowledge to perform effe…
Efficient multi-prompt evaluation of LLMs
Felipe Maia Polo, Ronald Xu, Lucas Weber +6
Most popular benchmarks for comparing LLMs rely on a limited set of prompt templates, which may not fully capture the LLMs' abilities and can affect the reproducibility of results…
A transfer learning framework for weak-to-strong generalization
Seamus Somerstep, Felipe Maia Polo, Moulinath Banerjee +3
Modern large language model (LLM) alignment techniques rely on human feedback, but it is unclear whether these techniques fundamentally limit the capabilities of aligned LLMs. In p…
tinyBenchmarks: evaluating LLMs with fewer examples
Felipe Maia Polo, Lucas Weber, Leshem Choshen +3
The versatility of large language models (LLMs) led to the creation of diverse benchmarks that thoroughly test a variety of language models' abilities. These benchmarks consist of…
Weak Supervision Performance Evaluation via Partial Identification
Felipe Maia Polo, Subha Maity, Mikhail Yurochkin +2
Programmatic Weak Supervision (PWS) enables supervised model training without direct access to ground truth labels, utilizing weak labels from heuristics, crowdsourcing, or pre-tra…
Fusing Models with Complementary Expertise
Hongyi Wang, Felipe Maia Polo, Yuekai Sun +3
Training AI models that generalize across tasks and domains has long been among the open problems driving AI research. The emergence of Foundation Models made it easier to obtain e…
Conditional independence testing under misspecified inductive biases
Felipe Maia Polo, Yuekai Sun, Moulinath Banerjee
Conditional independence (CI) testing is a fundamental and challenging task in modern statistics and machine learning. Many modern methods for CI testing rely on powerful supervise…
A unified framework for dataset shift diagnostics
Felipe Maia Polo, Rafael Izbicki, Evanildo Gomes Lacerda +2
Supervised learning techniques typically assume training data originates from the target population. Yet, in reality, dataset shift frequently arises, which, if not adequately take…
LegalNLP -- Natural Language Processing methods for the Brazilian Legal Language
Felipe Maia Polo, Gabriel Caiaffa Floriano Mendonça, Kauê Capellato J. Parreira +6
We present and make available pre-trained language models (Phraser, Word2Vec, Doc2Vec, FastText, and BERT) for the Brazilian legal language, a Python package with functions to faci…
Effects of personality traits in predicting grade retention of Brazilian students
Carmen Melo Toledo, Guilherme Mendes Bassedon, Jonathan Batista Ferreira +4
Student's grade retention is a key issue faced by many education systems, especially those in developing countries. In this paper, we seek to gauge the relevance of students' perso…
OCDE: Odds Conditional Density Estimator
Alex Akira Okuno, Felipe Maia Polo
Conditional density estimation (CDE) models can be useful for many statistical applications, especially because the full conditional density is estimated instead of traditional reg…
Predicting Legal Proceedings Status: Approaches Based on Sequential Text Data
Felipe Maia Polo, Itamar Ciochetti, Emerson Bertolo
The objective of this paper is to develop predictive models to classify Brazilian legal proceedings in three possible classes of status: (i) archived proceedings, (ii) active proce…
Skills to not fall behind in school
Felipe Maia Polo
Many recent studies emphasize how important the role of cognitive and social-emotional skills can be in determining people's quality of life. Although skills are of great importanc…