Publications (20)
BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversity
Chih-Hsuan Yang, Benjamin Feuer, Zaki Jubery +12
We introduce BioTrove, the largest publicly accessible dataset designed to advance AI applications in biodiversity. Curated from the iNaturalist platform and vetted to include only…
Towards Large Reasoning Models for Agriculture
Hossein Zaremehrjerdi, Shreyan Ganguly, Ashlyn Rairdin +17
Agricultural decision-making involves complex, context-specific reasoning, where choices about crops, practices, and interventions depend heavily on geographic, climatic, and econo…
Hidden in the Noise: Two-Stage Robust Watermarking for Images
Kasra Arabi, Benjamin Feuer, R. Teal Witter +2
As the quality of image generators continues to improve, deepfakes become a topic of considerable societal debate. Image watermarking allows responsible model owners to detect and…
A Meta-Analysis of Distributionally-Robust Models
Benjamin Feuer, Ameya Joshi, Chinmay Hegde
State-of-the-art image classifiers trained on massive datasets (such as ImageNet) have been shown to be vulnerable to a range of both intentional and incidental distribution shifts…
OpenThoughts-Agent: Data Recipes for Agentic Models
Negin Raoof, Richard Zhuang, Marianna Nezhurina +47
Agentic language models dramatically expand the applications of AI yet little is publicly known about how to curate training data for broadly capable agents. Existing open efforts…
Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks
Benjamin Feuer, Chinmay Hegde, Niv Cohen
Tabular classification has traditionally relied on supervised algorithms, which estimate the parameters of a prediction model using its training data. Recently, Prior-Data Fitted N…
TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
Benjamin Feuer, Robin Tibor Schirrmeister, Valeriia Cherepanova +5
While tabular classification has traditionally relied on from-scratch training, a recent breakthrough called prior-data fitted networks (PFNs) challenges this approach. Similar to…
When Do Neural Nets Outperform Boosted Trees on Tabular Data?
Duncan McElfresh, Sujay Khandagale, Jonathan Valverde +6
Tabular data is one of the most commonly used types of data in machine learning. Despite recent advances in neural nets (NNs) for tabular data, there is still an active discussion…
MARVIS: Modality Adaptive Reasoning over VISualizations
Benjamin Feuer, Lennart Purucker, Oussama Elachqar +1
Predictive applications of machine learning often rely on small (sub 1 Bn parameter) specialized models tuned to particular domains or modalities. Such models often achieve excelle…
WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training
Benjamin Feuer, Chinmay Hegde
Language model (LLM) post-training, from DPO to distillation, can refine behaviors and unlock new skills, but the open science supporting these post-training techniques is still in…
LiT Tuned Models for Efficient Species Detection
Andre Nakkab, Benjamin Feuer, Chinmay Hegde
Recent advances in training vision-language models have demonstrated unprecedented robustness and transfer learning effectiveness; however, standard computer vision datasets are im…
OpenThoughts: Data Recipes for Reasoning Models
Etash Guha, Ryan Marten, Sedrick Keh +47
Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best training recipes for reasoni…
When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validity
Benjamin Feuer, Chiung-Yi Tseng, Astitwa Sarthak Lathe +2
LLM-judged benchmarks are increasingly used to evaluate complex model behaviors, yet their design introduces failure modes absent in conventional ground-truth based benchmarks. We…
Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation
Benjamin Feuer, Lucas Rosenblatt, Oussama Elachqar
As AI models progress beyond simple chatbots into more complex workflows, we draw ever closer to the event horizon beyond which AI systems will be utilized in autonomous, self-main…
Distributionally Robust Classification on a Data Budget
Benjamin Feuer, Ameya Joshi, Minh Pham +1
Real world uses of deep learning require predictable model behavior under distribution shifts. Models such as CLIP show emergent natural distributional robustness comparable to hum…
ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models
Benjamin Feuer, Yurong Liu, Chinmay Hegde +1
Existing deep-learning approaches to semantic column type annotation (CTA) have important shortcomings: they rely on semantic types which are fixed at training time; require a larg…
Exploring Dataset-Scale Indicators of Data Quality
Benjamin Feuer, Chinmay Hegde
Modern computer vision foundation models are trained on massive amounts of data, incurring large economic and environmental costs. Recent research has suggested that improving data…
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
Benjamin Feuer, Micah Goldblum, Teresa Datta +5
The release of ChatGPT in November 2022 sparked an explosion of interest in post-training and an avalanche of new preference optimization (PO) methods. These methods claim superior…
SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification
Benjamin Feuer, Jiawei Xu, Niv Cohen +3
Data curation is the problem of how to collect and organize samples into a dataset that supports efficient learning. Despite the centrality of the task, little work has been devote…
Caption supervision enables robust learners
Benjamin Feuer, Ameya Joshi, Chinmay Hegde
Vision language (VL) models like CLIP are robust to natural distribution shifts, in part because CLIP learns on unstructured data using a technique called caption supervision; the…