"Skill Issues'': Data-Centric Optimization of Lakehouse Agents
arXiv:2606.01185
The paper proposes a data‑centric pipeline that optimizes the skills and environment files used by coding agents on a branching lakehouse, turning evaluation into state verification and achieving up to 28.6% reward improvement.
Abstract
Coding agents are becoming users of data infrastructure, but their success depends not only on model quality: it also depends on the skills and environment files that teach agents how to use a system. We study how to optimize these artifacts for agents operating on a branching lakehouse, Bauplan. In our setting, headless APIs and Git-like data primitives expose data workflows through code, branches, commits, and merges. Our central observation is that a branching lakehouse turns data-agent evaluation from an output-matching problem into a state-verification problem: agent-generated pipeline code induces concrete, inspectable lakehouse changes. We present a data-centric optimization pipeline that generates task-verifier pairs, executes candidate skills in isolated sandboxes, and scores trajectories using both trace-level signals and programmatic checks over lakehouse state. In a preliminary evaluation on hundreds of tasks, optimized skills improve held-out reward by up to 28.6%. These results suggest that write-path data workflows provide a useful substrate for optimizing agent skills beyond read-only tasks.
Pre-print of paper accepted at ADS @ VLDB 2026, Boston