5 papers
What Does Attention Transfer Transfer? Attention Structure and Robustness in Vision Transformers
Jesse Ponnock
Vision transformers (ViTs) trained to copy a pretrained teacher's attention maps recover most of fine-tuning's in-distribution accuracy yet fall measurably short of it under distri…
Reinforcement Learning in Super Mario Bros: Curriculum, Pedagogy, and Optimal Level Design in World 1-1
Jesse Ponnock, Lucas Ho
World 1-1 of Super Mario Bros is widely celebrated as a masterclass in game design: its progressive structure is credited with teaching players core mechanics through the level its…
The Data Efficiency Frontier of Financial Foundation Models: Scaling Laws from Continued Pretraining
Jesse Ponnock
Domain-adaptive pretraining (DAPT) offers a practical path to specializing large language models for high-value domains without full retraining. We conduct an early-stage scaling-l…
Real-Time RAG for the Identification of Supply Chain Vulnerabilities
Jesse Ponnock, Grace Kenneally, Michael Robert Briggs +5
New technologies in generative AI can enable deeper analysis into our nation's supply chains but truly informative insights require the continual updating and aggregation of massiv…
Generative AI for Strategic Plan Development
Jesse Ponnock
Given recent breakthroughs in Generative Artificial Intelligence (GAI) and Large Language Models (LLMs), more and more professional services are being augmented through Artificial…