6 citations · 16 across the 14 of their papers we have counts for
14 papers
GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
Kaiyuan Chen, Shuangyu Xie, Letian Fu +21
For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world ada…
Visual Sculpting: Visually-Aligned Planning Representations for Long-Horizon Robot Clay Sculpting
Peter Schaldenbrand, Jean Oh
Clay sculpting is a nuanced, artistic task involving dexterous manipulation with long-horizon planning to achieve high-level goals. As a robotics problem, we formulate clay sculpti…
TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Space
Zhixuan Liu, Peter Schaldenbrand, Yijun Li +5
In video diffusion transformers, visual patch tokens maintain explicit correspondence to space and time. We hypothesize that their channel dimension can serve as a semantic control…
ShapeShift: Text-to-Mosaic Synthesis via Semantic Phase-Field Guidance
Vihaan Misra, Peter Schaldenbrand, Jean Oh
We present ShapeShift, a method for arranging rigid objects into configurations that visually convey semantic concepts specified by natural language. While pretrained diffusion mod…
Spline-FRIDA: Towards Diverse, Humanlike Robot Painting Styles with a Sample-Efficient, Differentiable Brush Stroke Model
Lawrence Chen, Peter Schaldenbrand, Tanmay Shankar +2
A painting is more than just a picture on a wall; a painting is a process comprised of many intentional brush strokes, the shapes of which are an important component of a painting'…
CoFRIDA: Self-Supervised Fine-Tuning for Human-Robot Co-Painting
Peter Schaldenbrand, Gaurav Parmar, Jun-Yan Zhu +2
Prior robot painting and drawing work, such as FRIDA, has focused on decreasing the sim-to-real gap and expanding input modalities for users, but the interaction with these systems…
SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generation
Zhixuan Liu, Peter Schaldenbrand, Beverley-Claire Okogwu +5
Accurate representation in media is known to improve the well-being of the people who consume it. Generative image models trained on large web-crawled datasets such as LAION are kn…
Robot Synesthesia: A Sound and Emotion Guided AI Painter
Vihaan Misra, Peter Schaldenbrand, Jean Oh
If a picture paints a thousand words, sound may voice a million. While recent robotic painting and image synthesis methods have achieved progress in generating visuals from text in…
Towards Equitable Representation in Text-to-Image Synthesis Models with the Cross-Cultural Understanding Benchmark (CCUB) Dataset
Zhixuan Liu, Youeun Shin, Beverley-Claire Okogwu +5
It has been shown that accurate representation in media improves the well-being of the people who consume it. By contrast, inaccurate representations can negatively affect viewers…
Towards Real-Time Text2Video via CLIP-Guided, Pixel-Level Optimization
Peter Schaldenbrand, Zhixuan Liu, Jean Oh
We introduce an approach to generating videos based on a series of given language descriptions. Frames of the video are generated sequentially and optimized by guidance from the CL…
FRIDA: A Collaborative Robot Painter with a Differentiable, Real2Sim2Real Planning Environment
Peter Schaldenbrand, James McCann, Jean Oh
Painting is an artistic process of rendering visual content that achieves the high-level communication goals of an artist that may change dynamically throughout the creative proces…
StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation
Peter Schaldenbrand, Zhixuan Liu, Jean Oh
Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however,…
StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis
Peter Schaldenbrand, Zhixuan Liu, Jean Oh
Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however,…
Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agent
Peter Schaldenbrand, Jean Oh
The objective of most Reinforcement Learning painting agents is to minimize the loss between a target image and the paint canvas. Human painter artistry emphasizes important featur…