8 papers
Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization
Haoyue Liu, Xiaoyu Ma, Ye Chen +2
Automatic prompt optimization (APO) has been widely adopted to adapt vision-language models (VLMs) to downstream tasks without weight updates, yielding promising results. However,…
One Rewrite to Fix Them All? Type-Aware Repair Allocation for Text-to-Image Prompt Optimization
Haoyue Liu, Xiaoyu Ma, Ye Chen +2
Text-to-image (T2I) generators often fail to follow their prompts faithfully, producing wrong counts, swapped attributes, ambiguous relations, and illegible text. Prompt optimizati…
SIGMA: Skill-Incidence Graphs for Compositional Multi-Agent Design
Kun Zeng, Yu Huo, Siyu Zhang +5
Existing graph-based multi-agent system (MAS) designers mainly improve collaboration by optimizing communication topologies over predefined agents, roles, or groups. However, becau…
Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought
Yu Huo, Siyu Zhang, Kun Zeng +7
Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints, notably generative numeracy…
Off-Distribution Voices: Fanfiction Subgenres as Universal Vernacular Jailbreaks for Aligned LLMs
Zhongze Luo, Ruihe Shi, Zhenshuai Yin +3
Existing jailbreaks against aligned LLMs are discrete artifacts whose surface forms are easy to fingerprint and patch. We argue that the real failure mode is not any specific promp…
Select Smarter, Not More: Prompt-Aware Evaluation Scheduling with Submodular Guarantees
Xiaoyu Ma, Yiwen Li, Haoyue Liu +4
Automatic prompt optimization (APO) hinges on the quality of its evaluation signal, yet scoring every prompt candidate on the full training set is prohibitively expensive. Existing…