Publications (10)
AutoPhoto: Aesthetic Photo Capture using Reinforcement Learning
Hadi AlZayer, Hubert Lin, Kavita Bala
The process of capturing a well-composed photo is difficult and it takes years of experience to master. We propose a novel pipeline for an autonomous agent to automatically capture…
Learning Material-Aware Local Descriptors for 3D Shapes
Hubert Lin, Melinos Averkiou, Evangelos Kalogerakis +5
Material understanding is critical for design, geometric modeling, and analysis of functional objects. We enable material-aware 3D shape analysis by employing a projective convolut…
What Can Style Transfer and Paintings Do For Model Robustness?
Hubert Lin, Mitchell van Zuijlen, Sylvia C. Pont +2
A common strategy for improving model robustness is through data augmentations. Data augmentations encourage models to learn desired invariances, such as invariance to horizontal f…
DeepSemanticHPPC: Hypothesis-based Planning over Uncertain Semantic Point Clouds
Yutao Han, Hubert Lin, Jacopo Banfi +2
Planning in unstructured environments is challenging -- it relies on sensing, perception, scene reconstruction, and reasoning about various uncertainties. We propose DeepSemanticHP…
Block Annotation: Better Image Annotation for Semantic Segmentation with Sub-Image Decomposition
Hubert Lin, Paul Upchurch, Kavita Bala
Image datasets with high-quality pixel-level annotations are valuable for semantic segmentation: labelling every pixel in an image ensures that rare classes and small objects are a…
S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Modelwith Spatio-Temporal Visual Representation
Yichen Xie, Runsheng Xu, Tong He +9
The latest advancements in multi-modal large language models (MLLMs) have spurred a strong renewed interest in end-to-end motion planning approaches for autonomous driving. Many en…
WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios
Runsheng Xu, Hubert Lin, Wonseok Jeon +11
Vision-based end-to-end (E2E) driving has garnered significant interest in the research community due to its scalability and synergy with multimodal large language models (MLLMs).…
Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision
Mitchell J. P. van Zuijlen, Hubert Lin, Kavita Bala +2
A painter is free to modify how components of a natural scene are depicted, which can lead to a perceptually convincing image of the distal world. This signals a major difference b…
EMMA: End-to-End Multimodal Model for Autonomous Driving
Jyh-Jing Hwang, Runsheng Xu, Hubert Lin +11
We introduce EMMA, an End-to-end Multimodal Model for Autonomous driving. Built upon a multi-modal large language model foundation like Gemini, EMMA directly maps raw camera sensor…
Insights From A Large-Scale Database of Material Depictions In Paintings
Hubert Lin, Mitchell Van Zuijlen, Maarten W. A. Wijntjes +2
Deep learning has paved the way for strong recognition systems which are often both trained on and applied to natural images. In this paper, we examine the give-and-take relationsh…