6 papers
Teaching Vision-Language Models to Use the Scale They Are Given: Label-Free Equivariance Training for Metric Physical Reasoning
Kaizhen Tan, Yang Feng, Heqing Du +3
Metric questions about video, such as the speed of a moving object, require a vision-language model to convert visual measurements into physical units using a real-world reference…
Beyond Multimodal Alignment: Shared Physical Representations Across Sensors and Action Orders
Kaizhen Tan, Xin Xu, Siru Tao +4
Multimodal world models are often evaluated by whether different sensors produce similar representations. However, similar representations do not necessarily imply that the models…
What Can Latent World Models Know? Physical Information in Multimodal Predictive Representations
Kaizhen Tan, Sizhe Xu, Xin Xu +6
A central premise of latent world models is that predicting the future encourages representations to internalize the physics of their environment. We ask which physical quantities…
How Much Can a LoRA Adapter Memorize? Measuring Adapter Capacity in Bits
Kaizhen Tan, Heqing Du, Yang Feng
LoRA adapters are often shared on their own, and the amount of information they can hold about their training data bounds what sharing them can reveal. Following Morris et al. (202…
Confidence under Visual Token Pruning: Removed Evidence and Risk-Controlled Token Budgets for MLLMs
Kaizhen Tan, Yang Feng, Heqing Du +3
Visual token pruning speeds up multimodal large language models (MLLMs) by keeping a small subset of the visual tokens, and pruning methods are compared by the accuracy they retain…
CREG: Compass Relational Evidence Graph for Characterizing Directional Structure in VLM Spatial-Reasoning Attribution
Kaizhen Tan, Yang Feng, Heqing Du
Standard attribution heatmaps show where a vision-language model (VLM) focuses, but they do not reveal whether the recovered evidence is organized by the queried spatial relation o…