7 papers
LightMTP: Lightweight Latent Multi-Token Prediction
Tamara Czinczoll, Julie Kallini, Gerard de Melo +1
Next-token prediction (NTP) is the standard pretraining objective for large language models, yet it provides an explicit training signal only for the immediate next token, which ca…
Evaluating LLMs in Database Scenarios: A Lifecycle Benchmark for Assessing Their Potential in Core Database Tasks
Shunfan Zheng, Dongsheng Shi, Yue Li +3
Large Language Models (LLMs) are transforming database interaction paradigms, evolving from simple query translators to autonomous database administrators (DBAs). However, current…
Aligning Large Vision-Language Models at Test Time: A Trajectory-Guided Structured Sampling Approach
Tianbao Jiang, Weicong Ni, Gerard de Melo +1
Post-training reinforcement learning (RL) algorithms are commonly used to align large vision-language models (LVLMs) with human intent and the requirements of visual reasoning task…
AGMark: Attention-Guided Dynamic Watermarking for Large Vision-Language Models
Yue Li, Xin Yi, Dongsheng Shi +3
Watermarking has emerged as a pivotal solution for content traceability and intellectual property protection in large vision language models (LVLMs). However, vision-agnostic water…
From Construction to Injection: Edit-Based Fingerprints for Large Language Models
Yue Li, Xin Yi, Dongsheng Shi +3
Reliable model fingerprints are essential for protecting large language models (LLMs) against unauthorized redistribution and commercial misuse. In black-box deployment, verificati…
Image Tokens Matter: Mitigating Hallucination in Discrete Tokenizer-based Large Vision-Language Models via Latent Editing
Weixing Wang, Zifeng Ding, Jindong Gu +4
Large Vision-Language Models (LVLMs) with discrete image tokenizers unify multimodal representations by encoding visual inputs into a finite set of tokens. Despite their effectiven…
Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models
Yue Li, Xin Yi, Dongsheng Shi +3
With the increasing size of Large Vision-Language Models (LVLMs), network pruning techniques aimed at compressing models for deployment in resource-constrained environments have ga…