3 citations · 9 across the 10 of their papers we have counts for
11 papers
Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding
Yiheng Wang, Lichen Zhu, Yueqian Lin +4
Multimodal Large Language Models (MLLMs) have shown strong performance on video question answering, but their application to long-form videos is constrained by limited context leng…
MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models
Zhongxi Wang, Yueqian Lin, Jingyang Zhang +3
Safety evaluation of multimodal large language models requires tracking not only whether an attack succeeds, but also how the interaction unfolds across turns and input modalities.…
Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks
Chiyue Wei, Bowen Duan, Cong Guo +4
Spiking Neural Networks (SNNs) are gaining attention for their energy efficiency and biological plausibility, utilizing 0-1 activation sparsity through spike-driven computation. Wh…
Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing
Yudong Liu, Jingwei Sun, Yueqian Lin +6
Vision language models (VLMs) demonstrate strong capabilities in jointly processing visual and textual data. However, they often incur substantial computational overhead due to red…
Proactive Privacy Amnesia for Large Language Models: Safeguarding PII with Negligible Impact on Model Utility
Martin Kuo, Jingyang Zhang, Jianyi Zhang +9
With the rise of large language models (LLMs), increasing research has recognized their risk of leaking personally identifiable information (PII) under malicious attacks. Although…
SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval
Yueqian Lin, Yuzhe Fu, Jingyang Zhang +5
We introduce Speech Information Retrieval (SIR), a new long-context task for Speech Large Language Models (Speech LLMs), and present SPIRAL, a 1,012-sample benchmark testing models…
LoBAM: LoRA-Based Backdoor Attack on Model Merging
Ming Yin, Jingyang Zhang, Jingwei Sun +3
Model merging is an emerging technique that integrates multiple models fine-tuned on different tasks to create a versatile model that excels in multiple domains. This scheme, in th…
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
Cong Guo, Feng Cheng, Zhixu Du +21
The rapid development of large language models (LLMs) has significantly transformed the field of artificial intelligence, demonstrating remarkable capabilities in natural language…
FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learning
Minxue Tang, Yitu Wang, Jingyang Zhang +5
Federated Adversarial Training (FAT) can supplement robustness against adversarial examples to Federated Learning (FL), promoting a meaningful step toward trustworthy AI. However,…
Criticality Leveraged Adversarial Training (CLAT) for Boosted Performance via Parameter Efficiency
Bhavna Gopal, Huanrui Yang, Jingyang Zhang +2
Adversarial training enhances neural network robustness but suffers from a tendency to overfit and increased generalization errors on clean data. This work introduces CLAT, an inno…
Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models
Jingyang Zhang, Jingwei Sun, Eric Yeats +5
The problem of pre-training data detection for large language models (LLMs) has received growing attention due to its implications in critical issues like copyright violation and t…