129 citations · 494 across the 76 of their papers we have counts for
78 papers
SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence
Yao Zhang, Chenyang Lin, Shijie Tang +4
The rapid progress of Large Language Models has advanced agentic systems in decision-making, coordination, and task execution. Yet, existing agentic system generation frameworks la…
FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
Yao Zhang, Hewei Gao, Haokun Chen +3
Multimodal Large Language Models (MLLMs) excel in tasks like multimodal reasoning and cross-modal retrieval but face deployment challenges in real-world scenarios due to distribute…
Self-Evolving Multi-Agent Systems via Textual Backpropagation
Xiaowen Ma, Yunpu Ma, Chenyang Lin +6
Leveraging multiple Large Language Models (LLMs) has proven effective for addressing complex, high-dimensional tasks, but current approaches often rely on static, manually engineer…
METok: Multi-Stage Event-based Token Compression for Efficient Long Video Understanding
Mengyue Wang, Shuo Chen, Kristian Kersting +2
Recent advances in Video Large Language Models (VLLMs) have significantly enhanced their ability to understand video content. Nonetheless, processing long videos remains challengin…
CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Process
Jinhe Bi, Danqi Yan, Yifan Wang +8
Recent Large Reasoning Models significantly improve the reasoning ability of Large Language Models by learning to reason, exhibiting the promising performance in solving complex ta…
PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection
Jinhe Bi, Aniri, Zengjie Jin +11
Visual instruction tuning adapts pre-trained Multimodal Large Language Models (MLLMs) to follow human instructions for real-world applications. However, the rapid growth of these d…
LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering
Jinhe Bi, Yujun Wang, Haokun Chen +4
Multimodal Large Language Models (MLLMs) have significantly advanced visual tasks by integrating visual representations into large language models (LLMs). The textual modality, inh…
WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Exploration
Yao Zhang, Zijian Ma, Yunpu Ma +3
LLM-based autonomous agents often fail to execute complex web tasks that require dynamic interaction due to the inherent uncertainty and complexity of these environments. Existing…
DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models
Zifeng Ding, Yifeng Li, Yuan He +5
Learning useful representations for continuous-time dynamic graphs (CTDGs) is challenging, due to the concurrent need to span long node interaction histories and grasp nuanced temp…
Differentiable Quantum Architecture Search For Job Shop Scheduling Problem
Yize Sun, Jiarui Liu, Yunpu Ma +1
The Job shop scheduling problem (JSSP) plays a pivotal role in industrial applications, such as signal processing (SP) and steel manufacturing, involving sequencing machines and jo…
Quantum Architecture Search with Unsupervised Representation Learning
Yize Sun, Zixin Wu, Volker Tresp +1
Unsupervised representation learning presents new opportunities for advancing Quantum Architecture Search (QAS) on Noisy Intermediate-Scale Quantum (NISQ) devices. QAS is designed…
SPOT! Revisiting Video-Language Models for Event Understanding
Gengyuan Zhang, Jinhe Bi, Jindong Gu +2
Understanding videos is an important research topic for multimodal learning. Leveraging large-scale datasets of web-crawled video-text pairs as weak supervision has become a pre-tr…
zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
Zifeng Ding, Heling Cai, Jingpei Wu +4
Modeling evolving knowledge over temporal knowledge graphs (TKGs) has become a heated topic. Various methods have been proposed to forecast links on TKGs. Most of them are embeddin…
Explaining Deep Neural Networks for Bearing Fault Detection with Vibration Concepts
Thomas Decker, Michael Lebacher, Volker Tresp
Concept-based explanation methods, such as Concept Activation Vectors, are potent means to quantify how abstract or high-level characteristics of input data influence the predictio…
GraphextQA: A Benchmark for Evaluating Graph-Enhanced Large Language Models
Yuanchun Shen, Ruotong Liao, Zhen Han +2
While multi-modal models have successfully integrated information from image, video, and audio modalities, integrating graph modality into large language models (LLMs) remains unex…
GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models
Ruotong Liao, Xu Jia, Yangzhe Li +2
The rapid advancements in large language models (LLMs) have ignited interest in the temporal knowledge graph (tKG) domain, where conventional embedding-based and rule-based methods…
Adversarial Attacks on Tables with Entity Swap
Aneta Koleva, Martin Ringsquandl, Volker Tresp
The capabilities of large language models (LLMs) have been successfully applied in the context of table representation learning. The recently proposed tabular language models have…
FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning
Haokun Chen, Yao Zhang, Denis Krompass +2
Recently, foundation models have exhibited remarkable advancements in multi-modal learning. These models, equipped with millions (or billions) of parameters, typically require a su…
Multi-event Video-Text Retrieval
Gengyuan Zhang, Jisen Ren, Jindong Gu +1
Video-Text Retrieval (VTR) is a crucial multi-modal task in an era of massive video-text data on the Internet. A plethora of work characterized by using a two-stream Vision-Languag…
FedPop: Federated Population-based Hyperparameter Tuning
Haokun Chen, Denis Krompass, Jindong Gu +1
Federated Learning (FL) is a distributed machine learning (ML) paradigm, in which multiple clients collaboratively train ML models without centralizing their local data. Similar to…
Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs
Zifeng Ding, Jingcheng Wu, Jingpei Wu +2
Stemming from traditional knowledge graphs (KGs), hyper-relational KGs (HKGs) provide additional key-value pairs (i.e., qualifiers) for each KG fact that help to better restrict th…
Can Vision-Language Models be a Good Guesser? Exploring VLMs for Times and Location Reasoning
Gengyuan Zhang, Yurui Zhang, Kerui Zhang +1
Vision-Language Models (VLMs) are expected to be capable of reasoning with commonsense knowledge as human beings. One example is that humans can reason where and when an image is t…
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models
Shuo Chen, Jindong Gu, Zhen Han +3
Various adaptation methods, such as LoRA, prompts, and adapters, have been proposed to enhance the performance of pre-trained vision-language models in specific domains. The robust…
GRAtt-VIS: Gated Residual Attention for Auto Rectifying Video Instance Segmentation
Tanveer Hannan, Rajat Koner, Maximilian Bernhard +5
Recent trends in Video Instance Segmentation (VIS) have seen a growing reliance on online methods to model complex and lengthy video sequences. However, the degradation of represen…