1.6k citations · 2.7k across the 28 of their papers we have counts for
30 papers
Probabilistic Multi-Source Data Selection via Learnable Categorical Priors: Variance Reduction and Guaranteed Derandomization
Shizhan Cai, Qixin Zhang, Siyuan Ma +3
Bilevel data selection is a fundamental task with numerous applications in machine learning and statistics. A common strategy for tackling this problem is to relax the original bil…
FutureBridge: Token Selection Beyond Local Preference in Collaborative Decoding
Quanquan Li, Hongbo Zhang, Yihe Chi +9
Token-level collaboration allows a large language model (LLM) to assist a small language model (SLM) when their predictions diverge. Existing methods either use LLM-generated inter…
Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
SiYuan Ma, Yiqin Luo, Zhangji +8
Identical language-model answers can arise from hidden states that support different future computations, so current-answer probes do not establish a reusable internal interface. W…
Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning
Chengzhen Yu, Canran Xiao, Siyuan Ma +1
Vision-language alignment powers open-vocabulary recognition, retrieval, and LVLM grounding, yet natural captions are often underspecified, making similarity brittle and overly con…
Attention-Spectrum Regularization for Replay-Free Continual Multimodal LLMs
Chuangxin Zhao, Canran Xiao, Siyuan Ma +5
Multimodal large language models (MLLMs) are increasingly required to adapt to non-stationary streams of visual domains, question types, and user instructions, yet continual fine-t…
Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG
He Sun, Shinan Liu, Siyuan Ma +3
Modern Large Language Model (LLM) serving engines increasingly rely on Retrieval-Augmented Generation (RAG) and non-prefix Key-Value (KV) cache fusion to accelerate long-context, m…
Large Language Models as Amortized Pareto-Front Generators for Constrained Bi-Objective Convex Optimization
Peipei Xu, SiYuan Ma, Yaohua Liu +4
Generating feasible Pareto fronts for constrained bi-objective continuous optimization is central to multi-criteria decision-making. Existing methods usually rely on iterative scal…
BOOKAGENT: Orchestrating Safety-Aware Visual Narratives via Multi-Agent Cognitive Calibration
Bo Gao, Chang Liu, Yuyang Miao +2
Recent advancements in Large Generative Models (LGMs) have revolutionized multi-modal generation. However, generating illustrated storybooks remains an open challenge, where prior…
A comparative study on power delivery aspects of compute-in/near-memory approaches using DRAM
Siddhartha Raman Sundara Raman, Siyuan Ma, Lizy Kurian John
Compute-in-memory (PIM) mitigates the memory wall by performing computation within memory, reducing data movement and improving energy efficiency. DRAM-based PIM is particularly at…
The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook
Xinlei Yu, Zhangquan Chen, Yongbo He +36
Latent space is rapidly emerging as a native substrate for language-based models. While modern systems are still commonly understood through explicit token-level generation, an inc…
CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning
Siyuan Ma, Bo Gao, Zikai Xiao +6
Recent test-time reasoning methods improve performance by generating more candidate chains or searching over larger reasoning trees, but they typically lack explicit control over w…
MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hints
Yu Qi, Xinyi Xu, Ziyu Guo +10
Video generative models show emerging reasoning behaviors. It is essential to ensure that generated events remain causally consistent across frames for reliable deployment, a prope…
ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference
Siyuan Ma, Bo Gao, Xiaojun Jia +6
The paradigm of large language model (LLM) reasoning is shifting from parameter scaling to test-time compute scaling, yet many existing approaches still rely on uniform brute-force…
RACAM: Enhancing DRAM with Reuse-Aware Computation and Automated Mapping for ML Inference
Siyuan Ma, Jiajun Hu, Jeeho Ryoo +2
In-DRAM Processing-In-Memory (DRAM-PIM) has emerged as a promising approach to accelerate memory-intensive workloads by mitigating data transfer overhead between DRAM and the host…
Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis
Yu Qi, Haibo Zhao, Ziyu Guo +17
Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improving embodied agents. However, existing embodied benchmarks mainly…
Your Harness is Not Secure: Benchmarking Real-world Threat of Command Line Interface Agent
Weidi Luo, Qiming Zhang, Tianyu Lu +9
Command-line interface (CLI) agents powered by large language models (LLMs) can interpret natural-language requests, plan multi-step tasks, execute shell commands, and modify files…
Fast training of large kernel models with delayed projections
Amirhesam Abedsoltan, Siyuan Ma, Parthe Pandit +1
Classical kernel machines have historically faced significant challenges in scaling to large datasets and model sizes--a key ingredient that has driven the success of neural networ…
Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset
Yingzi Ma, Jiongxiao Wang, Fei Wang +10
Machine unlearning has emerged as an effective strategy for forgetting specific information in the training data. However, with the increasing integration of visual data, privacy c…
Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character
Siyuan Ma, Weidi Luo, Yu Wang +1
With the advent and widespread deployment of Multimodal Large Language Models (MLLMs), ensuring their safety has become increasingly critical. To achieve this objective, it require…
JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
Weidi Luo, Siyuan Ma, Xiaogeng Liu +2
With the rapid advancements in Multimodal Large Language Models (MLLMs), securing these models against malicious inputs while aligning them with human values has emerged as a criti…
PIMSAB: A Processing-In-Memory System with Spatially-Aware Communication and Bit-Serial-Aware Computation
Aman Arora, Jian Weng, Siyuan Ma +2
Bit-serial Processing-In-Memory (PIM) is an attractive paradigm for accelerator architectures, for parallel workloads such as Deep Learning (DL), because of its capability to achie…
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
Yiheng Liu, Tianle Han, Siyuan Ma +15
This paper presents a comprehensive survey of ChatGPT-related (GPT-3.5 and GPT-4) research, state-of-the-art large language models (LLM) from the GPT series, and their prospective…
Learning the joint distribution of two sequences using little or no paired data
Soroosh Mariooryad, Matt Shannon, Siyuan Ma +5
We present a noisy channel generative model of two sequences, for example text and speech, which enables uncovering the association between the two modalities when limited paired d…
Reconciling modern machine learning practice and the bias-variance trade-off
Mikhail Belkin, Daniel Hsu, Siyuan Ma +1
Breakthroughs in machine learning are rapidly changing science and society, yet our fundamental understanding of this technology has lagged far behind. Indeed, one of the central t…