4 citations · 16 across the 24 of their papers we have counts for
29 papers
HoloCosm: Beyond Single-Metric Benchmark Compression Across Multiple Metrics
Yupei Lin, Zhongzhan Huang, Junxin Li +5
Benchmark compression reduces evaluation cost by selecting a small set of samples. Existing methods typically guide selection using a single metric, although benchmarks often asses…
Are We Testing Too Much? A Survey of Benchmark Compression for Efficient Evaluation of Large Language Models
Guoming Ling, Yupei Lin, Zhongzhan Huang +5
Benchmarks are essential for evaluating and improving large language models (LLMs). Yet many widely used benchmarks are redundant, making evaluation unnecessarily expensive. Benchm…
Rules to Tools: Executable Checks for LLM Agents in Scientific Computing
Jingjie Ning, Guojiang Zhao, Chen Xu +4
Scientific coding agents receive equations, boundary conditions, and output requirements in writing, then must assess the programs they revise. Rules to Tools (R2T) supplies prepar…
Zipbench: Low-Cost Framework for Compressing Comprehensive Benchmarks of Large Language Models
Zhongzhan Huang, Junxin Li, Guoming Ling +3
Comprehensive benchmark suites are essential for improving large language models (LLMs), but many widely used benchmarks are redundant, making evaluation unnecessarily expensive. A…
Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents
Jingjie Ning, Shanshan Zhong, Xiaochuan Li +1
AI research agents combine public information and experimental feedback to produce measurable results. The Discovery Certification Protocol (DCP) turns an outcome claim into an exe…
The Implementation Lottery: Auditing Idea Reliability in Automated Research
Jingjie Ning, Shanshan Zhong, Xiaochuan Li +2
Automated research agents use program scores to judge ideas. We call variation in this evidence across implementations the implementation lottery. We introduce an Idea Reliability…
Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer
Jingjie Ning, Xiaochuan Li, Shanshan Zhong +2
Auto Research uses language-model agents to propose, implement, and evaluate machine-learning changes in a closed loop, but is usually judged by its terminal pipeline. A terminal s…
AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Web
Shanshan Zhong, Kate Shen, Chenyan Xiong
Agentic Web is an emerging paradigm where autonomous agents help users use online information. As the paradigm develops, content providers are also deploying agents to manage their…
Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality
George Ling, Shanshan Zhong, Richard Huang
Agent skills extend large language model (LLM) agents with reusable, program-like modules that define triggering conditions, procedural logic, and tool interactions. As these skill…
Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models
Guoming Ling, Zhongzhan Huang, Yupei Lin +4
Chain-of-Thought reasoning has significantly enhanced the problem-solving capabilities of Large Language Models. Unfortunately, current models generate reasoning steps sequentially…
What Generative Search Engines Like and How to Optimize Web Content Cooperatively
Yujiang Wu, Shanshan Zhong, Yubin Kim +1
By employing large language models (LLMs) to retrieve documents and generate natural language responses, Generative Engines, such as Google AI overview and ChatGPT, provide signifi…
Multimodal Representation-disentangled Information Bottleneck for Multimodal Recommendation
Hui Wang, Jinghui Qin, Wushao Wen +3
Multimodal data has significantly advanced recommendation systems by integrating diverse information sources to model user preferences and item characteristics. However, these syst…
AssoCiAm: A Benchmark for Evaluating Association Thinking while Circumventing Ambiguity
Yifan Liu, Wenkuan Zhao, Shanshan Zhong +4
Recent advancements in multimodal large language models (MLLMs) have garnered significant attention, offering a promising pathway toward artificial general intelligence (AGI). Amon…
4D-Animal: Freely Reconstructing Animatable 3D Animals from Videos
Shanshan Zhong, Jiawei Peng, Zehan Zheng +6
Existing methods for reconstructing animatable 3D animals from videos typically rely on sparse semantic keypoints to fit parametric models. However, obtaining such keypoints is lab…
MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models
Zhongzhan Huang, Guoming Ling, Shanshan Zhong +2
Long Context Understanding (LCU) is a critical area for exploration in current large language models (LLMs). However, due to the inherently lengthy nature of long-text data, existi…
RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs
Zhongzhan Huang, Guoming Ling, Yupei Lin +4
Routing large language models (LLMs) is a new paradigm that uses a router to recommend the best LLM from a pool of candidates for a given input. In this paper, our comprehensive an…
A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models
Zhongzhan Huang, Shanshan Zhong, Pan Zhou +3
Recently, numerous benchmarks have been developed to evaluate the logical reasoning abilities of large language models (LLMs). However, assessing the equally important creative cap…
MoExtend: Tuning New Experts for Modality and Task Extension
Shanshan Zhong, Shanghua Gao, Zhongzhan Huang +3
Large language models (LLMs) excel in various tasks but are primarily trained on text data, limiting their application scope. Expanding LLM capabilities to include vision-language…
Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima
Shanshan Zhong, Zhongzhan Huang, Daifeng Li +3
Multimodal recommender systems utilize various types of information to model user preferences and item features, helping users discover items aligned with their interests. The inte…
Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation
Shanshan Zhong, Zhongzhan Huang, Shanghua Gao +4
Chain-of-Thought (CoT) guides large language models (LLMs) to reason step-by-step, and can motivate their logical reasoning ability. While effective for logical tasks, CoT is not c…
Understanding Self-attention Mechanism via Dynamical System Perspective
Zhongzhan Huang, Mingfu Liang, Jinghui Qin +2
The self-attention mechanism (SAM) is widely used in various fields of artificial intelligence and has successfully boosted the performance of different models. However, current ex…
LSAS: Lightweight Sub-attention Strategy for Alleviating Attention Bias Problem
Shanshan Zhong, Wushao Wen, Jinghui Qin +2
In computer vision, the performance of deep neural networks (DNNs) is highly related to the feature extraction ability, i.e., the ability to recognize and focus on key pixel region…
SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models
Shanshan Zhong, Zhongzhan Huang, Wushao Wen +2
Diffusion models, which have emerged to become popular text-to-image generation models, can produce high-quality and content-rich images guided by textual prompts. However, there a…
ASR: Attention-alike Structural Re-parameterization
Shanshan Zhong, Zhongzhan Huang, Wushao Wen +2
The structural re-parameterization (SRP) technique is a novel deep learning technique that achieves interconversion between different network architectures through equivalent param…