419 citations · 682 across the 54 of their papers we have counts for
55 papers
Top-Down Microarchitecture Analysis Without Causal Modelling Is Dangerous!
Chenxi Wang, Lei Wang, Wanling Gao +3
Top-Down Microarchitecture Analysis (TMA) is the standard for identifying CPU bottlenecks. However, TMA implicitly models performance via a single causal path: Workload(dataset, al…
When AI Designs AI: Innovation or Imitation?
Yikang Yang, Zhengxin Yang, Luzhou Peng +4
Recent advances in LLM agents have made them increasingly capable of designing methods for complex AI tasks. This raises two central questions about agent-designed methods relative…
AutoMatBench: An Automatic Optimization Toolkit for the Acceleration of Material Properties Prediction Benchmarking
Hongxiao Li, Wanling Gao
Material property prediction (MPP) infers key properties from chemical composition and structure, accelerating the discovery and optimization of novel materials. In the realm of MP…
Inference of Component Effect on System Performance
Chenxi Wang, Lei Wang, Wanling Gao +5
In a computer system, multiple components--such as the CPU, memory, and others--work together as a system whose performance can be directly measured. However, the effect of a compo…
CombinationTS: A Modular Framework for Understanding Time-Series Forecasting Models
Xiaorui Wang, Fanda Fan, Chenxi Wang +9
Recent progress in time-series forecasting has led to rapidly increasing architectural complexity, yet many reported State-of-the-Art gains are statistically fragile or misattribut…
Probing Memes in LLMs: A Paradigm for the Entangled Evaluation World
Luzhou Peng, Zhengxin Yang, Honglu Ji +6
Current evaluation paradigms for large language models (LLMs) characterize models and datasets separately, yielding coarse descriptions: items in datasets are treated as pre-labele…
GraDE: A Graph Diffusion Estimator for Frequent Subgraph Discovery in Neural Architectures
Yikang Yang, Zhengxin Yang, Minghao Luo +5
Finding frequently occurring subgraph patterns or network motifs in neural architectures is crucial for optimizing efficiency, accelerating design, and uncovering structural insigh…
On Meta-Evaluation
Hongxiao Li, Chenxi Wang, Fanda Fan +4
Evaluation is the foundation of empirical science, yet the evaluation of evaluation itself -- so-called meta-evaluation -- remains strikingly underdeveloped. While methods such as…
Quantifying the Dynamics of Harm Caused by Retracted Research
Yunyou Huang, Jiahui Zhao, Dandan Cui +10
Despite enormous efforts devoted to understand the characteristics and impacts of retracted papers, little is known about the mechanisms underlying the dynamics of their harm and t…
Achieving Consistent and Comparable CPU Evaluation
Chenxi Wang, Lei Wang, Wanling Gao +5
The challenge of CPU evaluation lies in the fact that user-perceived performance metrics can only be measured on an independently running system consisting of the CPU and other ind…
Could Bibliometrics Reveal Top Science and Technology Achievements and Researchers? The Case for Evaluatology-based Science and Technology Evaluation
Guoxin Kang, Wanling Gao, Lei Wang +5
By utilizing statistical methods to analyze bibliographic data, bibliometrics faces inherent limitations in identifying the most significant science and technology achievements and…
Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models
Wanling Gao, Yunyou Huang, Dandan Cui +22
A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. S…
AI.vs.Clinician: Unveiling Intricate Interactions Between AI and Clinicians through an Open-Access Database
Wanling Gao, Yuan Liu, Zhuoming Yu +21
Artificial Intelligence (AI) plays a crucial role in medical field and has the potential to revolutionize healthcare practices. However, the success of AI models and their impacts…
Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture
Zhengxin Yang, Wanling Gao, Luzhou Peng +3
Designing and optimizing neural network architectures typically requires extensive expertise, starting with handcrafted designs and then manual or automated refinement. This depend…
Bridging the Gap Between Domain-specific Frameworks and Multiple Hardware Devices
Xu Wen, Wanling Gao, Lei Wang +1
The rapid development of domain-specific frameworks has presented us with a significant challenge: The current approach of implementing solutions on a case-by-case basis incurs a t…
Evaluatology: The Science and Engineering of Evaluation
Jianfeng Zhan, Lei Wang, Wanling Gao +10
Evaluation is a crucial aspect of human existence and plays a vital role in various fields. However, it is often approached in an empirical and ad-hoc manner, lacking consensus on…
AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AI
Fanda Fan, Chunjie Luo, Wanling Gao +1
The burgeoning field of Artificial Intelligence Generated Content (AIGC) is witnessing rapid advancements, particularly in video generation. This paper introduces AIGCBench, a pion…
AGIBench: A Multi-granularity, Multimodal, Human-referenced, Auto-scoring Benchmark for Large Language Models
Fei Tang, Wanling Gao, Luzhou Peng +1
Large language models (LLMs) like ChatGPT have revealed amazing intelligence. How to evaluate the question-solving abilities of LLMs and their degrees of intelligence is a hot-spot…
Does AI for science need another ImageNet Or totally different benchmarks? A case study of machine learning force fields
Yatao Li, Wanling Gao, Lei Wang +3
AI for science (AI4S) is an emerging research field that aims to enhance the accuracy and speed of scientific computing tasks using machine learning methods. Traditional AI benchma…
IterLara: A Turing Complete Algebra for Big Data, AI, Scientific Computing, and Database
Hongxiao Li, Wanling Gao, Lei Wang +1
\textsc{Lara} is a key-value algebra that aims at unifying linear and relational algebra with three types of operation abstraction. The study of \textsc{Lara}'s expressive ability…
WPC: Whole-picture Workload Characterization
Lei Wang, Kaiyong Yang, Chenxi Wang +7
This article raises an important and challenging workload characterization issue: can we uncover each critical component across the stacks contributing what percentages to any spec…
DCNetBench: Scaleable Data Center Network Benchmarking
Ke Liu, Wanling Gao, Chunjie Luo +7
Data center networking is the central infrastructure of the modern information society. However, benchmarking them is very challenging as the real-world network traffic is difficul…
CMLCompiler: A Unified Compiler for Classical Machine Learning
Xu Wen, Wanling Gao, Anzheng Li +3
Classical machine learning (CML) occupies nearly half of machine learning pipelines in production applications. Unfortunately, it fails to utilize the state-of-the-practice devices…
ToL: A Tensor of List-Based Unified Computation Model
Hongxiao Li, Wanling Gao, Lei Wang +1
Previous computation models either have equivalent abilities in representing all computations but fail to provide primitive operators for programming complex algorithms or lack gen…