9 citations · 17 across the 11 of their papers we have counts for
12 papers
MemDLM: Memory-Enhanced DLM Training
Zehua Pei, Hui-Ling Zhen, Weizhe Lin +4
Diffusion Language Models (DLMs) offer attractive advantages over Auto-Regressive (AR) models, such as full-attention parallel decoding and flexible generation. However, standard D…
MOSS: Efficient and Accurate FP8 LLM Training with Microscaling and Automatic Scaling
Yu Zhang, Hui-Ling Zhen, Mingxuan Yuan +1
Training large language models with FP8 formats offers significant efficiency gains. However, the reduced numerical precision of FP8 poses challenges for stable and accurate traini…
SCOPE: Prompt Evolution for Enhancing Agent Effectiveness
Zehua Pei, Hui-Ling Zhen, Shixiong Kai +4
Large Language Model (LLM) agents are increasingly deployed in environments that generate massive, dynamic contexts. However, a critical bottleneck remains: while agents have acces…
Behavioral Fingerprinting of Large Language Models
Zehua Pei, Hui-Ling Zhen, Ying Zhang +5
Current benchmarks for Large Language Models (LLMs) primarily focus on performance metrics, often failing to capture the nuanced behavioral characteristics that differentiate them.…
PreMoE: Proactive Inference for Efficient Mixture-of-Experts
Zehua Pei, Ying Zhang, Hui-Ling Zhen +6
Mixture-of-Experts (MoE) models offer dynamic computation, but are typically deployed as static full-capacity models, missing opportunities for deployment-specific specialization.…
Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
Zehua Pei, Hui-Ling Zhen, Lancheng Zou +5
Scaling large language models (LLMs) improves performance but significantly increases inference costs, with feed-forward networks (FFNs) consuming the majority of computational res…
MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
Yu Zhang, Mingzi Wang, Lancheng Zou +4
Transformer-based large language models (LLMs) have achieved remarkable success as model sizes continue to grow, yet their deployment remains challenging due to significant computa…
From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
Zehua Pei, Hui-Ling Zhen, Xianzhi Yu +3
Structured pruning of Generative Pre-trained Transformers (GPTs) offers a promising path to efficiency but often suffers from irreversible performance degradation due to the discar…
RTLRewriter: Methodologies for Large Models aided RTL Code Optimization
Xufeng Yao, Yiwen Wang, Xing Li +6
Register Transfer Level (RTL) code optimization is crucial for enhancing the efficiency and performance of digital circuits during early synthesis stages. Currently, optimization r…
HDLdebugger: Streamlining HDL debugging with Large Language Models
Xufeng Yao, Haoyang Li, Tsz Ho Chan +5
In the domain of chip design, Hardware Description Languages (HDLs) play a pivotal role. However, due to the complex syntax of HDLs and the limited availability of online resources…
DiLA: Enhancing LLM Tool Learning with Differential Logic Layer
Yu Zhang, Hui-Ling Zhen, Zehua Pei +4
Considering the challenges faced by large language models (LLMs) in logical reasoning and planning, prior efforts have sought to augment LLMs with access to external solvers. While…
BetterV: Controlled Verilog Generation with Discriminative Guidance
Zehua Pei, Hui-Ling Zhen, Mingxuan Yuan +2
Due to the growing complexity of modern Integrated Circuits (ICs), there is a need for automated circuit design methods. Recent years have seen rising research in hardware design l…