6 citations · 22 across the 17 of their papers we have counts for
19 papers
DeepOHeat-v2: Self-Improving Operator Learning for Fast and Trustworthy Thermal Optimization in 3D-IC Design
Xinling Yu, Yixing Li, Ziyue Liu +4
Thermal-aware optimization of multi-die 3D integrated circuits evaluates many designs, each a costly heat-equation solve. Operator-learning surrogates replace this solve with a fas…
ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload
Ziyue Liu, Zhengyang Wang, Ruijie Zhang +7
Pre-training large language models on massive GPU clusters has made hardware faults routine rather than rare, driving the need for resilient training systems. Yet existing framewor…
When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Models
Ziyu Liu, Tao Li, Tao Yang +4
Data-poisoning backdoors pose a practical threat to the fine-tuning of large language models (LLMs). Most existing attacks bind an attacker-selected behavior to fixed tokens, phras…
MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
Yupeng Su, Ruijie Zhang, Ziyue Liu +2
The Muon optimizer has emerged as a compelling alternative to Adam for training large language models, achieving remarkable computational savings through gradient orthogonalization…
TEON: Tensorized Orthonormalization Beyond Layer-Wise Muon for Large Language Model Pre-Training
Ruijie Zhang, Yequan Zhao, Ziyue Liu +5
The Muon optimizer has demonstrated strong empirical performance in pre-training large language models by performing matrix-level gradient (or momentum) orthogonalization in each l…
MUON+: Towards More Effective Muon via One Additional Normalization Step for LLM Pre-training
Ruijie Zhang, Yequan Zhao, Ziyue Liu +4
Muon has recently emerged as a strong optimizer for large language model pre-training, orthogonalizing the momentum matrix via Newton--Schulz polar iterations. A natural intuition…
BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models
Zhengyang Wang, Ziyue Liu, Ruijie Zhang +5
The scale of transformer model pre-training is constrained by the increasing computation and communication cost. Low-rank bottleneck architectures offer a promising solution to sig…
LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossing
Ruijie Zhang, Ziyue Liu, Zhengyang Wang +1
Training foundation models such as ViTs and LLMs requires tremendous computing cost. Low-rank matrix or tensor factorization offers a parameter-efficient alternative, but often dow…
DeepOHeat-v1: Efficient Operator Learning for Fast and Trustworthy Thermal Simulation and Optimization in 3D-IC Design
Xinling Yu, Ziyue Liu, Hai Li +5
Thermal analysis is crucial in 3D-IC design due to increased power density and complex heat dissipation paths. Although operator learning frameworks such as DeepOHeat~\cite{liu2023…
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
Jiajun Zhou, Yifan Yang, Kai Zhen +6
Language Models (LLMs) are often quantized to lower precision to reduce the memory cost and latency in inference. However, quantization often degrades model performance, thus fine-…
Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks
Yequan Zhao, Xinling Yu, Xian Xiao +6
Physics intelligence and digital twins often require rapid and repeated performance evaluation of various engineering systems (e.g. robots, autonomous vehicles, semiconductor chips…
CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
Ziyue Liu, Ruijie Zhang, Zhengyang Wang +7
The full-size MLPs and the projection layers in attention introduce tremendous model sizes of large language models (LLMs), consuming extensive computational resources in pre-train…
Separable Operator Networks
Xinling Yu, Sean Hooten, Ziyue Liu +4
Operator learning has become a powerful tool in machine learning for modeling complex physical systems governed by partial differential equations (PDEs). Although Deep Operator Net…
Real-Time FJ/MAC PDE Solvers via Tensorized, Back-Propagation-Free Optical PINN Training
Yequan Zhao, Xian Xiao, Xinling Yu +5
Solving partial differential equations (PDEs) numerically often requires huge computing time, energy cost, and hardware resources in practical applications. This has limited their…
Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks
Yequan Zhao, Xinling Yu, Zhixiong Chen +3
Backward propagation (BP) is widely used to compute the gradients in neural network training. However, it is hard to implement BP on edge devices due to the lack of hardware and so…
DeepOHeat: Operator Learning-based Ultra-fast Thermal Simulation in 3D-IC Design
Ziyue Liu, Yixing Li, Jing Hu +5
Thermal issue is a major concern in 3D integrated circuit (IC) design. Thermal optimization of 3D IC often requires massive expensive PDE simulations. Neural network-based thermal…
PIFON-EPT: MR-Based Electrical Property Tomography Using Physics-Informed Fourier Networks
Xinling Yu, José E. C. Serrallés, Ilias I. Giannakopoulos +4
We propose Physics-Informed Fourier Networks for Electrical Properties (EP) Tomography (PIFON-EPT), a novel deep learning-based method for EP reconstruction using noisy and/or inco…
MR-Based Electrical Property Reconstruction Using Physics-Informed Neural Networks
Xinling Yu, José E. C. Serrallés, Ilias I. Giannakopoulos +4
Electrical properties (EP), namely permittivity and electric conductivity, dictate the interactions between electromagnetic waves and biological tissue. EP can be potential biomark…
TT-PINN: A Tensor-Compressed Neural PDE Solver for Edge Computing
Ziyue Liu, Xinling Yu, Zheng Zhang
Physics-informed neural networks (PINNs) have been increasingly employed due to their capability of modeling complex physics systems. To achieve better expressiveness, increasingly…