Publications (230)
Communication-Optimal Distributed Dynamic Graph Clustering
Chun Jiang Zhu, Tan Zhu, Kam-Yiu Lam +2
We consider the problem of clustering graph nodes over large-scale dynamic graphs, such as citation networks, images and web networks, when graph updates such as node/edge insertio…
Optimizing Mixture of Block Attention
Guangxuan Xiao, Junxian Guo, Kasra Mazaheri +1
Mixture of Block Attention (MoBA) (Lu et al., 2025) is a promising building block for efficiently processing long contexts in LLMs by enabling queries to sparsely attend to a small…
SANA-Streaming: Real-time Streaming Video Editing with Hybrid Diffusion Transformer
Yuyang Zhao, Yicheng Pan, Qiyuan He +6
Real-time streaming video-to-video editing (V2V) is critical for interactive applications such as live broadcasting and gaming, yet it remains a formidable challenge due to the str…
QuantumNAT: Quantum Noise-Aware Training with Noise Injection, Quantization and Normalization
Hanrui Wang, Jiaqi Gu, Yongshan Ding +4
Parameterized Quantum Circuits (PQC) are promising towards quantum advantage on near-term quantum hardware. However, due to the large quantum noises (errors), the performance of PQ…
Artificial Intelligence-Based Multiscale Temporal Modeling for Anomaly Detection in Cloud Services
Lian Lian, Yilin Li, Song Han +3
This study proposes an anomaly detection method based on the Transformer architecture with integrated multiscale feature perception, aiming to address the limitations of temporal m…
SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer
Haoyi Zhu, Haozhe Liu, Yuyang Zhao +6
We introduce SANA-WM, an efficient 2.6B-parameter open-source world model natively trained for one-minute generation, synthesizing high-fidelity, 720p, minute-scale videos with pre…
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
Yujun Lin, Haotian Tang, Shang Yang +4
Quantization can accelerate large language model (LLM) inference. Going beyond INT8 quantization, the research community is actively exploring even lower precision, such as INT4. N…
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Guangxuan Xiao, Jiaming Tang, Jingwei Zuo +5
Deploying long-context large language models (LLMs) is essential but poses significant computational and memory challenges. Caching all Key and Value (KV) states across all attenti…
Path-Level Network Transformation for Efficient Architecture Search
Han Cai, Jiacheng Yang, Weinan Zhang +2
We introduce a new function-preserving transformation for efficient neural architecture search. This network transformation allows reusing previously trained networks and existing…
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
Han Cai, Ligeng Zhu, Song Han
Neural architecture search (NAS) has a great impact by automatically designing effective neural network architectures. However, the prohibitive computational demand of conventional…
Transitive Array: An Efficient GEMM Accelerator with Result Reuse
Cong Guo, Chiyue Wei, Jiaming Tang +4
Deep Neural Networks (DNNs) and Large Language Models (LLMs) have revolutionized artificial intelligence, yet their deployment faces significant memory and computational challenges…
SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity
Samir Khaki, Xiuyu Li, Junxian Guo +7
Fine-tuning LLMs is both computationally and memory-intensive. While parameter-efficient fine-tuning methods, such as QLoRA and DoRA, reduce the number of trainable parameters and…
Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models
Muyang Li, Ji Lin, Chenlin Meng +3
During image editing, existing deep generative models tend to re-synthesize the entire output from scratch, including the unedited regions. This leads to a significant waste of com…
Learning both Weights and Connections for Efficient Neural Networks
Song Han, Jeff Pool, John Tran +1
Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems. Also, conventional networks fix the architecture befor…
3D Aware Region Prompted Vision Language Model
An-Chieh Cheng, Yang Fu, Yukang Chen +10
We present Spatial Region 3D (SR-3D) aware vision-language model that connects single-view 2D images and multi-view 3D data through a shared visual token space. SR-3D supports flex…
Learning to Design Circuits
Hanrui Wang, Jiacheng Yang, Hae-Seung Lee +1
Analog IC design relies on human experts to search for parameters that satisfy circuit specifications with their experience and intuitions, which is highly labor intensive, time co…
Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
Luke J. Huang, Zhuoyang Zhang, Qinghao Hu +2
Asynchronous reinforcement learning has become increasingly central to scaling LLM post-training, delivering major throughput gains by decoupling rollout generation from policy upd…
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
Yihui He, Ji Lin, Zhijian Liu +3
Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets. Convention…
Offsite-Tuning: Transfer Learning without Full Model
Guangxuan Xiao, Ji Lin, Song Han
Transfer learning is important for foundation models to adapt to downstream tasks. However, many foundation models are proprietary, so users must share their data with model owners…
VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge
Vishwesh Nath, Wenqi Li, Dong Yang +22
Generalist vision language models (VLMs) have made significant strides in computer vision, but they fall short in specialized fields like healthcare, where expert knowledge is esse…
Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE
Haozhan Tang, Zerui Wang, Yuxian Gu +2
Modern LLMs are increasingly deployed in long-context applications such as retrieval-augmented generation, repository-level coding, and agentic workflows whose accumulated reasonin…
DISQ: Dynamic Iteration Skipping for Variational Quantum Algorithms
Junyao Zhang, Hanrui Wang, Gokul Subramanian Ravi +4
This paper proposes DISQ to craft a stable landscape for VQA training and tackle the noise drift challenge. DISQ adopts a "drift detector" with a reference circuit to identify and…
Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
Jack Cook, Junxian Guo, Guangxuan Xiao +7
As large language models have grown larger, interest has grown in low-precision numerical formats such as NVFP4 as a way to improve speed and reduce memory usage. However, quantizi…
Towards Approximate Model Checking DC and PDC Specifications
Changil Choe, Dang Van Hung, Song Han
DC has proved to be a promising tool for the specification and verification of functional requirements on the design of hard real-time systems. Many works were devoted to develop e…
Network Augmentation for Tiny Deep Learning
Han Cai, Chuang Gan, Ji Lin +1
We introduce Network Augmentation (NetAug), a new training method for improving the performance of tiny neural networks. Existing regularization techniques (e.g., data augmentation…
Analog electromagnetically induced transparency for circularly polarized wave using three dimensional chiral metamaterials
Hai Lin, Dong Yang, Song Han +2
In this paper, we theoretically and experimentally demonstrate a three dimensional metamaterial that can motivate electromagnetic induced transparency (EIT) by using circular polar…
QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits
Tianlong Chen, Zhenyu Zhang, Hanrui Wang +6
Parameterized Quantum Circuits (PQC) have obtained increasing popularity thanks to their great potential for near-term Noisy Intermediate-Scale Quantum (NISQ) computers. Achieving…
EIE: Efficient Inference Engine on Compressed Deep Neural Network
Song Han, Xingyu Liu, Huizi Mao +4
State-of-the-art deep neural networks (DNNs) have hundreds of millions of connections and are both computationally and memory intensive, making them difficult to deploy on embedded…
Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
Yonggan Fu, Lexington Whalen, Abhinav Garg +23
We introduce Nemotron-Labs-Diffusion, a tri-mode language model (LM) that unifies AR, diffusion, and self-speculation decoding within a single architecture. Trained with a joint AR…
EfficientViT-SAM: Accelerated Segment Anything Model Without Accuracy Loss
Zhuoyang Zhang, Han Cai, Song Han
We present EfficientViT-SAM, a new family of accelerated segment anything models. We retain SAM's lightweight prompt encoder and mask decoder while replacing the heavy image encode…
NAAS: Neural Accelerator Architecture Search
Yujun Lin, Mengtian Yang, Song Han
Data-driven, automatic design space exploration of neural accelerator architecture is desirable for specialization and productivity. Previous frameworks focus on sizing the numeric…
FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models
Bowen Xue, Zihan Min, Xingyang Li +8
Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. Howeve…
Efficient Sparse-Winograd Convolutional Neural Networks
Xingyu Liu, Jeff Pool, Song Han +1
Convolutional Neural Networks (CNNs) are computationally intensive, which limits their application on mobile devices. Their energy is dominated by the number of multiplies needed t…
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Yujun Lin, Song Han, Huizi Mao +2
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-…
Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardware
Keshawn Smith, Zhili Zhang, H M Sabbir Ahmad +5
Deep multi-agent reinforcement learning (MARL) has been demonstrated effectively in simulations for multi-robot problems. For autonomous vehicles, the development of vehicle-to-veh…
RobustAnalog: Fast Variation-Aware Analog Circuit Design Via Multi-task RL
Wei Shi, Hanrui Wang, Jiaqi Gu +4
Analog/mixed-signal circuit design is one of the most complex and time-consuming stages in the whole chip design process. Due to various process, voltage, and temperature (PVT) var…
Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation
Yihan Wang, Muyang Li, Han Cai +2
Pose estimation plays a critical role in human-centered vision applications. However, it is difficult to deploy state-of-the-art HRNet-based pose estimation models on resource-cons…
Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
Yonggan Fu, Lexington Whalen, Zhifan Ye +11
Diffusion language models (dLMs) have emerged as a promising paradigm that enables parallel, non-autoregressive generation, but their learning efficiency lags behind that of autore…
EGM: Efficient Visual Grounding Language Models
Guanqi Zhan, Changye Li, Zhijian Liu +4
Visual grounding is an essential capability of Visual Language Models (VLMs) to understand the real physical world. Previous state-of-the-art grounding visual language models usual…
LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation
Yukang Chen, Luozhou Wang, Wei Huang +13
We present LongLive-2.0, an NVFP4-based parallel infrastructure throughout the full training and inference workflow of long video generation, addressing speed and memory bottleneck…
TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Device
Ji Lin, Chuang Gan, Kuan Wang +1
The explosive growth in video streaming requires video understanding at high accuracy and low computation cost. Conventional 2D CNNs are computationally cheap but cannot capture te…
SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
Muyang Li, Yujun Lin, Zhekai Zhang +7
Diffusion models can effectively generate high-quality images. However, as they scale, rising memory demands and higher latency pose substantial deployment challenges. In this work…
Fast inference of deep neural networks in FPGAs for particle physics
Javier Duarte, Song Han, Philip Harris +8
Recent results at the Large Hadron Collider (LHC) have pointed to enhanced physics capabilities through the improvement of the real-time event processing techniques. Machine learni…
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
Yitong Li, Junsong Chen, Haopeng Li +6
Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost. Although many acceleration methods have been proposed, a ce…
Topological Valley-Reshaped Device: Bifunctional Waveguiding and Single-Beam Leaky-Wave Radiation for Terahertz Communication
Yulun Wu, Ziwei Wang, Faqian Chong +6
Topological photonics has emerged as a powerful platform for terahertz on-chip systems due to its robust waveguiding capabilities. However, directly extracting topological valley-l…
SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation
Junsong Chen, Shuchen Xue, Yuyang Zhao +6
This paper presents SANA-Sprint, an efficient diffusion model for ultra-fast text-to-image (T2I) generation. SANA-Sprint is built on a pre-trained foundation model and augmented wi…
A Fast Algorithm for Source-wise Round-trip Spanners
Chun Jiang Zhu, Song Han, Kam-Yiu Lam
In this paper, we study the problem of fast constructions of source-wise round-trip spanners in weighted directed graphs. For a source vertex set in a graph …
VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles
Alexander Amini, Tsun-Hsuan Wang, Igor Gilitschenski +5
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios. However, the poor photorealism and lack of d…
Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos
Ji Lin, Chuang Gan, Song Han
Deep video recognition is more computationally expensive than image recognition, especially on large-scale datasets like Kinetics [1]. Therefore, training scalability is essential…
Hide to Guide: Learning via Semantic Masking
Ruitao Liu, Qinghao Hu, Alex Hu +6
Reinforcement learning with verifiable rewards (RLVR) has become a powerful paradigm for improving language models on reasoning-intensive tasks, but its effectiveness is often limi…
Q-Pilot: Field Programmable Qubit Array Compilation with Flying Ancillas
Hanrui Wang, Daniel Bochen Tan, Pengyu Liu +4
Neutral atom arrays have become a promising platform for quantum computing, especially the field programmable qubit array (FPQA) endowed with the unique capability of atom movement…
Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow
Haocheng Xi, Charlie Ruan, Peiyuan Liao +7
Reinforcement learning (RL) is essential for enhancing the complex reasoning capabilities of large language models (LLMs). However, existing RL training pipelines are computational…
Radial Attention: Sparse Attention with Energy Decay for Long Video Generation
Xingyang Li, Muyang Li, Tianle Cai +11
Recent advances in diffusion models have enabled high-quality video generation, but the additional temporal dimension significantly increases computational costs, making training a…
SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers
Enze Xie, Junsong Chen, Junyu Chen +8
We introduce Sana, a text-to-image framework that can efficiently generate images up to 40964096 resolution. Sana can synthesize high-resolution, high-quality images with s…
ChainSplitter: Towards Blockchain-based Industrial IoT Architecture for Supporting Hierarchical Storage
Gang Wang, Zhijie Jerry Shi, Mark Nixon +1
The fast developing Industrial Internet of Things (IIoT) technologies provide a promising opportunity to build large-scale systems to connect numerous heterogeneous devices into th…
WorldModelBench: Judging Video Generation Models As World Models
Dacheng Li, Yunhao Fang, Yukang Chen +10
Video generation models have rapidly progressed, positioning themselves as video world models capable of supporting decision-making applications like robotics and autonomous drivin…
MCUNet: Tiny Deep Learning on IoT Devices
Ji Lin, Wei-Ming Chen, Yujun Lin +3
Machine learning on tiny IoT devices based on microcontroller units (MCU) is appealing but challenging: the memory of microcontrollers is 2-3 orders of magnitude smaller even than…
Lightning OPD 2.0: Mitigating Style Bias in Cross-Teacher On-Policy Distillation for Large Reasoning Models
Yecheng Wu, Song Han, Han Cai
The paper proposes Lightning OPD 2.0, a method that reduces style‑related bias when using on‑policy distillation across different teacher models, improving performance on mathemati…
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
Huizi Mao, Song Han, Jeff Pool +4
Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accel…
Supporting Deterministic Traffic on Standard NICs
Chuanyu Xue, Tianyu Zhang, Andrew Loveless +1
Networked mission-critical applications (e.g., avionic control and industrial automation systems) require deterministic packet transmissions to support a range of sensing and contr…
X-VILA: Cross-Modality Alignment for Large Language Model
Hanrong Ye, De-An Huang, Yao Lu +8
We introduce X-VILA, an omni-modality model designed to extend the capabilities of large language models (LLMs) by incorporating image, video, and audio modalities. By aligning mod…
Metamaterial analogue of Ising model
Longqing Cong, Vassili Savinov, Yogesh Kumar Srivastava +2
The interaction between microscopic particles has always been a fascinating and intriguing area of science. Direct interrogation of such interactions is often difficult or impossib…
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation
Yuchao Gu, Guian Fang, Yuxin Jiang +4
Few-step video generation has been significantly advanced by consistency distillation. However, the performance of consistency-distilled models often degrades as more sampling step…
PatchNet -- Short-range Template Matching for Efficient Video Processing
Huizi Mao, Sibo Zhu, Song Han +1
Object recognition is a fundamental problem in many video processing tasks, accurately locating seen objects at low computation cost paves the way for on-device video recognition.…
Defensive Quantization: When Efficiency Meets Robustness
Ji Lin, Chuang Gan, Song Han
Neural network quantization is becoming an industry standard to efficiently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs. However, we observe…
Scaling Vision Pre-Training to 4K Resolution
Baifeng Shi, Boyi Li, Han Cai +8
High-resolution perception of visual details is crucial for daily tasks. Current vision pre-training, however, is still limited to low resolutions (e.g., 378 x 378 pixels) due to t…
Sparse Refinement for Efficient High-Resolution Semantic Segmentation
Zhijian Liu, Zhuoyang Zhang, Samir Khaki +5
Semantic segmentation empowers numerous real-world applications, such as autonomous driving and augmented/mixed reality. These applications often operate on high-resolution images…
Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving
Kewei Zhang, Jin Wang, Sensen Gao +9
End-to-end autonomous driving via Vision-Language-Action (VLA) models demands a precarious balance between high-fidelity trajectory planning and efficient inference. Existing parad…
JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search
Dongyun Zou, Zhuoyang Zhang, Junyu Chen +8
We introduce JetViT, a novel family of hybrid-architecture Vision Transformer (ViT) models that match the accuracy of state-of-the-art full-attention vision foundation models while…
Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM
Chengyue Wu, Shiyi Lan, Yonggan Fu +9
Vision-language models (VLMs) predominantly rely on autoregressive decoding, which generates tokens one at a time and fundamentally limits inference throughput. This limitation is…
FP4 Explore, BF16 Train: Diffusion Reinforcement Learning via Efficient Rollout Scaling
Yitong Li, Junsong Chen, Shuchen Xue +8
Reinforcement-Learning-based post-training has recently emerged as a promising paradigm for aligning text-to-image diffusion models with human preferences. In recent studies, incre…
TopGen: Topology-Aware Bottom-Up Generator for Variational Quantum Circuits
Jinglei Cheng, Hanrui Wang, Zhiding Liang +3
Variational Quantum Algorithms (VQA) are promising to demonstrate quantum advantages on near-term devices. Designing ansatz, a variational circuit with parameterized gates, is of p…
FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
Guangxuan Xiao, Tianwei Yin, William T. Freeman +2
Diffusion models excel at text-to-image generation, especially in subject-driven generation for personalized images. However, existing methods are inefficient due to the subject-sp…
StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation
Tianrui Feng, Zhi Li, Shuo Yang +11
Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered. Previous image-based streaming diffusion models have powere…
Efficient and Robust LiDAR-Based End-to-End Navigation
Zhijian Liu, Alexander Amini, Sibo Zhu +3
Deep learning has been used to demonstrate end-to-end neural network learning for autonomous vehicle control from raw sensory input. While LiDAR sensors provide reliably accurate i…
Grounded 3D-Aware Spatial Vision-Language Modeling
An-Chieh Cheng, Yang Fu, Yatai Ji +12
We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding-…
HART: Efficient Visual Generation with Hybrid Autoregressive Transformer
Haotian Tang, Yecheng Wu, Shang Yang +7
We introduce Hybrid Autoregressive Transformer (HART), an autoregressive (AR) visual generation model capable of directly generating 1024x1024 images, rivaling diffusion models in…
Qplacer: Frequency-Aware Component Placement for Superconducting Quantum Computers
Junyao Zhang, Hanrui Wang, Qi Ding +7
Noisy Intermediate-Scale Quantum (NISQ) computers face a critical limitation in qubit numbers, hindering their progression towards large-scale and fault-tolerant quantum computing.…
Condition-Aware Neural Network for Controlled Image Generation
Han Cai, Muyang Li, Zhuoyang Zhang +3
We present Condition-Aware Neural Network (CAN), a new method for adding control to image generative models. In parallel to prior conditional control methods, CAN controls the imag…
Scaling RL to Long Videos
Yukang Chen, Wei Huang, Baifeng Shi +11
We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of…
EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction
Han Cai, Junyan Li, Muyan Hu +2
High-resolution dense prediction enables many appealing real-world applications, such as computational photography, autonomous driving, etc. However, the vast computational cost ma…
Transformer-QEC: Quantum Error Correction Code Decoding with Transferable Transformers
Hanrui Wang, Pengyu Liu, Kevin Shao +5
Quantum computing has the potential to solve problems that are intractable for classical systems, yet the high error rates in contemporary quantum devices often exceed tolerable li…
Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
Morteza Mardani, Enhao Gong, Joseph Y. Cheng +8
Magnetic resonance image (MRI) reconstruction is a severely ill-posed linear inverse task demanding time and resource intensive computations that can substantially trade off {\it a…
Point-Voxel CNN for Efficient 3D Deep Learning
Zhijian Liu, Haotian Tang, Yujun Lin +1
We present Point-Voxel CNN (PVCNN) for efficient, fast 3D deep learning. Previous work processes 3D data using either voxel-based or point-based NN models. However, both approaches…
Time-Sensitive Networking (TSN) for Industrial Automation: Current Advances and Future Directions
Tianyu Zhang, Gang Wang, Chuanyu Xue +3
With the introduction of Cyber-Physical Systems (CPS) and Internet of Things (IoT) technologies, the automation industry is undergoing significant changes, particularly in improvin…
Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models
NVIDIA, :, Aaron Blakeman +198
As inference-time scaling becomes critical for enhanced reasoning capabilities, it is increasingly becoming important to build models that are efficient to infer. We introduce Nemo…
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer
Junsong Chen, Yuyang Zhao, Jincheng Yu +17
We introduce SANA-Video, a small diffusion model that can efficiently generate videos up to 720x1280 resolution and minute-length duration. SANA-Video synthesizes high-resolution,…
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Han Cai, Ji Lin, Yujun Lin +5
Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech reco…
Twilight: Adaptive Attention Sparsity with Hierarchical Top- Pruning
Chaofan Lin, Jiaming Tang, Shuo Yang +6
Leveraging attention sparsity to accelerate long-context large language models (LLMs) has been a hot research topic. However, current algorithms such as sparse attention or key-val…
QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits
Hanrui Wang, Yongshan Ding, Jiaqi Gu +5
Quantum noise is the key challenge in Noisy Intermediate-Scale Quantum (NISQ) computers. Previous work for mitigating noise has primarily focused on gate-level or pulse-level noise…
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
Hanrui Wang, Zhanghao Wu, Zhijian Liu +4
Transformers are ubiquitous in Natural Language Processing (NLP) tasks, but they are difficult to be deployed on hardware due to the intensive computation. To enable low-latency in…
Hardware-Centric AutoML for Mixed-Precision Quantization
Kuan Wang, Zhijian Liu, Yujun Lin +2
Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference. Emergent DNN hardware accelerators begin to support mixed precision (1…
HAQ: Hardware-Aware Automated Quantization with Mixed Precision
Kuan Wang, Zhijian Liu, Yujun Lin +2
Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference. Emergent DNN hardware accelerators begin to support mixed precision (1…
Classification of Neurological Gait Disorders Using Multi-task Feature Learning
Ioannis Papavasileiou, Wenlong Zhang, Xin Wang +3
As our population ages, neurological impairments and degeneration of the musculoskeletal system yield gait abnormalities, which can significantly reduce quality of life. Gait rehab…
OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM
Hanrong Ye, Chao-Han Huck Yang, Arushi Goel +29
Advancing machine intelligence requires developing the ability to perceive across multiple modalities, much as humans sense the world. We introduce OmniVinci, an initiative to buil…
LongLive: Real-time Interactive Long Video Generation
Shuai Yang, Wei Huang, Ruihang Chu +9
We present LongLive, a frame-level autoregressive (AR) framework for real-time and interactive long video generation. Long video generation presents challenges in both efficiency a…
Demonstration of broadband topological slow light
Fujia Chen, Haoran Xue, Yuang Pan +12
Slow-light devices are able to significantly enhance light-matter interaction due to the reduced group velocity of light, but a very low group velocity is usually achieved in a nar…
GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning
Hanrui Wang, Kuan Wang, Jiacheng Yang +4
Automatic transistor sizing is a challenging problem in circuit design due to the large design space, complex performance trade-offs, and fast technological advancements. Although…
Inverse-Designed On-Chip Terahertz Three-Channel Mode and Wavelength Division Demultiplexer
Faqian Chong, Yulun Wu, Bingtao Gao +3
High-performance multimode/multiwavelength (de)multiplexer is one of the most pivotal photonic devices for advanced on-chip interconnect systems. Traditional on-chip photonic (de)m…
QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs
Wei Huang, Yi Ge, Shuai Yang +11
We propose QeRL, a Quantization-enhanced Reinforcement Learning framework for large language models (LLMs). While RL is essential for LLMs' reasoning capabilities, it is resource-i…