1.3k citations · 2.4k across the 21 of their papers we have counts for
21 papers
TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
Haotian Tang, Shang Yang, Zhijian Liu +6
Sparse convolution plays a pivotal role in emerging workloads, including point cloud processing in AR/VR, autonomous driving, and graph understanding in recommendation systems. Sin…
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
Yukang Chen, Shengju Qian, Haotian Tang +4
We present LongLoRA, an efficient fine-tuning approach that extends the context sizes of pre-trained large language models (LLMs), with limited computation cost. Typically, trainin…
MapPrior: Bird's-Eye View Map Layout Estimation with Generative Models
Xiyue Zhu, Vlas Zyrianov, Zhijian Liu +1
Despite tremendous advancements in bird's-eye view (BEV) perception, existing models fall short in generating realistic and coherent semantic map layouts, and they fail to account…
SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer
Xuanyao Chen, Zhijian Liu, Haotian Tang +3
High-resolution images enable neural networks to learn richer visual representations. However, this improved performance comes at the cost of growing computational complexity, hind…
FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer
Zhijian Liu, Xinyu Yang, Haotian Tang +2
Transformer, as an alternative to CNN, has been proven effective in many modalities (e.g., texts and images). For 3D point cloud transformers, existing efforts focus primarily on p…
PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution
Zhijian Liu, Haotian Tang, Shengyu Zhao +2
3D neural networks are widely used in real-world applications (e.g., AR/VR headsets, self-driving cars). They are required to be fast and accurate; however, limited hardware resour…
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Han Cai, Ji Lin, Yujun Lin +5
Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech reco…
TorchSparse: Efficient Point Cloud Inference Engine
Haotian Tang, Zhijian Liu, Xiuyu Li +2
Deep learning on point clouds has received increased attention thanks to its wide applications in AR/VR and autonomous driving. These applications require low latency and high accu…
VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles
Alexander Amini, Tsun-Hsuan Wang, Igor Gilitschenski +5
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios. However, the poor photorealism and lack of d…
LocTex: Learning Data-Efficient Visual Representations from Localized Textual Supervision
Zhijian Liu, Simon Stent, Jie Li +2
Computer vision tasks such as object detection and semantic/instance segmentation rely on the painstaking annotation of large training datasets. In this paper, we propose LocTex th…
Efficient and Robust LiDAR-Based End-to-End Navigation
Zhijian Liu, Alexander Amini, Sibo Zhu +3
Deep learning has been used to demonstrate end-to-end neural network learning for autonomous vehicle control from raw sensory input. While LiDAR sensors provide reliably accurate i…
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
Tianzhe Wang, Kuan Wang, Han Cai +3
We present APQ for efficient deep learning inference on resource-constrained hardware. Unlike previous methods that separately search the neural architecture, pruning policy, and q…
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
Hanrui Wang, Zhanghao Wu, Zhijian Liu +4
Transformers are ubiquitous in Natural Language Processing (NLP) tasks, but they are difficult to be deployed on hardware due to the intensive computation. To enable low-latency in…
Lite Transformer with Long-Short Range Attention
Zhanghao Wu, Zhijian Liu, Ji Lin +2
Transformer has become ubiquitous in natural language processing (e.g., machine translation, question answering); however, it requires enormous amount of computations to achieve hi…
Point-Voxel CNN for Efficient 3D Deep Learning
Zhijian Liu, Haotian Tang, Yujun Lin +1
We present Point-Voxel CNN (PVCNN) for efficient, fast 3D deep learning. Previous work processes 3D data using either voxel-based or point-based NN models. However, both approaches…
Deep Leakage from Gradients
Ligeng Zhu, Zhijian Liu, Song Han
Exchanging gradients is a widely used method in modern multi-node machine learning system (e.g., distributed training, collaborative learning). For a long time, people believed tha…
Design Automation for Efficient Deep Learning Computing
Song Han, Han Cai, Ligeng Zhu +4
Efficient deep learning computing requires algorithm and hardware co-design to enable specialization: we usually need to change the algorithm to reduce memory footprint and improve…
Unsupervised Discovery of Parts, Structure, and Dynamics
Zhenjia Xu, Zhijian Liu, Chen Sun +4
Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose…
HAQ: Hardware-Aware Automated Quantization with Mixed Precision
Kuan Wang, Zhijian Liu, Yujun Lin +2
Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference. Emergent DNN hardware accelerators begin to support mixed precision (1…
Physical Primitive Decomposition
Zhijian Liu, William T. Freeman, Joshua B. Tenenbaum +1
Objects are made of parts, each with distinct geometry, physics, functionality, and affordances. Developing such a distributed, physical, interpretable representation of objects wi…
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
Yihui He, Ji Lin, Zhijian Liu +3
Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets. Convention…