481 citations · 1.1k across the 27 of their papers we have counts for
30 papers
The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization
Chao Li, Shigeng Wang, Anbang Yao
Post-training quantization (PTQ) methods typically use sequential quantization that partitions a pre-trained LLM into a series of units (e.g., transformer blocks), with one unit qu…
Kinematic MeanFlow: One-Step Action Generation Policy for Robotic Foundation Models
Jiawei Fan, Sifeng Wang, Yuqing Hou +1
In this paper, we study how to achieve one-step action generation in Robotic Foundation Models (RFMs), aiming to overcome the high inference latency of multi-step flow matching. Me…
Small Scale Data-Free Knowledge Distillation
He Liu, Yikai Wang, Huaping Liu +2
Data-free knowledge distillation is able to utilize the knowledge learned by a large teacher network to augment the training of a smaller student network without accessing the orig…
KernelWarehouse: Towards Parameter-Efficient Dynamic Convolution
Chao Li, Anbang Yao
Dynamic convolution learns a linear mixture of static kernels weighted with their sample-dependent attentions, demonstrating superior performance compared to normal convolution…
Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition
Dongqi Cai, Yangyuxuan Kang, Anbang Yao +1
This paper presents Ske2Grid, a new representation learning framework for improved skeleton-based action recognition. In Ske2Grid, we define a regular convolution operation upon a…
NORM: Knowledge Distillation via N-to-One Representation Matching
Xiaolong Liu, Lujun Li, Chao Li +1
Existing feature distillation methods commonly adopt the One-to-one Representation Matching between any pre-selected teacher-student layer pair. In this paper, we present N-to-One…
Compacting Binary Neural Networks by Sparse Kernel Selection
Yikai Wang, Wenbing Huang, Yinpeng Dong +2
Binary Neural Network (BNN) represents convolution weights with 1-bit values, which enhances the efficiency of storage and computation. This paper is motivated by a previously reve…
3D Human Pose Lifting with Grid Convolution
Yangyuxuan Kang, Yuyang Liu, Anbang Yao +2
Existing lifting networks for regressing 3D human poses from 2D single-view poses are typically constructed with linear layers based on graph-structured representation learning. In…
Omni-Dimensional Dynamic Convolution
Chao Li, Aojun Zhou, Anbang Yao
Learning a single static convolutional kernel in each convolutional layer is the common training paradigm of modern Convolutional Neural Networks (CNNs). Instead, recent research i…
Efficient Meta-Tuning for Content-aware Neural Video Delivery
Xiaoqi Li, Jiaming Liu, Shizun Wang +6
Recently, Deep Neural Networks (DNNs) are utilized to reduce the bandwidth and improve the quality of Internet video delivery. Existing methods train corresponding content-aware su…
Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks
Yikai Wang, Yi Yang, Fuchun Sun +1
In the low-bit quantization field, training Binary Neural Networks (BNNs) is the extreme solution to ease the deployment of deep models on resource-constrained devices, having the…
Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion
Yikai Wang, Fuchun Sun, Ming Lu +1
We propose a compact and effective framework to fuse multimodal features at multiple layers in a single network. The framework consists of two innovative fusion schemes. Firstly, u…
AccSS3D: Accelerator for Spatially Sparse 3D DNNs
Om Ji Omer, Prashant Laddha, Gurpreet S Kalsi +6
Semantic understanding and completion of real world scenes is a foundational primitive of 3D Visual perception widely used in high-level applications such as robotics, medical imag…
LID 2020: The Learning from Imperfect Data Challenge Results
Yunchao Wei, Shuai Zheng, Ming-Ming Cheng +32
Learning from imperfect data becomes an issue in many industrial applications after the research community has made profound progress in supervised learning from perfectly annotate…
Knowledge Transfer via Dense Cross-Layer Mutual-Distillation
Anbang Yao, Dawei Sun
Knowledge Distillation (KD) based methods adopt the one-way Knowledge Transfer (KT) scheme in which training a lower-capacity student network is guided by a pre-trained high-capaci…
CASNet: Common Attribute Support Network for image instance and panoptic segmentation
Xiaolong Liu, Yuqing Hou, Anbang Yao +2
Instance segmentation and panoptic segmentation is being paid more and more attention in recent years. In comparison with bounding box based object detection and semantic segmentat…
PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer
Duo Li, Anbang Yao, Qifeng Chen
Despite their strong modeling capacities, Convolutional Neural Networks (CNNs) are often scale-sensitive. For enhancing the robustness of CNNs to scale variance, multi-scale featur…
Learning to Learn Parameterized Classification Networks for Scalable Input Images
Duo Li, Anbang Yao, Qifeng Chen
Convolutional Neural Networks (CNNs) do not have a predictable recognition behavior with respect to the input resolution change. This prevents the feasibility of deployment on diff…
Resolution Switchable Networks for Runtime Efficient Image Recognition
Yikai Wang, Fuchun Sun, Duo Li +1
We propose a general method to train a single convolutional neural network which is capable of switching image resolutions at inference. Thus the running speed can be selected to m…
Learning Two-View Correspondences and Geometry Using Order-Aware Network
Jiahui Zhang, Dawei Sun, Zixin Luo +6
Establishing correspondences between two images requires both local and global spatial context. Given putative correspondences of feature points in two views, in this paper, we pro…
HBONet: Harmonious Bottleneck on Two Orthogonal Dimensions
Duo Li, Aojun Zhou, Anbang Yao
MobileNets, a class of top-performing convolutional neural network architectures in terms of accuracy and efficiency trade-off, are increasingly used in many resourceaware vision a…
Efficient Semantic Scene Completion Network with Spatial Group Convolution
Jiahui Zhang, Hao Zhao, Anbang Yao +3
We introduce Spatial Group Convolution (SGC) for accelerating the computation of 3D dense prediction tasks. SGC is orthogonal to group convolution, which works on spatial dimension…
Deeply-supervised Knowledge Synergy
Dawei Sun, Anbang Yao, Aojun Zhou +1
Convolutional Neural Networks (CNNs) have become deeper and more complicated compared with the pioneering AlexNet. However, current prevailing training scheme follows the previous…
A Closed-form Solution to Universal Style Transfer
Ming Lu, Hao Zhao, Anbang Yao +3
Universal style transfer tries to explicitly minimize the losses in feature space, thus it does not require training on any pre-defined styles. It usually uses different layers of…