15 citations · 69 across the 15 of their papers we have counts for
15 papers
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
Taiqiang Wu, Chaofan Tao, Jiahao Wang +3
Kullback-Leiber divergence has been widely used in Knowledge Distillation (KD) to compress Large Language Models (LLMs). Contrary to prior assertions that reverse Kullback-Leibler…
A Spectral Perspective towards Understanding and Improving Adversarial Robustness
Binxiao Huang, Rui Lin, Chaofan Tao +1
Deep neural networks (DNNs) are incredibly vulnerable to crafted, imperceptible adversarial perturbations. While adversarial training (AT) has proven to be an effective defense app…
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
Dachuan Shi, Chaofan Tao, Anyi Rao +3
Recent vision-language models have achieved tremendous advances. However, their computational costs are also escalating dramatically, making model acceleration exceedingly critical…
DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inference
Jiajun Zhou, Jiajun Wu, Yizhao Gao +7
To accelerate the inference of deep neural networks (DNNs), quantization with low-bitwidth numbers is actively researched. A prominent challenge is to quantize the DNN models into…
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers
Dachuan Shi, Chaofan Tao, Ying Jin +3
Real-world data contains a vast amount of multimodal information, among which vision and language are the two most representative modalities. Moreover, increasingly heavier models,…
Frequency Regularization for Improving Adversarial Robustness
Binxiao Huang, Chaofan Tao, Rui Lin +1
Deep neural networks are incredibly vulnerable to crafted, human-imperceptible adversarial perturbations. Although adversarial training (AT) has proven to be an effective defense a…
LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling
Dongsheng Chen, Chaofan Tao, Lu Hou +3
Recent large-scale video-language pre-trained models have shown appealing performance on various downstream tasks. However, the pre-training process is computationally expensive du…
ODG-Q: Robust Quantization via Online Domain Generalization
Chaofan Tao, Ngai Wong
Quantizing neural networks to low-bitwidth is important for model deployment on resource-limited edge hardware. Although a quantized network has a smaller model size and memory foo…
What Do Adversarially trained Neural Networks Focus: A Fourier Domain-based Study
Binxiao Huang, Chaofan Tao, Rui Lin +1
Although many fields have witnessed the superior performance brought about by deep learning, the robustness of neural networks remains an open issue. Specifically, a small adversar…
Compression of Generative Pre-trained Language Models via Quantization
Chaofan Tao, Lu Hou, Wei Zhang +5
The increasing size of generative Pre-trained Language Models (PLMs) has greatly increased the demand for model compression. Despite various methods to compress BERT or its variant…
Interpretable Mammographic Image Classification using Case-Based Reasoning and Deep Learning
Alina Jade Barnett, Fides Regina Schwartz, Chaofan Tao +4
When we deploy machine learning models in high-stakes medical settings, we must ensure these models make accurate predictions that are consistent with known medical science. Inhere…
IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammography
Alina Jade Barnett, Fides Regina Schwartz, Chaofan Tao +4
Interpretability in machine learning models is important in high-stakes decisions, such as whether to order a biopsy based on a mammographic exam. Mammography poses important chall…
FAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformation
Chaofan Tao, Rui Lin, Quan Chen +3
Learning convolutional neural networks (CNNs) with low bitwidth is challenging because performance may drop significantly after quantization. Prior arts often discretize the networ…
Dynamic and Static Context-aware LSTM for Multi-agent Motion Prediction
Chaofan Tao, Qinhong Jiang, Lixin Duan +1
Multi-agent motion prediction is challenging because it aims to foresee the future trajectories of multiple agents (\textit{e.g.} pedestrians) simultaneously in a complicated scene…
MiniMax Entropy Network: Learning Category-Invariant Features for Domain Adaptation
Chaofan Tao, Fengmao Lv, Lixin Duan +1
How to effectively learn from unlabeled data from the target domain is crucial for domain adaptation, as it helps reduce the large performance gap due to domain shift or distributi…