most citedLanguage model compression with weighted low-rank factorization

20 citations · 26 across the 6 of their papers we have counts for

collaborators

6 papers

cs.IR2023

DeepTagger: Knowledge Enhanced Named Entity Recognition for Web-Based Ads Queries

Simiao Zuo, Pengfei Tang, Xinyu Hu +3

Named entity recognition (NER) is a crucial task for online advertisement. State-of-the-art solutions leverage pre-trained language models for this task. However, three major chall…

cs.CL2023★ 1 cited

Dual-Alignment Pre-training for Cross-lingual Sentence Embedding

Ziheng Li, Shaohan Huang, Zihan Zhang +7

Recent studies have shown that dual encoder models trained with the sentence-level translation ranking task are effective methods for cross-lingual sentence embedding. However, our…

cs.CL2023★ 5 cited

TrojText: Test-time Invisible Textual Trojan Insertion

Qian Lou, Yepeng Liu, Bo Feng

In Natural Language Processing (NLP), intelligent neuron models can be susceptible to textual Trojan attacks. Such attacks occur when Trojan models behave normally for standard inp…

cs.CL2022

Numerical Optimizations for Weighted Low-rank Estimation on Language Model

Ting Hua, Yen-Chang Hsu, Felicity Wang +3

Singular value decomposition (SVD) is one of the most popular compression methods that approximate a target matrix with smaller matrices. However, standard SVD treats the parameter…

cs.CV2022

ESTAS: Effective and Stable Trojan Attacks in Self-supervised Encoders with One Target Unlabelled Sample

Jiaqi Xue, Qian Lou

Emerging self-supervised learning (SSL) has become a popular image representation encoding method to obviate the reliance on labeled data and learn rich representations from large-…

cs.LG2022★ 20 cited

Language model compression with weighted low-rank factorization

Yen-Chang Hsu, Ting Hua, Sungen Chang +3

Factorizing a large matrix into small matrices is a popular strategy for model compression. Singular value decomposition (SVD) plays a vital role in this compression strategy, appr…