205 citations · 380 across the 13 of their papers we have counts for
6 papers · 1 filter
Sparse-softmax: A Simpler and Faster Alternative Softmax Transformation
Shaoshi Sun, Zhenyuan Zhang, BoCheng Huang +4
The softmax function is widely used in artificial neural networks for the multiclass classification problems, where the softmax transformation enforces the output to be positive an…
ZARTS: On Zero-order Optimization for Neural Architecture Search
Xiaoxing Wang, Wenxuan Guo, Junchi Yan +2
Differentiable architecture search (DARTS) has been a popular one-shot paradigm for NAS due to its high efficiency. It introduces trainable architecture parameters to represent the…
BioCopy: A Plug-And-Play Span Copy Mechanism in Seq2Seq Models
Yi Liu, Guoan Zhang, Puning Yu +2
Copy mechanisms explicitly obtain unchanged tokens from the source (input) sequence to generate the target (output) sequence under the neural seq2seq framework. However, most of th…
RoFormer: Enhanced Transformer with Rotary Position Embedding
Jianlin Su, Yu Lu, Shengfeng Pan +3
Position encoding recently has shown effective in the transformer architecture. It enables valuable supervision for dependency modeling between elements at different positions of t…
Whitening Sentence Representations for Better Semantics and Faster Retrieval
Jianlin Su, Jiarun Cao, Weijie Liu +1
Pre-training models such as BERT have achieved great success in many natural language processing tasks. However, how to obtain better sentence representation through these pre-trai…
A2-FPN for Semantic Segmentation of Fine-Resolution Remotely Sensed Images
Rui Li, Shunyi Zheng, Ce Zhang +2
Semantic segmentation using fine-resolution remotely sensed images plays a critical role in many practical applications, such as urban planning, environmental protection, natural a…