6 citations · 17 across the 4 of their papers we have counts for
4 papers
Zero-th Order Algorithm for Softmax Attention Optimization
Yichuan Deng, Zhihang Li, Sridhar Mahadevan +1
Large language models (LLMs) have brought about significant transformations in human society. Among the crucial computations in LLMs, the softmax unit holds great importance. Its h…
Attention Scheme Inspired Softmax Regression
Yichuan Deng, Zhihang Li, Zhao Song
Large language models (LLMs) have made transformed changes for human society. One of the key computation in LLMs is the softmax unit. This operation is important in LLMs because it…
Solving Regularized Exp, Cosh and Sinh Regression Problems
Zhihang Li, Zhao Song, Tianyi Zhou
In modern machine learning, attention computation is a fundamental task for training large language models such as Transformer, GPT-4 and ChatGPT. In this work, we study exponentia…
Self-Supervised Modality-Aware Multiple Granularity Pre-Training for RGB-Infrared Person Re-Identification
Lin Wan, Qianyan Jing, Zongyuan Sun +3
RGB-Infrared person re-identification (RGB-IR ReID) aims to associate people across disjoint RGB and IR camera views. Currently, state-of-the-art performance of RGB-IR ReID is not…