147 citations · 1k across the 56 of their papers we have counts for
57 papers
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
Songqiao Su, Xiaoya Li, Albert Wang +3
In this paper, we propose CUDA-L2, a system that combines large language models (LLMs) and reinforcement learning (RL) to automatically optimize Half-precision General Matrix Multi…
CRINN: Contrastive Reinforcement Learning for Approximate Nearest Neighbor Search
Xiaoya Li, Albert Wang, Guoyin Wang +2
Approximate nearest-neighbor search (ANNS) algorithms have become increasingly critical for recent AI applications, particularly in retrieval-augmented generation (RAG) and agent-b…
CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
Xiaoya Li, Albert Wang, Guoyin Wang +2
The exponential growth in demand for GPU computing resources has created an urgent need for automated CUDA optimization strategies. While recent advances in LLMs show promise for c…
FaceID-6M: A Large-Scale, Open-Source FaceID Customization Dataset
Shuhe Wang, Xiaoya Li, Jiwei Li +8
Due to the data-driven nature of current face identity (FaceID) customization methods, all state-of-the-art models rely on large-scale datasets containing millions of high-quality…
Turn That Frown Upside Down: FaceID Customization via Cross-Training Data
Shuhe Wang, Xiaoya Li, Xiaofei Sun +4
Existing face identity (FaceID) customization methods perform well but are limited to generating identical faces as the input, while in real-world applications, users often desire…
FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learning
Zhonghua Jiang, Jimin Xu, Shengyu Zhang +5
Federated learning (FL) is a promising technology for data privacy and distributed optimization, but it suffers from data imbalance and heterogeneity among clients. Existing FL met…
Reinforcement Learning Enhanced LLMs: A Survey
Shuhe Wang, Shengyu Zhang, Jie Zhang +7
Reinforcement learning (RL) enhanced large language models (LLMs), particularly exemplified by DeepSeek-R1, have exhibited outstanding performance. Despite the effectiveness in imp…
Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning
Shuhe Wang, Guoyin Wang, Yizhong Wang +3
Packing, initially utilized in the pre-training phase, is an optimization technique designed to maximize hardware resource efficiency by combining different training sequences to f…
Sim-GPT: Text Similarity via GPT Annotated Data
Shuhe Wang, Beiming Cao, Shengyu Zhang +5
Due to the lack of a large collection of high-quality labeled sentence pairs with textual similarity scores, existing approaches for Semantic Textual Similarity (STS) mostly rely o…
Sentiment Analysis through LLM Negotiations
Xiaofei Sun, Xiaoya Li, Shengyu Zhang +5
A standard paradigm for sentiment analysis is to rely on a singular LLM and makes the decision in a single round under the framework of in-context learning. This framework suffers…
Instruction Tuning for Large Language Models: A Survey
Shengyu Zhang, Linfeng Dong, Xiaoya Li +8
This paper surveys research works in the quickly advancing field of instruction tuning (IT), which can also be referred to as supervised fine-tuning (SFT)\footnote{In this paper, u…
Pushing the Limits of ChatGPT on NLP Tasks
Xiaofei Sun, Linfeng Dong, Xiaoya Li +8
Despite the success of ChatGPT, its performances on most NLP tasks are still well below the supervised baselines. In this work, we looked into the causes, and discovered that its s…
Text Classification via Large Language Models
Xiaofei Sun, Xiaoya Li, Jiwei Li +4
Despite the remarkable success of large-scale Language Models (LLMs) such as GPT-3, their performances still significantly underperform fine-tuned models in the task of text classi…
GPT-NER: Named Entity Recognition via Large Language Models
Shuhe Wang, Xiaofei Sun, Xiaoya Li +5
Despite the fact that large-scale Language Models (LLM) have achieved SOTA performances on a variety of NLP tasks, its performance on NER is still significantly below supervised ba…
Backdoor Attacks with Input-unique Triggers in NLP
Xukun Zhou, Jiwei Li, Tianwei Zhang +3
Backdoor attack aims at inducing neural models to make incorrect predictions for poison data while keeping predictions on the clean dataset unchanged, which creates a considerable…
GNN-SL: Sequence Labeling Based on Nearest Examples via GNN
Shuhe Wang, Yuxian Meng, Rongbin Ouyang +4
To better handle long-tail cases in the sequence labeling (SL) task, in this work, we introduce graph neural networks sequence labeling (GNN-SL), which augments the vanilla SL mode…
NN-NER: Named Entity Recognition with Nearest Neighbor Search
Shuhe Wang, Xiaoya Li, Yuxian Meng +4
Inspired by recent advances in retrieval augmented methods in NLP~\citep{khandelwal2019generalization,khandelwal2020nearest,meng2021gnn}, in this paper, we introduce a nearest…
Faster Nearest Neighbor Machine Translation
Shuhe Wang, Jiwei Li, Yuxian Meng +5
NN based neural machine translation (NN-MT) has achieved state-of-the-art results in a variety of MT tasks. One significant shortcoming of NN-MT lies in its inefficiency i…
A General Framework for Defending Against Backdoor Attacks via Influence Graph
Xiaofei Sun, Jiwei Li, Xiaoya Li +5
In this work, we propose a new and general framework to defend against backdoor attacks, inspired by the fact that attack triggers usually follow a \textsc{specific} type of attack…
Triggerless Backdoor Attack for NLP Tasks with Clean Labels
Leilei Gan, Jiwei Li, Tianwei Zhang +6
Backdoor attacks pose a new threat to NLP models. A standard strategy to construct poisoned data in backdoor attacks is to insert triggers (e.g., rare words) into selected sentence…
Interpreting Deep Learning Models in Natural Language Processing: A Review
Xiaofei Sun, Diyi Yang, Xiaoya Li +6
Neural network models have achieved state-of-the-art performances in a wide range of natural language processing (NLP) tasks. However, a long-standing criticism against neural netw…
BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models
Kangjie Chen, Yuxian Meng, Xiaofei Sun +4
Pre-trained Natural Language Processing (NLP) models can be easily adapted to a variety of downstream language tasks. This significantly accelerates the development of language mod…
GNN-LM: Language Modeling based on Global Contexts via GNN
Yuxian Meng, Shi Zong, Xiaoya Li +4
Inspired by the notion that ``{\it to copy is easier than to memorize}``, in this work, we introduce GNN-LM, which extends the vanilla neural language model (LM) by allowing to ref…
OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts
Shuhe Wang, Yuxian Meng, Xiaoya Li +3
In order to better simulate the real human conversation process, models need to generate dialogue utterances based on not only preceding textual contexts but also visual contexts.…