12 citations · 45 across the 37 of their papers we have counts for
9 papers · 1 filter
A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration
Yingqian Cui, Pengfei He, Xianfeng Tang +4
Few-shot Chain-of-Thought (CoT) prompting has demonstrated strong performance in improving the reasoning capabilities of large language models (LLMs). While theoretical investigati…
Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning
Pengfei He, Zitao Li, Yue Xing +3
Zero-shot reasoning methods with Large Language Models (LLMs) offer significant advantages including great generalization to novel tasks and reduced dependency on human-crafted exa…
Towards the Effect of Examples on In-Context Learning: A Theoretical Case Study
Pengfei He, Yingqian Cui, Han Xu +4
In-context learning (ICL) has emerged as a powerful capability for large language models (LLMs) to adapt to downstream tasks by leveraging a few (demonstration) examples. Despite i…
Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data
Shenglai Zeng, Jiankun Zhang, Pengfei He +7
Retrieval-augmented generation (RAG) enhances the outputs of language models by integrating relevant information retrieved from external knowledge sources. However, when the retrie…
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
Yuping Lin, Pengfei He, Han Xu +4
Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents. Although there are diverse jailbreak attack…
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)
Shenglai Zeng, Jiankun Zhang, Pengfei He +8
Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model with proprietary and private data, where data privacy is a pivotal concern. Whereas extens…
Copyright Protection in Generative AI: A Technical Perspective
Jie Ren, Han Xu, Pengfei He +10
Generative AI has witnessed rapid advancement in recent years, expanding their capabilities to create synthesized content such as text, images, audio, and code. The high fidelity a…
Data Poisoning for In-context Learning
Pengfei He, Han Xu, Yue Xing +3
In the domain of large language models (LLMs), in-context learning (ICL) has been recognized for its innovative ability to adapt to new tasks, relying on examples rather than retra…
Superiority of Multi-Head Attention in In-Context Linear Regression
Yingqian Cui, Jie Ren, Pengfei He +2
We present a theoretical analysis of the performance of transformer with softmax attention in in-context learning with linear regression tasks. While the existing literature predom…