2.1k citations · 2.5k across the 14 of their papers we have counts for
15 papers
Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMs
Zhangying Feng, Qianglong Chen, Ning Lu +6
The development of reasoning capabilities represents a critical frontier in large language models (LLMs) research, where reinforcement learning (RL) and process reward models (PRMs…
Emage: Non-Autoregressive Text-to-Image Generation
Zhangyin Feng, Runyi Hu, Liangxin Liu +7
Autoregressive and diffusion models drive the recent breakthroughs on text-to-image generation. Despite their huge success of generating high-realistic images, a common shortcoming…
Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications
Zhangyin Feng, Weitao Ma, Weijiang Yu +7
Large language models (LLMs) exhibit superior performance on various natural language tasks, but they are susceptible to issues stemming from outdated data and domain-specific limi…
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
Lei Huang, Weijiang Yu, Weitao Ma +8
The emergence of large language models (LLMs) has marked a significant breakthrough in natural language processing (NLP), fueling a paradigm shift in information acquisition. Never…
Retrieval-Generation Synergy Augmented Large Language Models
Zhangyin Feng, Xiaocheng Feng, Dezhi Zhao +2
Large language models augmented with task-relevant documents have demonstrated impressive performance on knowledge-intensive tasks. However, regarding how to obtain effective docum…
SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills
Zhangyin Feng, Yong Dai, Fan Zhang +6
Traditional multitask learning methods basically can only exploit common knowledge in task- or language-wise, which lose either cross-language or cross-task knowledge. This paper p…
Improved Visual Story Generation with Adaptive Context Modeling
Zhangyin Feng, Yuchen Ren, Xinmiao Yu +4
Diffusion models developed on top of powerful text-to-image generation models like Stable Diffusion achieve remarkable success in visual story generation. However, the best-perform…
One Model, Multiple Modalities: A Sparsely Activated Approach for Text, Sound, Image, Video and Code
Yong Dai, Duyu Tang, Liangxin Liu +7
People perceive the world with multiple senses (e.g., through hearing sounds, reading words and seeing objects). However, most existing AI systems only process an individual modali…
Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errors
Cong Zhou, Yong Dai, Duyu Tang +4
Chinese BERT models achieve remarkable progress in dealing with grammatical errors of word substitution. However, they fail to handle word insertion and deletion because BERT assum…
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
Yong Dai, Linyang Li, Cong Zhou +5
Whole word masking (WWM), which masks all subwords corresponding to a word at once, makes a better English BERT model. For the Chinese language, however, there is no subword becaus…
Exploring and Adapting Chinese GPT to Pinyin Input Method
Minghuan Tan, Yong Dai, Duyu Tang +5
While GPT has become the de-facto method for text generation tasks, its application to pinyin input method remains unexplored. In this work, we make the first exploration to levera…
Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words
Zhangyin Feng, Duyu Tang, Cong Zhou +6
The standard BERT adopts subword-based tokenization, which may break a word into two or more wordpieces (e.g., converting "lossless" to "loss" and "less"). This will bring inconven…
GraphCodeBERT: Pre-training Code Representations with Data Flow
Daya Guo, Shuo Ren, Shuai Lu +15
Pre-trained models for programming language have achieved dramatic empirical improvements on a variety of code-related tasks such as code search, code completion, code summarizatio…
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
Wanjun Zhong, Duyu Tang, Zhangyin Feng +7
Verifying the correctness of a textual statement requires not only semantic reasoning about the meaning of words, but also symbolic reasoning about logical operations like count, s…
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Zhangyin Feng, Daya Guo, Duyu Tang +8
We present CodeBERT, a bimodal pre-trained model for programming language (PL) and nat-ural language (NL). CodeBERT learns general-purpose representations that support downstream N…