1.1k citations · 1.2k across the 11 of their papers we have counts for
11 papers
Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
Zhenyu Hou, Pengfan Du, Yilin Niu +7
This study explores the scaling properties of Reinforcement Learning from Human Feedback (RLHF) in Large Language Models (LLMs). Although RLHF is considered an important step in po…
Generalizing Graph Transformers Across Diverse Graphs and Tasks via Pre-training
Yufei He, Zhenyu Hou, Yukuo Cen +5
Graph pre-training has been concentrated on graph-level tasks involving small graphs (e.g., molecular graphs) or learning node representations on a fixed graph. Extending graph pre…
GraphAlign: Pretraining One Graph Neural Network on Multiple Graphs via Feature Alignment
Zhenyu Hou, Haozhan Li, Yukuo Cen +2
Graph self-supervised learning (SSL) holds considerable promise for mining and learning with graph-structured data. Yet, a significant challenge in graph SSL lies in the feature di…
ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline
Yifan Xu, Xiao Liu, Xinghan Liu +9
Large language models (LLMs) have shown excellent mastering of human language, but still struggle in real-world applications that require mathematical problem-solving. While many s…
ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback
Zhenyu Hou, Yilin Niu, Zhengxiao Du +8
ChatGLM is a free-to-use AI service powered by the ChatGLM family of large language models (LLMs). In this paper, we present the ChatGLM-RLHF pipeline -- a reinforcement learning f…
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner
Zhenyu Hou, Yufei He, Yukuo Cen +4
Graph self-supervised learning (SSL), including contrastive and generative approaches, offers great potential to address the fundamental challenge of label scarcity in real-world g…
GraphMAE: Self-Supervised Masked Graph Autoencoders
Zhenyu Hou, Xiao Liu, Yukuo Cen +4
Self-supervised learning (SSL) has been extensively explored in recent years. Particularly, generative SSL has seen emerging success in natural language processing and other AI fie…
SCR: Training Graph Neural Networks with Consistency Regularization
Chenhui Zhang, Yufei He, Yukuo Cen +7
We present the SCR framework for enhancing the training of graph neural networks (GNNs) with consistency regularization. Regularization is a set of strategies used in Machine Learn…
Understanding WeChat User Preferences and "Wow" Diffusion
Fanjin Zhang, Jie Tang, Xueyi Liu +9
WeChat is the largest social instant messaging platform in China, with 1.1 billion monthly active users. "Top Stories" is a novel friend-enhanced recommendation engine in WeChat, i…
CogDL: A Comprehensive Library for Graph Deep Learning
Yukuo Cen, Zhenyu Hou, Yan Wang +15
Graph neural networks (GNNs) have attracted tremendous attention from the graph learning community in recent years. It has been widely adopted in various real-world applications fr…
Self-supervised Learning: Generative or Contrastive
Xiao Liu, Fanjin Zhang, Zhenyu Hou +4
Deep supervised learning has achieved great success in the last decade. However, its deficiencies of dependence on manual labels and vulnerability to attacks have driven people to…