1.1k citations · 1.2k across the 11 of their papers we have counts for
5 papers · 1 filter
Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
Zhenyu Hou, Pengfan Du, Yilin Niu +7
This study explores the scaling properties of Reinforcement Learning from Human Feedback (RLHF) in Large Language Models (LLMs). Although RLHF is considered an important step in po…
Generalizing Graph Transformers Across Diverse Graphs and Tasks via Pre-training
Yufei He, Zhenyu Hou, Yukuo Cen +5
Graph pre-training has been concentrated on graph-level tasks involving small graphs (e.g., molecular graphs) or learning node representations on a fixed graph. Extending graph pre…
GraphAlign: Pretraining One Graph Neural Network on Multiple Graphs via Feature Alignment
Zhenyu Hou, Haozhan Li, Yukuo Cen +2
Graph self-supervised learning (SSL) holds considerable promise for mining and learning with graph-structured data. Yet, a significant challenge in graph SSL lies in the feature di…
ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline
Yifan Xu, Xiao Liu, Xinghan Liu +9
Large language models (LLMs) have shown excellent mastering of human language, but still struggle in real-world applications that require mathematical problem-solving. While many s…
ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback
Zhenyu Hou, Yilin Niu, Zhengxiao Du +8
ChatGLM is a free-to-use AI service powered by the ChatGLM family of large language models (LLMs). In this paper, we present the ChatGLM-RLHF pipeline -- a reinforcement learning f…