43 citations · 181 across the 27 of their papers we have counts for
16 papers · 1 filter
Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling
Xingyue Huang, Xueying Ding, Mingxuan Ju +3
Softmax attention struggles with long contexts due to structural limitations: the strict sum-to-one constraint forces attention sinks on irrelevant tokens, and probability mass dis…
GiGL: Large-Scale Graph Neural Networks at Snapchat
Tong Zhao, Yozen Liu, Matthew Kolodner +12
Recent advances in graph machine learning (ML) with the introduction of Graph Neural Networks (GNNs) have led to a widespread interest in applying these approaches to business appl…
Node Duplication Improves Cold-start Link Prediction
Zhichun Guo, Tong Zhao, Yozen Liu +5
Graph Neural Networks (GNNs) are prominent in graph machine learning and have shown state-of-the-art performance in Link Prediction (LP) tasks. Nonetheless, recent studies show tha…
Graph Transformers for Large Graphs
Vijay Prakash Dwivedi, Yozen Liu, Anh Tuan Luu +3
Transformers have recently emerged as powerful neural networks for graph learning, showcasing state-of-the-art performance on several graph property prediction tasks. However, thes…
Context-Aware Prediction of User Engagement on Online Social Platforms
Heinrich Peters, Yozen Liu, Francesco Barbieri +3
The success of online social platforms hinges on their ability to predict and understand user behavior at scale. Here, we present data suggesting that context-aware modeling approa…
CARL-G: Clustering-Accelerated Representation Learning on Graphs
William Shiao, Uday Singh Saini, Yozen Liu +3
Self-supervised learning on graphs has made large strides in achieving great performance in various downstream tasks. However, many state-of-the-art methods suffer from a number of…
Link Prediction with Non-Contrastive Learning
William Shiao, Zhichun Guo, Tong Zhao +3
A recent focal area in the space of graph neural networks (GNNs) is graph self-supervised learning (SSL), which aims to derive useful node representations without labeled data. Not…
Linkless Link Prediction via Relational Distillation
Zhichun Guo, William Shiao, Shichang Zhang +4
Graph Neural Networks (GNNs) have shown exceptional performance in the task of link prediction. Despite their effectiveness, the high latency brought by non-trivial neighborhood da…
Empowering Graph Representation Learning with Test-Time Graph Transformation
Wei Jin, Tong Zhao, Jiayuan Ding +3
As powerful tools for representation learning on graphs, graph neural networks (GNNs) have facilitated various applications from drug discovery to recommender systems. Nevertheless…
MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization
Xiaotian Han, Tong Zhao, Yozen Liu +2
Training graph neural networks (GNNs) on large graphs is complex and extremely time consuming. This is attributed to overheads caused by sparse matrix multiplication, which are sid…
GStarX: Explaining Graph Neural Networks with Structure-Aware Cooperative Games
Shichang Zhang, Yozen Liu, Neil Shah +1
Explaining machine learning models is an important and increasingly popular area of research interest. The Shapley value from game theory has been proposed as a prime approach to c…
Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
Shichang Zhang, Yozen Liu, Yizhou Sun +1
Graph Neural Networks (GNNs) are popular for graph machine learning and have shown great results on wide node classification tasks. Yet, they are less popular for practical deploym…
Graph Condensation for Graph Neural Networks
Wei Jin, Lingxiao Zhao, Shichang Zhang +3
Given the prevalence of large-scale graphs in real-world applications, the storage and time for training neural models have raised increasing concerns. To alleviate the concerns, w…
A Unified View on Graph Neural Networks as Graph Signal Denoising
Yao Ma, Xiaorui Liu, Tong Zhao +3
Graph Neural Networks (GNNs) have risen to prominence in learning representations for graph structured data. A single GNN layer typically consists of a feature transformation and a…
Data Augmentation for Graph Neural Networks
Tong Zhao, Yozen Liu, Leonardo Neves +3
Data augmentation has been widely used to improve generalizability of machine learning models. However, comparatively little work studies data augmentation for graphs. This is larg…
Characterizing and Forecasting User Engagement with In-app Action Graph: A Case Study of Snapchat
Yozen Liu, Xiaolin Shi, Lucas Pierce +1
While mobile social apps have become increasingly important in people's daily life, we have limited understanding on what motivates users to engage with these apps. In this paper,…