43 citations · 181 across the 27 of their papers we have counts for
28 papers
Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling
Xingyue Huang, Xueying Ding, Mingxuan Ju +3
Softmax attention struggles with long contexts due to structural limitations: the strict sum-to-one constraint forces attention sinks on irrelevant tokens, and probability mass dis…
Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings
Xueying Ding, Xingyue Huang, Mingxuan Ju +5
Large language models produce powerful text embeddings, but their causal attention mechanism restricts the flow of information from later to earlier tokens, degrading representatio…
Beyond Unimodal Boundaries: Generative Recommendation with Multimodal Semantics
Jing Zhu, Mingxuan Ju, Yozen Liu +3
Generative recommendation (GR) has become a powerful paradigm in recommendation systems that implicitly links modality and semantics to item representation, in contrast to previous…
GiGL: Large-Scale Graph Neural Networks at Snapchat
Tong Zhao, Yozen Liu, Matthew Kolodner +12
Recent advances in graph machine learning (ML) with the introduction of Graph Neural Networks (GNNs) have led to a widespread interest in applying these approaches to business appl…
GraphHash: Graph Clustering Enables Parameter Efficiency in Recommender Systems
Xinyi Wu, Donald Loveland, Runjin Chen +7
Deep recommender systems rely heavily on large embedding tables to handle high-cardinality categorical features such as user/item identifiers, and face significant memory constrain…
Robust Training Objectives Improve Embedding-based Retrieval in Industrial Recommendation Systems
Matthew Kolodner, Mingxuan Ju, Zihao Fan +5
Improving recommendation systems (RS) can greatly enhance the user experience across many domains, such as social media. Many RS utilize embedding-based retrieval (EBR) approaches…
How Does Message Passing Improve Collaborative Filtering?
Mingxuan Ju, William Shiao, Zhichun Guo +4
Collaborative filtering (CF) has exhibited prominent results for recommender systems and been broadly utilized for real-world applications. A branch of research enhances CF methods…
Improving Out-of-Vocabulary Handling in Recommendation Systems
William Shiao, Mingxuan Ju, Zhichun Guo +5
Recommendation systems (RS) are an increasingly relevant area for both academic and industry researchers, given their widespread impact on the daily online experiences of billions…
USE: Dynamic User Modeling with Stateful Sequence Models
Zhihan Zhou, Qixiang Fang, Leonardo Neves +5
User embeddings play a crucial role in user engagement forecasting and personalized services. Recent advances in sequence modeling have sparked interest in learning user embeddings…
Node Duplication Improves Cold-start Link Prediction
Zhichun Guo, Tong Zhao, Yozen Liu +5
Graph Neural Networks (GNNs) are prominent in graph machine learning and have shown state-of-the-art performance in Link Prediction (LP) tasks. Nonetheless, recent studies show tha…
Graph Transformers for Large Graphs
Vijay Prakash Dwivedi, Yozen Liu, Anh Tuan Luu +3
Transformers have recently emerged as powerful neural networks for graph learning, showcasing state-of-the-art performance on several graph property prediction tasks. However, thes…
General-Purpose User Modeling with Behavioral Logs: A Snapchat Case Study
Qixiang Fang, Zhihan Zhou, Francesco Barbieri +6
Learning general-purpose user representations based on user behavioral logs is an increasingly popular user modeling approach. It benefits from easily available, privacy-friendly y…
Context-Aware Prediction of User Engagement on Online Social Platforms
Heinrich Peters, Yozen Liu, Francesco Barbieri +3
The success of online social platforms hinges on their ability to predict and understand user behavior at scale. Here, we present data suggesting that context-aware modeling approa…
CARL-G: Clustering-Accelerated Representation Learning on Graphs
William Shiao, Uday Singh Saini, Yozen Liu +3
Self-supervised learning on graphs has made large strides in achieving great performance in various downstream tasks. However, many state-of-the-art methods suffer from a number of…
Cultural Differences in Friendship Network Behaviors: A Snapchat Case Study
Agrima Seth, Jiyin Cao, Xiaolin Shi +3
Culture shapes people's behavior, both online and offline. Surprisingly, there is sparse research on how cultural context affects network formation and content consumption on socia…
Link Prediction with Non-Contrastive Learning
William Shiao, Zhichun Guo, Tong Zhao +3
A recent focal area in the space of graph neural networks (GNNs) is graph self-supervised learning (SSL), which aims to derive useful node representations without labeled data. Not…
Reciprocity, Homophily, and Social Network Effects in Pictorial Communication: A Case Study of Bitmoji Stickers
Julie Jiang, Ron Dotsch, Mireia Triguero Roura +4
Pictorial emojis and stickers are commonly used in online social networking to facilitate and aid communications. We delve into the use of Bitmoji stickers, a highly expressive for…
Linkless Link Prediction via Relational Distillation
Zhichun Guo, William Shiao, Shichang Zhang +4
Graph Neural Networks (GNNs) have shown exceptional performance in the task of link prediction. Despite their effectiveness, the high latency brought by non-trivial neighborhood da…
Empowering Graph Representation Learning with Test-Time Graph Transformation
Wei Jin, Tong Zhao, Jiayuan Ding +3
As powerful tools for representation learning on graphs, graph neural networks (GNNs) have facilitated various applications from drug discovery to recommender systems. Nevertheless…
MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization
Xiaotian Han, Tong Zhao, Yozen Liu +2
Training graph neural networks (GNNs) on large graphs is complex and extremely time consuming. This is attributed to overheads caused by sparse matrix multiplication, which are sid…
Flashlight: Scalable Link Prediction with Effective Decoders
Yiwei Wang, Bryan Hooi, Yozen Liu +3
Link prediction (LP) has been recognized as an important task in graph learning with its broad practical applications. A typical application of LP is to retrieve the top scoring ne…
GStarX: Explaining Graph Neural Networks with Structure-Aware Cooperative Games
Shichang Zhang, Yozen Liu, Neil Shah +1
Explaining machine learning models is an important and increasingly popular area of research interest. The Shapley value from game theory has been proposed as a prime approach to c…
Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
Shichang Zhang, Yozen Liu, Yizhou Sun +1
Graph Neural Networks (GNNs) are popular for graph machine learning and have shown great results on wide node classification tasks. Yet, they are less popular for practical deploym…
Graph Condensation for Graph Neural Networks
Wei Jin, Lingxiao Zhao, Shichang Zhang +3
Given the prevalence of large-scale graphs in real-world applications, the storage and time for training neural models have raised increasing concerns. To alleviate the concerns, w…