2.4k citations · 6.2k across the 17 of their papers we have counts for
17 papers
FedGKD: Unleashing the Power of Collaboration in Federated Graph Neural Networks
Qiying Pan, Ruofan Wu, Tengfei Liu +3
Federated training of Graph Neural Networks (GNN) has become popular in recent years due to its ability to perform graph-related tasks under data isolation scenarios while preservi…
Rapid Image Labeling via Neuro-Symbolic Learning
Yifeng Wang, Zhi Tu, Yiwen Xiang +4
The success of Computer Vision (CV) relies heavily on manually annotated data. However, it is prohibitively expensive to annotate images in key domains such as healthcare, where da…
The Effect of Masking Strategies on Knowledge Retention by Language Models
Jonas Wallat, Tianyi Zhang, Avishek Anand
Language models retain a significant amount of world knowledge from their pre-training stage. This allows knowledgeable models to be applied to knowledge-intensive tasks prevalent…
Neural Frailty Machine: Beyond proportional hazard assumption in neural survival regressions
Ruofan Wu, Jiawei Qiao, Mingzhe Wu +5
We present neural frailty machine (NFM), a powerful and flexible neural modeling framework for survival regressions. The NFM framework utilizes the classical idea of multiplicative…
Human-in-the-Loop Schema Induction
Tianyi Zhang, Isaac Tham, Zhaoyi Hou +12
Schema induction builds a graph representation explaining how events unfold in a scenario. Existing approaches have been based on information retrieval (IR) and information extract…
Efficient Graph Field Integrators Meet Point Clouds
Krzysztof Choromanski, Arijit Sehanobish, Han Lin +13
We present two new classes of algorithms for efficient field integration on graphs encoding point clouds. The first class, SeparatorFactorization(SF), leverages the bounded genus o…
LADIS: Language Disentanglement for 3D Shape Editing
Ian Huang, Panos Achlioptas, Tianyi Zhang +3
Natural language interaction is a promising direction for democratizing 3D shape design. However, existing methods for text-driven 3D shape editing face challenges in producing dec…
Decentralized Training of Foundation Models in Heterogeneous Environments
Binhang Yuan, Yongjun He, Jared Quincy Davis +6
Training foundation models, such as GPT-3 and PaLM, can be extremely expensive, often involving tens of thousands of GPUs running continuously for months. These models are typicall…
TempLM: Distilling Language Models into Template-Based Generators
Tianyi Zhang, Mina Lee, Lisa Li +2
While pretrained language models (PLMs) have greatly improved text generation, they have also been known to produce unfaithful or inappropriate content. In contrast, classic templa…
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies
Tianyi Zhang, Tatsunori Hashimoto
We study how masking and predicting tokens in an unsupervised fashion can give rise to linguistic structures and downstream performance gains. Recent theories have suggested that p…
Revisiting Few-sample BERT Fine-tuning
Tianyi Zhang, Felix Wu, Arzoo Katiyar +2
This paper is a study of fine-tuning of BERT contextual representations, with focus on commonly observed instabilities in few-sample scenarios. We identify several factors that cau…
Identifying Mislabeled Data using the Area Under the Margin Ranking
Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg +1
Not all data in a typical training set help with generalization; some samples can be overly ambiguous or outrightly mislabeled. This paper introduces a new method to identify such…
QPyTorch: A Low-Precision Arithmetic Simulation Framework
Tianyi Zhang, Zhiqiu Lin, Guandao Yang +1
Low-precision training reduces computational cost and produces efficient models. Recent research in developing new low-precision training algorithms often relies on simulation to e…
SWALP : Stochastic Weight Averaging in Low-Precision Training
Guandao Yang, Tianyi Zhang, Polina Kirichenko +3
Low precision operations can provide scalability, memory savings, portability, and energy efficiency. This paper proposes SWALP, an approach to low precision training that averages…
BERTScore: Evaluating Text Generation with BERT
Tianyi Zhang, Varsha Kishore, Felix Wu +2
We propose BERTScore, an automatic evaluation metric for text generation. Analogously to common metrics, BERTScore computes a similarity score for each token in the candidate sente…
Simplifying Graph Convolutional Networks
Felix Wu, Tianyi Zhang, Amauri Holanda de Souza +3
Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive i…