26 citations · 102 across the 21 of their papers we have counts for
21 papers
Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation
Chen Dun, Mirian Hipolito Garcia, Guoqing Zheng +3
Large Language Models (LLMs) have the ability to solve a variety of tasks, such as text summarization and mathematical questions, just out of the box, but they are often trained wi…
Hybrid-RACA: Hybrid Retrieval-Augmented Composition Assistance for Real-time Text Prediction
Menglin Xia, Xuchao Zhang, Camille Couturier +3
Large language models (LLMs) enhanced with retrieval augmentation has shown great performance in many applications. However, the computational demands for these models pose a chall…
Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
Yehya Farhat, Hamza ElMokhtar Shili, Fangshuo Liao +7
Mixture-of-Experts (MoEs) achieve scalability by dynamically activating subsets of their components. Yet, understanding how expertise emerges through joint training of gating mecha…
Boosting Natural Language Generation from Instructions with Meta-Learning
Budhaditya Deb, Guoqing Zheng, Ahmed Hassan Awadallah
Recent work has shown that language models (LMs) trained with multi-task \textit{instructional learning} (MTIL) can solve diverse NLP tasks in zero- and few-shot settings with impr…
Toward Robust Graph Semi-Supervised Learning against Extreme Data Scarcity
Kaize Ding, Elnaz Nouri, Guoqing Zheng +2
The success of graph neural networks on graph-based web mining highly relies on abundant human-annotated data, which is laborious to obtain in practice. When only few labeled nodes…
ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
Yue Zhao, Guoqing Zheng, Subhabrata Mukherjee +2
Existing works on anomaly detection (AD) rely on clean labels from human annotators that are expensive to acquire in practice. In this work, we propose a method to leverage weak/no…
Pathologies of Pre-trained Language Models in Few-shot Fine-tuning
Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah +1
Although adapting pre-trained language models with few examples has shown promising performance on text classification, there is a lack of understanding of where the performance ga…
Knowledge Infused Decoding
Ruibo Liu, Guoqing Zheng, Shashank Gupta +5
Pre-trained language models (LMs) have been shown to memorize a substantial amount of knowledge from the pre-training corpora; however, they are still limited in recalling factuall…
CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
Subhabrata Mukherjee, Xiaodong Liu, Guoqing Zheng +6
Most recent progress in natural language understanding (NLU) has been driven, in part, by benchmarks such as GLUE, SuperGLUE, SQuAD, etc. In fact, many NLU models have now matched…
A Conditional Generative Matching Model for Multi-lingual Reply Suggestion
Budhaditya Deb, Guoqing Zheng, Milad Shokouhi +1
We study the problem of multilingual automated reply suggestions (RS) model serving many languages simultaneously. Multilingual models are often challenged by model capacity and se…
MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces
Srinagesh Sharma, Guoqing Zheng, Ahmed Hassan Awadallah
Albeit the universal representational power of pre-trained language models, adapting them onto a specific NLP task still requires a considerably large amount of labeled data. Effec…
WALNUT: A Benchmark on Semi-weakly Supervised Learning for Natural Language Understanding
Guoqing Zheng, Giannis Karamanolakis, Kai Shu +1
Building machine learning models for natural language understanding (NLU) tasks relies heavily on labeled data. Weak supervision has been proven valuable when large amount of label…
A Dataset and Baselines for Multilingual Reply Suggestion
Mozhi Zhang, Wei Wang, Budhaditya Deb +3
Reply suggestion models help users process emails and chats faster. Previous work only studies English reply suggestion. Instead, we present MRS, a multilingual reply suggestion da…
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning
Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee +3
The combination of multilingual pre-trained representations and cross-lingual transfer learning is one of the most effective methods for building functional NLP systems for low-res…
Self-Training with Weak Supervision
Giannis Karamanolakis, Subhabrata Mukherjee, Guoqing Zheng +1
State-of-the-art deep neural networks require large-scale labeled training data that is often expensive to obtain or not available for many tasks. Weak supervision in the form of d…
Learning with Weak Supervision for Email Intent Detection
Kai Shu, Subhabrata Mukherjee, Guoqing Zheng +3
Email remains one of the most frequently used means of online communication. People spend a significant amount of time every day on emails to exchange information, manage tasks and…
Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake News
Kai Shu, Guoqing Zheng, Yichuan Li +4
Social media has greatly enabled people to participate in online activities at an unprecedented rate. However, this unrestricted access also exacerbates the spread of misinformatio…
Meta Label Correction for Noisy Label Learning
Guoqing Zheng, Ahmed Hassan Awadallah, Susan Dumais
Leveraging weak or noisy supervision for building effective machine learning models has long been an important research problem. Its importance has further increased recently due t…
Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
Guokun Lai, Bohan Li, Guoqing Zheng +1
How to model distribution of sequential data, including but not limited to speech and human motions, is an important ongoing research problem. It has been demonstrated that model c…
Asymmetric Variational Autoencoders
Guoqing Zheng, Yiming Yang, Jaime Carbonell
Variational inference for latent variable models is prevalent in various machine learning problems, typically solved by maximizing the Evidence Lower Bound (ELBO) of the true data…
Convolutional Normalizing Flows
Guoqing Zheng, Yiming Yang, Jaime Carbonell
Bayesian posterior inference is prevalent in various machine learning problems. Variational inference provides one way to approximate the posterior distribution, however its expres…