1.5k citations · 2.5k across the 34 of their papers we have counts for
35 papers
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization
Cheng Deng, Tianhang Zhang, Zhongmou He +9
Large language models (LLMs) have achieved great success in general domains of natural language processing. In this paper, we bring LLMs to the realm of geoscience with the objecti…
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Ziwei He, Meng Yang, Minwei Feng +4
The transformer model is known to be computationally demanding, and prohibitively costly for long sequences, as the self-attention module uses a quadratic time and space complexity…
Asymmetric Polynomial Loss For Multi-Label Classification
Yusheng Huang, Jiexing Qi, Xinbing Wang +1
Various tasks are reformulated as multi-label classification problems, in which the binary cross-entropy (BCE) loss is frequently utilized for optimizing well-designed models. Howe…
Text Classification in the Wild: a Large-scale Long-tailed Name Normalization Dataset
Jiexing Qi, Shuhao Li, Zhixin Guo +5
Real-world data usually exhibits a long-tailed distribution,with a few frequent labels and a lot of few-shot labels. The study of institution name normalization is a perfect applic…
Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing
Yunchong Song, Chenghu Zhou, Xinbing Wang +1
Most graph neural networks follow the message passing mechanism. However, it faces the over-smoothing problem when multiple times of message passing is applied to a graph, causing…
Adapting Knowledge for Few-shot Table-to-Text Generation
Zhixin Guo, Minyxuan Yan, Jiexing Qi +4
Pretrained language models (PLMs) have made remarkable progress in table-to-text generation tasks. However, the lack of domain-specific knowledge makes it challenging to bridge the…
Few-Shot Table-to-Text Generation with Prompt Planning and Knowledge Memorization
Zhixin Guo, Minyxuan Yan, Jiexing Qi +5
Pre-trained language models (PLM) have achieved remarkable advancement in table-to-text generation tasks. However, the lack of labeled domain-specific knowledge and the topology ga…
Text Editing as Imitation Game
Ning Shi, Bin Tang, Bo Yuan +4
Text editing, such as grammatical error correction, arises naturally from imperfect textual data. Recent works frame text editing as a multi-round sequence tagging task, where oper…
Syntax-guided Localized Self-attention by Constituency Syntactic Distance
Shengyuan Hou, Jushi Kai, Haotian Xue +5
Recent works have revealed that Transformers are implicitly learning the syntactic information in its lower layers from data, albeit is highly dependent on the quality and scale of…
INFINITY: A Simple Yet Effective Unsupervised Framework for Graph-Text Mutual Conversion
Yi Xu, Luoyi Fu, Zhouhan Lin +2
Graph-to-text (G2T) generation and text-to-graph (T2G) triple extraction are two essential tasks for constructing and applying knowledge graphs. Existing unsupervised approaches tu…
Transkimmer: Transformer Learns to Layer-wise Skim
Yue Guan, Zhengyi Li, Jingwen Leng +2
Transformer architecture has become the de-facto model for many machine learning tasks from natural language processing and computer vision. As such, improving its computational ef…
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL
Jiexing Qi, Jingyao Tang, Ziwei He +6
Relational structures such as schema linking and schema encoding have been validated as a key component to qualitatively translating natural language into SQL queries. However, int…
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition
Xichen Pan, Peiyu Chen, Yichen Gong +3
Training Transformer-based models demands a large amount of data, while obtaining aligned and labelled data in multimodality is rather cost-demanding, especially for audio-visual s…
Block-Skim: Efficient Question Answering for Transformer
Yue Guan, Zhengyi Li, Jingwen Leng +3
Transformer models have achieved promising results on natural language processing (NLP) tasks including extractive question answering (QA). Common Transformer encoders used in NLP…
Incorporating External POS Tagger for Punctuation Restoration
Ning Shi, Wei Wang, Boxin Wang +3
Punctuation restoration is an important post-processing step in automatic speech recognition. Among other kinds of external information, part-of-speech (POS) taggers provide inform…
Annotation Inconsistency and Entity Bias in MultiWOZ
Kun Qian, Ahmad Beirami, Zhouhan Lin +4
MultiWOZ is one of the most popular multi-domain task-oriented dialog datasets, containing 10K+ annotated dialogs covering eight domains. It has been widely accepted as a benchmark…
Reciprocal Supervised Learning Improves Neural Machine Translation
Minkai Xu, Mingxuan Wang, Zhouhan Lin +3
Despite the recent success on image classification, self-training has only achieved limited gains on structured prediction tasks such as neural machine translation (NMT). This is m…
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
Wenyu Du, Zhouhan Lin, Yikang Shen +3
It is commonly believed that knowledge of syntactic structure should improve language modeling. However, effectively and computationally efficiently incorporating syntactic structu…
Revisit Systematic Generalization via Meaningful Learning
Ning Shi, Boxin Wang, Wei Wang +2
Humans can systematically generalize to novel compositions of existing concepts. Recent studies argue that neural networks appear inherently ineffective in such cognitive capacity,…
Ordered Memory
Yikang Shen, Shawn Tan, Arian Hosseini +3
Stack-augmented recurrent neural networks (RNNs) have been of interest to the deep learning community for some time. However, the difficulty of training memory models remains a pro…
Interactive Language Learning by Question Answering
Xingdi Yuan, Marc-Alexandre Cote, Jie Fu +4
Humans observe and interact with the world to acquire knowledge. However, most existing machine reading comprehension (MRC) tasks miss the interactive, information-seeking componen…
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni +6
Recurrent Neural Networks (RNNs) with attention mechanisms have obtained state-of-the-art results for many sequence processing tasks. Most of these models use a simple form of enco…
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob +3
In this work, we propose a novel constituency parsing scheme. The model predicts a vector of real-valued scalars, named syntactic distances, for each split position in the input se…
A Deep Reinforcement Learning Chatbot (Short Version)
Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain +15
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capa…