1.5k citations · 2.5k across the 34 of their papers we have counts for
22 papers · 1 filter
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization
Cheng Deng, Tianhang Zhang, Zhongmou He +9
Large language models (LLMs) have achieved great success in general domains of natural language processing. In this paper, we bring LLMs to the realm of geoscience with the objecti…
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Ziwei He, Meng Yang, Minwei Feng +4
The transformer model is known to be computationally demanding, and prohibitively costly for long sequences, as the self-attention module uses a quadratic time and space complexity…
Text Classification in the Wild: a Large-scale Long-tailed Name Normalization Dataset
Jiexing Qi, Shuhao Li, Zhixin Guo +5
Real-world data usually exhibits a long-tailed distribution,with a few frequent labels and a lot of few-shot labels. The study of institution name normalization is a perfect applic…
Adapting Knowledge for Few-shot Table-to-Text Generation
Zhixin Guo, Minyxuan Yan, Jiexing Qi +4
Pretrained language models (PLMs) have made remarkable progress in table-to-text generation tasks. However, the lack of domain-specific knowledge makes it challenging to bridge the…
Few-Shot Table-to-Text Generation with Prompt Planning and Knowledge Memorization
Zhixin Guo, Minyxuan Yan, Jiexing Qi +5
Pre-trained language models (PLM) have achieved remarkable advancement in table-to-text generation tasks. However, the lack of labeled domain-specific knowledge and the topology ga…
Text Editing as Imitation Game
Ning Shi, Bin Tang, Bo Yuan +4
Text editing, such as grammatical error correction, arises naturally from imperfect textual data. Recent works frame text editing as a multi-round sequence tagging task, where oper…
Syntax-guided Localized Self-attention by Constituency Syntactic Distance
Shengyuan Hou, Jushi Kai, Haotian Xue +5
Recent works have revealed that Transformers are implicitly learning the syntactic information in its lower layers from data, albeit is highly dependent on the quality and scale of…
INFINITY: A Simple Yet Effective Unsupervised Framework for Graph-Text Mutual Conversion
Yi Xu, Luoyi Fu, Zhouhan Lin +2
Graph-to-text (G2T) generation and text-to-graph (T2G) triple extraction are two essential tasks for constructing and applying knowledge graphs. Existing unsupervised approaches tu…
Transkimmer: Transformer Learns to Layer-wise Skim
Yue Guan, Zhengyi Li, Jingwen Leng +2
Transformer architecture has become the de-facto model for many machine learning tasks from natural language processing and computer vision. As such, improving its computational ef…
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL
Jiexing Qi, Jingyao Tang, Ziwei He +6
Relational structures such as schema linking and schema encoding have been validated as a key component to qualitatively translating natural language into SQL queries. However, int…
Block-Skim: Efficient Question Answering for Transformer
Yue Guan, Zhengyi Li, Jingwen Leng +3
Transformer models have achieved promising results on natural language processing (NLP) tasks including extractive question answering (QA). Common Transformer encoders used in NLP…
Incorporating External POS Tagger for Punctuation Restoration
Ning Shi, Wei Wang, Boxin Wang +3
Punctuation restoration is an important post-processing step in automatic speech recognition. Among other kinds of external information, part-of-speech (POS) taggers provide inform…
Annotation Inconsistency and Entity Bias in MultiWOZ
Kun Qian, Ahmad Beirami, Zhouhan Lin +4
MultiWOZ is one of the most popular multi-domain task-oriented dialog datasets, containing 10K+ annotated dialogs covering eight domains. It has been widely accepted as a benchmark…
Reciprocal Supervised Learning Improves Neural Machine Translation
Minkai Xu, Mingxuan Wang, Zhouhan Lin +3
Despite the recent success on image classification, self-training has only achieved limited gains on structured prediction tasks such as neural machine translation (NMT). This is m…
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
Wenyu Du, Zhouhan Lin, Yikang Shen +3
It is commonly believed that knowledge of syntactic structure should improve language modeling. However, effectively and computationally efficiently incorporating syntactic structu…
Revisit Systematic Generalization via Meaningful Learning
Ning Shi, Boxin Wang, Wei Wang +2
Humans can systematically generalize to novel compositions of existing concepts. Recent studies argue that neural networks appear inherently ineffective in such cognitive capacity,…
Interactive Language Learning by Question Answering
Xingdi Yuan, Marc-Alexandre Cote, Jie Fu +4
Humans observe and interact with the world to acquire knowledge. However, most existing machine reading comprehension (MRC) tasks miss the interactive, information-seeking componen…
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
Yikang Shen, Zhouhan Lin, Athul Paul Jacob +3
In this work, we propose a novel constituency parsing scheme. The model predicts a vector of real-valued scalars, named syntactic distances, for each split position in the input se…
A Deep Reinforcement Learning Chatbot (Short Version)
Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain +15
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capa…
Neural Language Modeling by Jointly Learning Syntax and Lexicon
Yikang Shen, Zhouhan Lin, Chin-Wei Huang +1
We propose a neural language model capable of unsupervised syntactic structure induction. The model leverages the structure information to form better semantic representations and…
A Deep Reinforcement Learning Chatbot
Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain +15
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capa…
A Structured Self-attentive Sentence Embedding
Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos +4
This paper proposes a new model for extracting an interpretable sentence embedding by introducing self-attention. Instead of using a vector, we use a 2-D matrix to represent the em…