19 citations · 63 across the 20 of their papers we have counts for
20 papers
Macroeconomic Forecasting and Machine Learning
Ta-Chung Chi, Ting-Han Fan, Raffaele M. Ghigliazza +3
We forecast the full conditional distribution of macroeconomic outcomes by systematically integrating three key principles: using high-dimensional data with appropriate regularizat…
Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation
Ta-Chung Chi, Ting-Han Fan, Alexander I. Rudnicky
An ideal length-extrapolatable Transformer language model can handle sequences longer than the training length without any fine-tuning. Such long-context utilization capability rel…
Advancing Regular Language Reasoning in Linear Recurrent Neural Networks
Ting-Han Fan, Ta-Chung Chi, Alexander I. Rudnicky
In recent studies, linear recurrent neural networks (LRNNs) have achieved Transformer-level performance in natural language and long-range modeling, while offering rapid parallel t…
Structured Dialogue Discourse Parsing
Ta-Chung Chi, Alexander I. Rudnicky
Dialogue discourse parsing aims to uncover the internal structure of a multi-participant conversation by finding all the discourse~\emph{links} and corresponding~\emph{relations}.…
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
Yau-Shian Wang, Ta-Chung Chi, Ruohong Zhang +1
We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification. We formulate text classification as a neural…
Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings
Ta-Chung Chi, Ting-Han Fan, Li-Wei Chen +2
The use of positional embeddings in transformer language models is widely accepted. However, recent research has called into question the necessity of such embeddings. We further e…
Transformer Working Memory Enables Regular Language Reasoning and Natural Language Length Extrapolation
Ta-Chung Chi, Ting-Han Fan, Alexander I. Rudnicky +1
Unlike recurrent models, conventional wisdom has it that Transformers cannot perfectly model regular languages. Inspired by the notion of working memory, we propose a new Transform…
Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis
Ta-Chung Chi, Ting-Han Fan, Alexander I. Rudnicky +1
Length extrapolation permits training a transformer language model on short sequences that preserves perplexities when tested on substantially longer sequences. A relative position…
On Task-Adaptive Pretraining for Dialogue Response Selection
Tzu-Hsiang Lin, Ta-Chung Chi, Anna Rumshisky
Recent advancements in dialogue response selection (DRS) are based on the \textit{task-adaptive pre-training (TAP)} approach, by first initializing their model with BERT~\cite{devl…
Training Discrete Deep Generative Models via Gapped Straight-Through Estimator
Ting-Han Fan, Ta-Chung Chi, Alexander I. Rudnicky +1
While deep generative models have succeeded in image processing, natural language processing, and reinforcement learning, training that involves discrete random variables remains c…
KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation
Ta-Chung Chi, Ting-Han Fan, Peter J. Ramadge +1
Relative positional embeddings (RPE) have received considerable attention since RPEs effectively model the relative distance among tokens and enable length extrapolation. We propos…
Are you doing what I say? On modalities alignment in ALFRED
Ting-Rui Chiang, Yi-Ting Yeh, Ta-Chung Chi +1
ALFRED is a recently proposed benchmark that requires a model to complete tasks in simulated house environments specified by instructions in natural language. We hypothesize that k…
Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection
Ta-Chung Chi, Alexander I. Rudnicky
Dialogue disentanglement aims to group utterances in a long and multi-participant dialogue into threads. This is useful for discourse analysis and downstream applications such as d…
Automatic Speech Verification Spoofing Detection
Shentong Mo, Haofan Wang, Pinxu Ren +1
Automatic speech verification (ASV) is the technology to determine the identity of a person based on their voice. While being convenient for identity verification, we should aim fo…
Just Ask:An Interactive Learning Framework for Vision and Language Navigation
Ta-Chung Chi, Mihail Eric, Seokhwan Kim +2
In the vision and language navigation task, the agent may encounter ambiguous situations that are hard to interpret by just relying on visual information and natural language instr…
BCWS: Bilingual Contextual Word Similarity
Ta-Chung Chi, Ching-Yen Shih, Yun-Nung Chen
This paper introduces the first dataset for evaluating English-Chinese Bilingual Contextual Word Similarity, namely BCWS (https://github.com/MiuLab/BCWS). The dataset consists of 2…
CLUSE: Cross-Lingual Unsupervised Sense Embeddings
Ta-Chung Chi, Yun-Nung Chen
This paper proposes a modularized sense induction and representation learning model that jointly learns bilingual sense embeddings that align well in the vector space, where the cr…
xSense: Learning Sense-Separated Sparse Representations and Textual Definitions for Explainable Word Sense Networks
Ting-Yun Chang, Ta-Chung Chi, Shang-Chi Tsai +1
Despite the success achieved on various natural language processing tasks, word embeddings are difficult to interpret due to the dense vector representations. This paper focuses on…
Dynamic Time-Aware Attention to Speaker Roles and Contexts for Spoken Language Understanding
Po-Chun Chen, Ta-Chung Chi, Shang-Yu Su +1
Spoken language understanding (SLU) is an essential component in conversational systems. Most SLU component treats each utterance independently, and then the following components a…
Speaker Role Contextual Modeling for Language Understanding and Dialogue Policy Learning
Ta-Chung Chi, Po-Chun Chen, Shang-Yu Su +1
Language understanding (LU) and dialogue policy learning are two essential components in conversational systems. Human-human dialogues are not well-controlled and often random and…