98 citations · 255 across the 21 of their papers we have counts for
5 papers · 1 filter
Evaluating Contextual Embeddings and their Extraction Layers for Depression Assessment
Matthew Matero, Albert Hung, H. Andrew Schwartz
Recent works have demonstrated ability to assess aspects of mental health from personal discourse. At the same time, pre-trained contextual word embedding models have grown to domi…
MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection
Matthew Matero, Nikita Soni, Niranjan Balasubramanian +1
Much of natural language processing is focused on leveraging large capacity language models, typically trained over single messages with a task of predicting one or more tokens. Ho…
On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformers
Tianchu Ji, Shraddhan Jain, Michael Ferdman +3
How much information do NLP tasks really need from a transformer's attention mechanism at application-time (inference)? From recent work, we know that there is sparsity in transfor…
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality
Adithya V Ganesan, Matthew Matero, Aravind Reddy Ravula +2
In human-level NLP tasks, such as predicting mental health, personality, or demographics, the number of observations is often smaller than the standard 768+ hidden state sizes of e…
Discourse Relation Embeddings: Representing the Relations between Discourse Segments in Social Media
Youngseo Son, Vasudha Varadarajan, H Andrew Schwartz
Discourse relations are typically modeled as a discrete class that characterizes the relation between segments of text (e.g. causal explanations, expansions). However, such predefi…