106 citations · 169 across the 23 of their papers we have counts for
24 papers · 1 filter
(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement
Zach Studdiford, Kanishka Misra
Language models learn about grammatical number primarily from co-occurrence, and show frequency effects as a result---sometimes taken to indicate that they do not learn abstract ``…
Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions
Wesley Scivetti, Ethan Wilcox, Nathan Schneider +2
Grasping the semantics of rare constructions (form-meaning pairings) has been shown to be a challenging problem that has currently only been solved by the largest LLMs. It remains…
Cross-Modal Taxonomic Generalization in (Vision-) Language Models
Tianyang Xu, Marcelo Sandoval-Castaneda, Karen Livescu +2
What is the interplay between semantic representations learned by language models (LM) from surface form alone to those learned from more grounded evidence? We study this question…
Targeted Linguistic Analysis of Sign Language Models with Minimal Translation Pairs
Serpil Karabüklü, Kanishka Misra, Shester Gueuwou +3
Models of sign language have historically lagged behind those for spoken language (text and speech). Recent work has greatly improved their performance on tasks like sign language…
Hey, wait a minute: on at-issue sensitivity in Language Models
Sanghee J. Kim, Kanishka Misra
Evaluating the naturalness of dialogue in language models (LMs) is not trivial: notions of 'naturalness' vary, and scalable quantitative metrics remain limited. This study leverage…
Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It
Yulu Qin, Dheeraj Varghese, Adam Dahlgren Lindström +3
Does vision-and-language (VL) training change the linguistic representations of language models in meaningful ways? Most results in the literature have shown inconsistent or margin…
Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs
William Sheffield, Kanishka Misra, Valentina Pyatkin +3
Discourse particles are crucial elements that subtly shape the meaning of text. These words, often polyfunctional, give rise to nuanced and often quite disparate semantic/discourse…
Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models
Qing Yao, Kanishka Misra, Leonie Weissweiler +1
Language models (LMs) tend to show human-like preferences on a number of syntactic phenomena, but the extent to which these are attributable to direct exposure to the phenomena or…
Characterizing the Role of Similarity in the Property Inferences of Language Models
Juan Diego Rodriguez, Aaron Mueller, Kanishka Misra
Property inheritance -- a phenomenon where novel properties are projected from higher level categories (e.g., birds) to lower level ones (e.g., sparrows) -- provides a unique windo…
A systematic framework for generating novel experimental hypotheses from language models
Kanishka Misra, Najoung Kim
Neural language models (LMs) have been shown to capture complex linguistic patterns, yet their utility in understanding human language and more broadly, human cognition, remains de…
Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs
Kanishka Misra, Kyle Mahowald
Language models learn rare syntactic phenomena, but the extent to which this is attributable to generalization vs. memorization is a major open question. To that end, we iterativel…
Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently
Kanishka Misra, Allyson Ettinger, Kyle Mahowald
Recent zero-shot evaluations have highlighted important limitations in the abilities of language models (LMs) to perform meaning extraction. However, it is now well known that LMs…
Abstraction via exemplars? A representational case study on lexical category inference in BERT
Kanishka Misra, Najoung Kim
Exemplar based accounts are often considered to be in direct opposition to pure linguistic abstraction in explaining language learners' ability to generalize to novel expressions.…
Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks
Kanishka Misra, Cicero Nogueira dos Santos, Siamak Shakeri
Despite readily memorizing world knowledge about entities, pre-trained language models (LMs) struggle to compose together two or more facts to perform multi-hop reasoning in questi…
Large Language Models Can Be Easily Distracted by Irrelevant Context
Freda Shi, Xinyun Chen, Kanishka Misra +5
Large language models have achieved impressive performance on various natural language processing tasks. However, so far they have been evaluated primarily on benchmarks where all…
Language model acceptability judgements are not always robust to context
Koustuv Sinha, Jon Gauthier, Aaron Mueller +4
Targeted syntactic evaluations of language models ask whether models show stable preferences for syntactically acceptable content over minimal-pair unacceptable inputs. Most target…
COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models
Kanishka Misra, Julia Taylor Rayz, Allyson Ettinger
A characteristic feature of human semantic cognition is its ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate th…
A Property Induction Framework for Neural Language Models
Kanishka Misra, Julia Taylor Rayz, Allyson Ettinger
To what extent can experience from language contribute to our conceptual knowledge? Computational explorations of this question have shed light on the ability of powerful neural la…
minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models
Kanishka Misra
We present minicons, an open source library that provides a standard API for researchers interested in conducting behavioral and representational analyses of transformer-based lang…
On Semantic Cognition, Inductive Generalization, and Language Models
Kanishka Misra
My doctoral research focuses on understanding semantic knowledge in neural network models trained solely to predict natural language (referred to as language models, or LMs), by dr…
Do language models learn typicality judgments from text?
Kanishka Misra, Allyson Ettinger, Julia Taylor Rayz
Building on research arguing for the possibility of conceptual and categorical knowledge acquisition through statistics contained in language, we evaluate predictive language model…
Finding Fuzziness in Neural Network Models of Language Processing
Kanishka Misra, Julia Taylor Rayz
Humans often communicate by using imprecise language, suggesting that fuzzy concepts with unclear boundaries are prevalent in language use. In this paper, we test the extent to whi…
Exploring Lexical Irregularities in Hypothesis-Only Models of Natural Language Inference
Qingyuan Hu, Yi Zhang, Kanishka Misra +1
Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE) is the task of predicting the entailment relation between a pair of sentences (premise and hypothesis). Thi…
Exploring BERT's Sensitivity to Lexical Cues using Tests from Semantic Priming
Kanishka Misra, Allyson Ettinger, Julia Taylor Rayz
Models trained to estimate word probabilities in context have become ubiquitous in natural language processing. How do these models use lexical cues in context to inform their word…