10 citations · 12 across the 18 of their papers we have counts for
18 papers
From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs
Yi Zhang, Julia Rayz
We introduce a Bloom-aligned framework for measuring educational control in Large Language Models (LLMs): the ability to preserve a task's instructional intent while shifting its c…
A Fuzzy Evaluation of Sentence Encoders on Grooming Risk Classification
Geetanjali Bihani, Julia Rayz
With the advent of social media, children are becoming increasingly vulnerable to the risk of grooming in online settings. Detecting grooming instances in an online conversation po…
Evaluating Language Models on Grooming Risk Estimation Using Fuzzy Theory
Geetanjali Bihani, Tatiana Ringenberg, Julia Rayz
Encoding implicit language presents a challenge for language models, especially in high-risk domains where maintaining high precision is important. Automated detection of online ch…
The Reliability Paradox: Exploring How Shortcut Learning Undermines Language Model Calibration
Geetanjali Bihani, Julia Rayz
The advent of pre-trained language models (PLMs) has enabled significant performance gains in the field of natural language processing. However, recent studies have found PLMs to s…
Hire Me or Not? Examining Language Model's Behavior with Occupation Attributes
Damin Zhang, Yi Zhang, Geetanjali Bihani +1
With the impressive performance in various downstream tasks, large language models (LLMs) have been widely integrated into production pipelines, like recruitment and recommendation…
Learning Shortcuts: On the Misleading Promise of NLU in Language Models
Geetanjali Bihani, Julia Taylor Rayz
The advent of large language models (LLMs) has enabled significant performance gains in the field of natural language processing. However, recent studies have found that LLMs often…
Calibration Error Estimation Using Fuzzy Binning
Geetanjali Bihani, Julia Taylor Rayz
Neural network-based decisions tend to be overconfident, where their raw outcome probabilities do not align with the true decision probabilities. Calibration of neural networks is…
Counteracts: Testing Stereotypical Representation in Pre-trained Language Models
Damin Zhang, Julia Rayz, Romila Pradhan
Recently, language models have demonstrated strong performance on various natural language understanding tasks. Language models trained on large human-generated corpus encode not o…
COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models
Kanishka Misra, Julia Taylor Rayz, Allyson Ettinger
A characteristic feature of human semantic cognition is its ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate th…
A Property Induction Framework for Neural Language Models
Kanishka Misra, Julia Taylor Rayz, Allyson Ettinger
To what extent can experience from language contribute to our conceptual knowledge? Computational explorations of this question have shed light on the ability of powerful neural la…
On Information Hiding in Natural Language Systems
Geetanjali Bihani, Julia Taylor Rayz
With data privacy becoming more of a necessity than a luxury in today's digital world, research on more robust models of privacy preservation and information security is on the ris…
Do language models learn typicality judgments from text?
Kanishka Misra, Allyson Ettinger, Julia Taylor Rayz
Building on research arguing for the possibility of conceptual and categorical knowledge acquisition through statistics contained in language, we evaluate predictive language model…
Low Anisotropy Sense Retrofitting (LASeR) : Towards Isotropic and Sense Enriched Representations
Geetanjali Bihani, Julia Taylor Rayz
Contextual word representation models have shown massive improvements on a multitude of NLP tasks, yet their word sense disambiguation capabilities remain poorly explained. To addr…
Fuzzy Classification of Multi-intent Utterances
Geetanjali Bihani, Julia Taylor Rayz
Current intent classification approaches assign binary intent class memberships to natural language utterances while disregarding the inherent vagueness in language and the corresp…
Finding Fuzziness in Neural Network Models of Language Processing
Kanishka Misra, Julia Taylor Rayz
Humans often communicate by using imprecise language, suggesting that fuzzy concepts with unclear boundaries are prevalent in language use. In this paper, we test the extent to whi…
Modeling Fuzzy Cluster Transitions for Topic Tracing
Xiaonan Jing, Yi Zhang, Qingyuan Hu +1
Twitter can be viewed as a data source for Natural Language Processing (NLP) tasks. The continuously updating data streams on Twitter make it challenging to trace real-time topic e…
Tracing Topic Transitions with Temporal Graph Clusters
Xiaonan Jing, Qingyuan Hu, Yi Zhang +1
Twitter serves as a data source for many Natural Language Processing (NLP) tasks. It can be challenging to identify topics on Twitter due to continuous updating data stream. In thi…
Exploring Lexical Irregularities in Hypothesis-Only Models of Natural Language Inference
Qingyuan Hu, Yi Zhang, Kanishka Misra +1
Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE) is the task of predicting the entailment relation between a pair of sentences (premise and hypothesis). Thi…