1 citations · 2 across the 9 of their papers we have counts for
9 papers
Aligning Language Model Benchmarks with Pairwise Preferences
Marco Gutierrez, Xinyi Leng, Hannah Cyberey +3
Language model benchmarks are pervasive and computationally-efficient proxies for real-world downstream performance. However, many recent works find that benchmarks often fail to p…
White-Box Sensitivity Auditing with Steering Vectors
Hannah Cyberey, Yangfeng Ji, David Evans
Algorithmic audits are essential tools for examining systems for properties required by regulators or desired by operators. Current audits of large language models (LLMs) primarily…
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control
Hannah Cyberey, David Evans
Large language models (LLMs) have transformed the way we access information. These models are often tuned to refuse to comply with requests that are considered harmful and to produ…
Unsupervised Concept Vector Extraction for Bias Control in LLMs
Hannah Cyberey, Yangfeng Ji, David Evans
Large language models (LLMs) are known to perpetuate stereotypes and exhibit biases. Various strategies have been proposed to mitigate these biases, but most work studies biases as…
Do Prevalent Bias Metrics Capture Allocational Harms from LLMs?
Hannah Cyberey, Yangfeng Ji, David Evans
Allocational harms occur when resources or opportunities are unfairly withheld from specific groups. Many proposed bias measures ignore the discrepancy between predictions, which a…
Addressing Both Statistical and Causal Gender Fairness in NLP Models
Hannah Chen, Yangfeng Ji, David Evans
Statistical fairness stipulates equivalent outcomes for every protected group, whereas causal fairness prescribes that a model makes the same prediction for an individual regardles…
Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models
Hannah Chen, Yangfeng Ji, David Evans
Traditional (fickle) adversarial examples involve finding a small perturbation that does not change an input's true label but confuses the classifier into outputting a different pr…
Finding Friends and Flipping Frenemies: Automatic Paraphrase Dataset Augmentation Using Graph Theory
Hannah Chen, Yangfeng Ji, David Evans
Most NLP datasets are manually labeled, so suffer from inconsistent labeling or limited size. We propose methods for automatically improving datasets by viewing them as graphs with…
Pointwise Paraphrase Appraisal is Potentially Problematic
Hannah Chen, Yangfeng Ji, David Evans
The prevailing approach for training and evaluating paraphrase identification models is constructed as a binary classification problem: the model is given a pair of sentences, and…