331 citations · 909 across the 15 of their papers we have counts for
16 papers
D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation
Aida Mostafazadeh Davani, Mark Díaz, Dylan Baker +1
While human annotations play a crucial role in language technologies, annotator subjectivity has long been overlooked in data collection. Recent studies that have critically examin…
Discipline and Label: A WEIRD Genealogy and Social Theory of Data Annotation
Andrew Smart, Ding Wang, Ellis Monk +4
Data annotation remains the sine qua non of machine learning and AI. Recent empirical work on data annotation has begun to highlight the importance of rater diversity for fairness,…
Disentangling Perceptions of Offensiveness: Cultural and Moral Correlates
Aida Davani, Mark Díaz, Dylan Baker +1
Perception of offensiveness is inherently subjective, shaped by the lived experiences and socio-cultural values of the perceivers. Recent years have seen substantial efforts to bui…
SoUnD Framework: Analyzing (So)cial Representation in (Un)structured (D)ata
Mark Díaz, Sunipa Dev, Emily Reif +2
The unstructured nature of data used in foundation model development is a challenge to systematic analyses for making data use and documentation decisions. From a Responsible AI pe…
GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives
Vinodkumar Prabhakaran, Christopher Homan, Lora Aroyo +6
Human annotation plays a core role in machine learning -- annotations for supervised models, safety guardrails for generative models, and human feedback for reinforcement learning,…
A Joint Fermi-GBM and Swift-BAT Analysis of Gravitational-Wave Candidates from the Third Gravitational-wave Observing Run
C. Fletcher, J. Wood, R. Hamburg +1696
We present Fermi Gamma-ray Burst Monitor (Fermi-GBM) and Swift Burst Alert Telescope (Swift-BAT) searches for gamma-ray/X-ray counterparts to gravitational wave (GW) candidate even…
Search for Eccentric Black Hole Coalescences during the Third Observing Run of LIGO and Virgo
The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration +1772
Despite the growing number of confident binary black hole coalescences observed through gravitational waves so far, the astrophysical origin of these binaries remains uncertain. Or…
PaLM 2 Technical Report
Rohan Anil, Andrew M. Dai, Orhan Firat +125
We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 i…
Search for gravitational-lensing signatures in the full third observing run of the LIGO-Virgo network
The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration +1692
Gravitational lensing by massive objects along the line of sight to the source causes distortions of gravitational wave-signals; such distortions may reveal information about funda…
(Re)Defining Expertise in Machine Learning Development
Mark Díaz, Angela D. R. Smith
Domain experts are often engaged in the development of machine learning systems in a variety of ways, such as in data collection and evaluation of system performance. At the same t…
Open data from the third observing run of LIGO, Virgo, KAGRA and GEO
The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration +1741
The global network of gravitational-wave observatories now includes five detectors, namely LIGO Hanford, LIGO Livingston, Virgo, KAGRA, and GEO 600. These detectors collected data…
Power to the People? Opportunities and Challenges for Participatory AI
Abeba Birhane, William Isaac, Vinodkumar Prabhakaran +4
Participatory approaches to artificial intelligence (AI) and machine learning (ML) are gaining momentum: the increased attention comes partly with the view that participation opens…
Search for gravitational waves from Scorpius X-1 with a hidden Markov model in O3 LIGO data
The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration +1669
Results are presented for a semi-coherent search for continuous gravitational waves from the low-mass X-ray binary Scorpius X-1, using a hidden Markov model (HMM) to allow for spin…
Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation
Remi Denton, Mark Díaz, Ian Kivlichan +2
Human annotations play a crucial role in machine learning (ML) research and development. However, the ethical considerations around the processes and decisions that go into buildin…
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
Aida Mostafazadeh Davani, Mark Díaz, Vinodkumar Prabhakaran
Majority voting and averaging are common approaches employed to resolve annotator disagreements and derive single ground truth labels from multiple annotations. However, annotators…
On Releasing Annotator-Level Labels and Information in Datasets
Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, Mark Díaz
A common practice in building NLP datasets, especially using crowd-sourced annotations, involves obtaining multiple annotator judgements on the same data instances, which are then…