167 citations · 428 across the 84 of their papers we have counts for
11 papers · 1 filter
Disentangling Fact from Grid Cell Fiction in Trained Deep Path Integrators
Rylan Schaeffer, Mikail Khona, Sanmi Koyejo +1
Work on deep learning-based models of grid cells suggests that grid cells generically and robustly arise from optimizing networks to path integrate, i.e., track one's spatial posit…
What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes
Victor Lecomte, Kushal Thaman, Rylan Schaeffer +3
Polysemantic neurons -- neurons that activate for a set of unrelated features -- have been seen as a significant obstacle towards interpretability of task-optimized deep networks,…
Testing Assumptions Underlying a Unified Theory for the Origin of Grid Cells
Rylan Schaeffer, Mikail Khona, Adrian Bertagnoli +2
Representing and reasoning about physical space is fundamental to animal survival, and the mammalian lineage expresses a wealth of specialized neural representations that encode sp…
Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells
Rylan Schaeffer, Mikail Khona, Tzuhsuan Ma +3
To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is…
Representation Engineering: A Top-Down Approach to AI Transparency
Andy Zou, Long Phan, Sarah Chen +18
In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights f…
Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
Brando Miranda, Alycia Lee, Sudharsan Sundar +4
Current trends in pre-training Large Language Models (LLMs) primarily focus on the scaling of model and dataset size. While the quality of pre-training data is considered an import…
Adaptive Compression in Federated Learning via Side Information
Berivan Isik, Francesco Pase, Deniz Gunduz +3
The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL). Among existing approaches, sta…
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Boxin Wang, Weixin Chen, Hengzhi Pei +16
Generative Pre-trained Transformer (GPT) models have exhibited exciting progress in their capabilities, capturing the interest of practitioners and the public alike. Yet, while the…
Pairwise Ranking Losses of Click-Through Rates Prediction for Welfare Maximization in Ad Auctions
Boxiang Lyu, Zhe Feng, Zachary Robertson +1
We study the design of loss functions for click-through rates (CTR) to optimize (social) welfare in advertising auctions. Existing works either only focus on CTR predictions withou…
Globalizing Fairness Attributes in Machine Learning: A Case Study on Health in Africa
Mercy Nyamewaa Asiedu, Awa Dieng, Abigail Oppong +3
With growing machine learning (ML) applications in healthcare, there have been calls for fairness in ML to understand and mitigate ethical concerns these systems may pose. Fairness…
Longitudinal assessment of demographic representativeness in the Medical Imaging and Data Resource Center Open Data Commons
Heather M. Whitney, Natalie Baughan, Kyle J. Myers +11
Purpose: The Medical Imaging and Data Resource Center (MIDRC) open data commons was launched to accelerate the development of artificial intelligence (AI) algorithms to help addres…