27 citations · 85 across the 17 of their papers we have counts for
18 papers
Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
Rylan Schaeffer, Joshua Kazdan, Yegor Denisov-Blanch +11
Science progresses by iteratively advancing and correcting humanity's understanding of the world. In machine learning (ML) research, rapid advancements have led to an explosion of…
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Ji-Ung Lee, Haritz Puerto, Betty van Aken +8
Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters. Large model sizes makes computational cos…
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
Zheng-Xin Yong, Ruochen Zhang, Jessica Zosa Forde +12
While code-mixing is a common linguistic practice in many parts of the world, collecting high-quality and low-cost code-mixed data remains a challenge for natural language processi…
Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex
Charles Lovering, Jessica Zosa Forde, George Konidaris +2
AlphaZero, an approach to reinforcement learning that couples neural networks and Monte Carlo tree search (MCTS), has produced state-of-the-art strategies for traditional board gam…
One Venue, Two Conferences: The Separation of Chinese and American Citation Networks
Bingchen Zhao, Yuling Gu, Jessica Zosa Forde +1
At NeurIPS, American and Chinese institutions cite papers from each other's regions substantially less than they cite endogamously. We build a citation graph to quantify this divid…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Reduce, Reuse, Recycle: Improving Training Efficiency with Distillation
Cody Blakeney, Jessica Zosa Forde, Jonathan Frankle +2
Methods for improving the efficiency of deep network training (i.e. the resources required to achieve a given level of model quality) are of immediate benefit to deep learning prac…
Efficient Methods for Natural Language Processing: A Survey
Marcos Treviso, Ji-Ung Lee, Tianchu Ji +19
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance mea…
Strengthening Subcommunities: Towards Sustainable Growth in AI Research
Andi Peng, Jessica Zosa Forde, Yonadav Shavit +1
AI's rapid growth has been felt acutely by scholarly venues, leading to growing pains within the peer review process. These challenges largely center on the inability of specific s…
Does DQN really learn? Exploring adversarial training schemes in Pong
Bowen He, Sreehari Rammohan, Jessica Forde +1
In this work, we study two self-play training schemes, Chainer and Pool, and show they lead to improved agent performance in Atari Pong compared to a standard DQN agent -- trained…
Model Selection's Disparate Impact in Real-World Deep Learning Applications
Jessica Zosa Forde, A. Feder Cooper, Kweku Kwegyir-Aggrey +2
Algorithmic fairness has emphasized the role of biased data in automated decision outcomes. Recently, there has been a shift in attention to sources of bias that implicate fairness…
Hyperparameter Optimization Is Deceiving Us, and How to Stop It
A. Feder Cooper, Yucheng Lu, Jessica Zosa Forde +1
Recent empirical work shows that inconsistent results based on choice of hyperparameter optimization (HPO) configuration are a widespread problem in ML research. When comparing two…
Bespoke vs. Prêt-à-Porter Lottery Tickets: Exploiting Mask Similarity for Trainable Sub-Network Finding
Michela Paganini, Jessica Zosa Forde
The observation of sparse trainable sub-networks within over-parametrized networks - also known as Lottery Tickets (LTs) - has prompted inquiries around their trainability, scaling…
dagger: A Python Framework for Reproducible Machine Learning Experiment Orchestration
Michela Paganini, Jessica Zosa Forde
Many research directions in machine learning, particularly in deep learning, involve complex, multi-stage experiments, commonly involving state-mutating operations acting on models…
Streamlining Tensor and Network Pruning in PyTorch
Michela Paganini, Jessica Forde
In order to contrast the explosion in size of state-of-the-art machine learning models that can be attributed to the empirical advantages of over-parametrization, and due to the ne…
On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks
Michela Paganini, Jessica Forde
We examine how recently documented, fundamental phenomena in deep learning models subject to pruning are affected by changes in the pruning procedure. Specifically, we analyze diff…
Individual predictions matter: Assessing the effect of data ordering in training fine-tuned CNNs for medical imaging
John R. Zech, Jessica Zosa Forde, Michael L. Littman
We reproduced the results of CheXNet with fixed hyperparameters and 50 different random seeds to identify 14 finding in chest radiographs (x-rays). Because CheXNet fine-tunes a pre…
The Scientific Method in the Science of Machine Learning
Jessica Zosa Forde, Michela Paganini
In the quest to align deep learning with the sciences to address calls for rigor, safety, and interpretability in machine learning systems, this contribution identifies key missing…