519 citations · 4.1k across the 233 of their papers we have counts for
22 papers · 1 filter
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1
With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associatio…
Cross-Modal Fine-Tuning: Align then Refine
Junhong Shen, Liam Li, Lucio M. Dery +4
Fine-tuning large-scale pretrained models has led to tremendous progress in well-studied modalities such as vision and NLP. However, similar gains have not been observed in many ot…
Mega: Moving Average Equipped Gated Attention
Xuezhe Ma, Chunting Zhou, Xiang Kong +5
The design choices in the Transformer attention mechanism, including weak inductive bias and quadratic computational complexity, have limited its application for modeling long sequ…
AANG: Automating Auxiliary Learning
Lucio M. Dery, Paul Michel, Mikhail Khodak +2
Auxiliary objectives, supplementary learning signals that are introduced to help aid learning on data-starved or highly complex end-tasks, are commonplace in machine learning. Whil…
Distributionally Robust Models with Parametric Likelihood Ratios
Paul Michel, Tatsunori Hashimoto, Graham Neubig
As machine learning models are deployed ever more broadly, it becomes increasingly important that they are not only able to perform well on their training distribution, but also yi…
Learning to Scaffold: Optimizing Model Explanations for Teaching
Patrick Fernandes, Marcos Treviso, Danish Pruthi +2
Modern machine learning models are opaque, and as a result there is a burgeoning academic subfield on methods that explain these models' behavior. However, what is the precise goal…
DataLab: A Platform for Data Analysis and Intervention
Yang Xiao, Jinlan Fu, Weizhe Yuan +5
Despite data's crucial role in machine learning, most existing tools and research tend to focus on systems on top of existing data rather than how to interpret and manipulate data.…
Learning to Superoptimize Real-world Programs
Alex Shypula, Pengcheng Yin, Jeremy Lacomis +3
Program optimization is the process of modifying software to execute more efficiently. Superoptimizers attempt to find the optimal program by employing significantly more expensive…
Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative
Lucio M. Dery, Paul Michel, Ameet Talwalkar +1
In most settings of practical concern, machine learning practitioners know in advance what end-task they wish to boost with auxiliary tasks. However, widely used methods for levera…
Examining and Combating Spurious Features under Distribution Shift
Chunting Zhou, Xuezhe Ma, Paul Michel +1
A central goal of machine learning is to learn robust representations that capture the causal relationship between inputs features and output labels. However, minimizing empirical…
Modeling the Second Player in Distributionally Robust Optimization
Paul Michel, Tatsunori Hashimoto, Graham Neubig
Distributionally robust optimization (DRO) provides a framework for training machine learning models that are able to perform well on a collection of related data distributions (th…
Learning Structural Edits via Incremental Tree Transformations
Ziyu Yao, Frank F. Xu, Pengcheng Yin +2
While most neural generative models generate outputs in a single pass, the human creative process is usually one of iterative building and refinement. Recent work has proposed mode…
Weight Poisoning Attacks on Pre-trained Models
Keita Kurita, Paul Michel, Graham Neubig
Recently, NLP has seen a surge in the usage of large pre-trained models. Users download weights of models pre-trained on large datasets, then fine-tune the weights on a task of the…
Optimizing Data Usage via Differentiable Rewards
Xinyi Wang, Hieu Pham, Paul Michel +3
To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content…
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
Bohan Li, Junxian He, Graham Neubig +2
When trained effectively, the Variational Autoencoder (VAE) is both a powerful language model and an effective representation learning framework. In practice, however, VAEs are tra…
Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
Junxian He, Daniel Spokoyny, Graham Neubig +1
The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique. By using a neural inference network to app…
Learning to Represent Edits
Pengcheng Yin, Graham Neubig, Miltiadis Allamanis +2
We introduce the problem of learning distributed representations of edits. By combining a "neural editor" with an "edit encoder", our models learn to represent the salient informat…
Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks
Hao Zhang, Shizhen Xu, Graham Neubig +4
Recent deep learning (DL) models have moved beyond static network architectures to dynamic ones, handling data where the network structure changes every example, such as sequences…
A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models
Kartik Goyal, Graham Neubig, Chris Dyer +1
Beam search is a desirable choice of test-time decoding algorithm for neural sequence models because it potentially avoids search errors made by simpler greedy methods. However, ty…
On-the-fly Operation Batching in Dynamic Computation Graphs
Graham Neubig, Yoav Goldberg, Chris Dyer
Dynamic neural network toolkits such as PyTorch, DyNet, and Chainer offer more flexibility for implementing models that cope with data of varying dimensions and structure, relative…
Controllable Invariance through Adversarial Feature Learning
Qizhe Xie, Zihang Dai, Yulun Du +2
Learning meaningful representations that maintain the content necessary for a particular task while filtering away detrimental variations is a problem of great interest in machine…
Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML
Xuezhe Ma, Pengcheng Yin, Jingzhou Liu +2
Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a…