21 citations · 78 across the 33 of their papers we have counts for
34 papers
SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning
Tyler Kastner, Nimrod De La Vega, Amir-massoud Farahmand
Neural networks trained on nonstationary tasks frequently lose the ability to fit new targets, a phenomenon referred to as loss of plasticity. We identify a novel source of plastic…
Press Start to Charge: Videogaming the Online Centralized Charging Scheduling Problem
Alireza Ghahtarani, Martin Cousineau, Amir-massoud Farahmand +1
We study the online centralized charging scheduling problem (OCCSP). In this problem, a central authority must decide, in real time, when to charge dynamically arriving electric ve…
Majority of the Bests: Improving Best-of-N via Bootstrapping
Amin Rakhsha, Kanika Madan, Tianyu Zhang +2
Sampling multiple outputs from a Large Language Model (LLM) and selecting the most frequent (Self-consistency) or highest-scoring (Best-of-N) candidate is a popular approach to ach…
Relative Entropy Pathwise Policy Optimization
Claas Voelcker, Axel Brunnbauer, Marcel Hussing +6
Score-function based methods for policy learning, such as REINFORCE and PPO, have delivered strong results in game-playing and robotics, yet their high variance often undermines tr…
Calibrated Value-Aware Model Learning with Probabilistic Environment Models
Claas Voelcker, Anastasiia Pedan, Arash Ahmadian +3
The idea of value-aware model learning, that models should produce accurate value estimates, has gained prominence in model-based reinforcement learning. The MuZero loss, which pen…
A Truncated Newton Method for Optimal Transport
Mete Kemertas, Amir-massoud Farahmand, Allan D. Jepson
Developing a contemporary optimal transport (OT) solver requires navigating trade-offs among several critical requirements: GPU parallelization, scalability to high-dimensional pro…
PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Sampling
Avery Ma, Yangchen Pan, Amir-massoud Farahmand
Many-shot jailbreaking circumvents the safety alignment of LLMs by exploiting their ability to process long input sequences. To achieve this, the malicious target prompt is prefixe…
MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL
Claas A Voelcker, Marcel Hussing, Eric Eaton +2
Building deep reinforcement learning (RL) agents that find a good policy with few samples has proven notoriously challenging. To achieve sample efficiency, recent work has explored…
Deflated Dynamics Value Iteration
Jongmin Lee, Amin Rakhsha, Ernest K. Ryu +1
The Value Iteration (VI) algorithm is an iterative procedure to compute the value function of a Markov decision process, and is the basis of many reinforcement learning (RL) algori…
PID Accelerated Temporal Difference Algorithms
Mark Bedaywi, Amin Rakhsha, Amir-massoud Farahmand
Long-horizon tasks, which have a large discount factor, pose a challenge for most conventional reinforcement learning (RL) algorithms. Algorithms such as Value Iteration and Tempor…
When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning
Claas Voelcker, Tyler Kastner, Igor Gilitschenski +1
We investigate the impact of auxiliary learning tasks such as observation reconstruction and latent self-prediction on the representation learning problem in reinforcement learning…
Dissecting Deep RL with High Update Ratios: Combatting Value Divergence
Marcel Hussing, Claas Voelcker, Igor Gilitschenski +2
We show that deep reinforcement learning algorithms can retain their ability to learn without resetting network parameters in settings where the number of gradient updates greatly…
Maximum Entropy Model Correction in Reinforcement Learning
Amin Rakhsha, Mete Kemertas, Mohammad Ghavamzadeh +1
We propose and theoretically analyze an approach for planning with an approximate model in reinforcement learning that can reduce the adverse impact of model error. If the model is…
Improving Adversarial Transferability via Model Alignment
Avery Ma, Amir-massoud Farahmand, Yangchen Pan +2
Neural networks are susceptible to adversarial perturbations that are transferable across different models. In this paper, we introduce a novel model alignment technique aimed at i…
Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods
Avery Ma, Yangchen Pan, Amir-massoud Farahmand
Stochastic gradient descent (SGD) and adaptive gradient methods, such as Adam and RMSProp, have been widely used in training deep neural networks. We empirically show that while th…
Efficient and Accurate Optimal Transport with Mirror Descent and Conjugate Gradients
Mete Kemertas, Allan D. Jepson, Amir-massoud Farahmand
We propose Mirror Descent Optimal Transport (MDOT), a novel method for solving discrete optimal transport (OT) problems with high precision, by unifying temperature annealing in en…
Distributional Model Equivalence for Risk-Sensitive Reinforcement Learning
Tyler Kastner, Murat A. Erdogdu, Amir-massoud Farahmand
We consider the problem of learning models for risk-sensitive reinforcement learning. We theoretically demonstrate that proper value equivalence, a method of learning models which…
-models: Effective Decision-Aware Reinforcement Learning with Latent Models
Claas A Voelcker, Arash Ahmadian, Romina Abachi +2
The idea of decision-aware model learning, that models should be accurate where it matters for decision-making, has gained prominence in model-based reinforcement learning. While p…
Operator Splitting Value Iteration
Amin Rakhsha, Andrew Wang, Mohammad Ghavamzadeh +1
We introduce new planning and reinforcement learning algorithms for discounted MDPs that utilize an approximate model of the environment to accelerate the convergence of the value…
Value Gradient weighted Model-Based Reinforcement Learning
Claas Voelcker, Victor Liao, Animesh Garg +1
Model-based reinforcement learning (MBRL) is a sample efficient technique to obtain control policies, yet unavoidable modeling errors often lead performance deterioration. The mode…
The act of remembering: a study in partially observable reinforcement learning
Rodrigo Toro Icarte, Richard Valenzano, Toryn Q. Klassen +3
Reinforcement Learning (RL) agents typically learn memoryless policies---policies that only consider the last observation when selecting actions. Learning memoryless policies is ef…
Understanding and Mitigating the Limitations of Prioritized Experience Replay
Yangchen Pan, Jincheng Mei, Amir-massoud Farahmand +4
Prioritized Experience Replay (ER) has been empirically shown to improve sample efficiency across many domains and attracted great attention; however, there is little theoretical u…
SOAR: Second-Order Adversarial Regularization
Avery Ma, Fartash Faghri, Nicolas Papernot +1
Adversarial training is a common approach to improving the robustness of deep neural networks against adversarial examples. In this work, we propose a novel regularization approach…
Policy-Aware Model Learning for Policy Gradient Methods
Romina Abachi, Mohammad Ghavamzadeh, Amir-massoud Farahmand
This paper considers the problem of learning a model in model-based reinforcement learning (MBRL). We examine how the planning module of an MBRL algorithm uses the model, and propo…