543 citations · 612 across the 34 of their papers we have counts for
40 papers
What Information Buys in Active Inference
Melih Kandemir, Mustafa Mert Çelikok
Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscience, while the established RL…
Stein Variational Distributional Reinforcement Learning: Score-Guided Particles for Multivariate Return Distributions
Manuel Haußmann, Melih Kandemir
Decisions that weigh several reward signals against each other require the joint distribution of the return vector. Categorical critics scale exponentially in the reward dimension,…
Let CSP Be Your ANCHOR: Adaptive Crystal Search over Frozen Structure Priors
Emma Lei Hovmand, Jonas Elsborg, Melih Kandemir +1
De novo crystal generation (DNG) models decide where to search in composition space and how to generate structures with one set of weights. We argue that discovery is better served…
Hitting Time Isomorphism for Multi-Stage Planning with Foundation Policies
Magnus Victor Boock, Abdullah Akgül, Mustafa Mert Çelikok +1
We present a new operator-theoretic representation learning framework for offline reinforcement learning that recovers the directed temporal geometry of a controlled Markov process…
Fitted Q-Iteration with Continuous Actions and Adaptive Data: Smoothed Coverage and Minimax Rates
Manuel Haußmann, Manuel Haussmann, Mustafa Mert Çelikok +1
Fitted Q-iteration (FQI) is well understood when the data come from a fixed distribution and the action set is finite, whereas value-based continuous control collects its data adap…
Policy-based Tuning of Autoregressive Image Models with Instance- and Distribution-Level Rewards
Orhun Bugra Baran, Melih Kandemir, Ramazan Gokberk Cinbis
Autoregressive (AR) models are highly effective for image generation, yet their standard maximum-likelihood estimation training lacks direct optimization for sample quality and div…
Distributional Active Inference
Abdullah Akgül, Gulcin Baykal, Manuel Haußmann +2
Optimal control of complex environments with robotic systems faces two complementary and intertwined challenges: efficient organization of sensory state information and far-sighted…
ObjectRL: An Object-Oriented Reinforcement Learning Codebase
Gulcin Baykal, Abdullah Akgül, Manuel Haussmann +4
ObjectRL is an open-source Python codebase for deep reinforcement learning (RL), designed for research-oriented prototyping with minimal programming effort. Unlike existing codebas…
Adaptive Ensemble Aggregation for Actor-Critics
Nicklas Werge, Yi-Shan Wu, Manuel Haussmann +2
Ensembles are ubiquitous in off-policy actor-critic learning, yet their efficacy depends critically on how they are aggregated. Current methods typically rely on static rules or ta…
Deep Actor-Critics with Tight Risk Certificates
Bahareh Tasdighi, Manuel Haussmann, Yi-Shan Wu +2
Deep actor-critic algorithms have reached a level where they influence everyday life. They are a driving force behind continual improvement of large language models through user fe…
Overcoming Non-stationary Dynamics with Evidential Proximal Policy Optimization
Abdullah Akgül, Gulcin Baykal, Manuel Haußmann +1
Continuous control of non-stationary environments is a major challenge for deep reinforcement learning algorithms. The time-dependency of the state transition dynamics aggravates t…
In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
Amelia Jiménez-Sánchez, Natalia-Rozalia Avlona, Sarah de Boer +26
Datasets play a critical role in medical imaging research, yet issues such as label quality, shortcuts, and metadata are often overlooked. This lack of attention may harm the gener…
Disentanglement with Factor Quantized Variational Autoencoders
Gulcin Baykal, Melih Kandemir, Gozde Unal
Disentangled representation learning aims to represent the underlying generative factors of a dataset in a latent representation independently of one another. In our work, we propo…
Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learning
Abdullah Akgül, Manuel Haußmann, Melih Kandemir
Current approaches to model-based offline reinforcement learning often incorporate uncertainty-based reward penalization to address the distributional shift problem. These approach…
Improving Actor-Critic Training with Steerable Action-Value Approximation Errors
Bahareh Tasdighi, Nicklas Werge, Yi-Shan Wu +1
Off-policy actor-critic algorithms have shown strong potential in deep reinforcement learning for continuous control tasks. Their success primarily comes from leveraging pessimisti…
Calibrating Bayesian UNet++ for Sub-Seasonal Forecasting
Busra Asan, Abdullah Akgül, Alper Unal +2
Seasonal forecasting is a crucial task when it comes to detecting the extreme heat and colds that occur due to climate change. Confidence in the predictions should be reliable sinc…
Deep Exploration with PAC-Bayes
Bahareh Tasdighi, Manuel Haussmann, Nicklas Werge +2
Reinforcement learning (RL) for continuous control under delayed rewards is an under-explored problem despite its significance in real-world applications. Many complex skills are b…
Demystifying the Myths and Legends of Nonconvex Convergence of SGD
Aritra Dutta, El Houcine Bergou, Soumia Boucherouite +3
Stochastic gradient descent (SGD) and its variants are the main workhorses for solving large-scale optimization problems with nonconvex objective functions. Although the convergenc…
EdVAE: Mitigating Codebook Collapse with Evidential Discrete Variational Autoencoders
Gulcin Baykal, Melih Kandemir, Gozde Unal
Codebook collapse is a common problem in training deep generative models with discrete representation spaces like Vector Quantized Variational Autoencoders (VQ-VAEs). We observe th…
Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixtures
Hamish Flynn, David Reeb, Melih Kandemir +1
We present improved algorithms with worst-case regret guarantees for the stochastic linear bandit problem. The widely used "optimism in the face of uncertainty" principle reduces a…
Estimation of Counterfactual Interventions under Uncertainties
Juliane Weilbach, Sebastian Gerwinn, Melih Kandemir +1
Counterfactual analysis is intuitively performed by humans on a daily basis eg. "What should I have done differently to get the loan approved?". Such counterfactual questions also…
Sampling-Free Probabilistic Deep State-Space Models
Andreas Look, Melih Kandemir, Barbara Rakitsch +1
Many real-world dynamical systems can be described as State-Space Models (SSMs). In this formulation, each observation is emitted by a latent state, which follows first-order Marko…
Weighted Sequential Bayesian Inference for Non-Stationary Linear Contextual Bandits
Nicklas Werge, Yi-Shan Wu, Abdullah Akgül +1
In non-stationary linear contextual bandits, existing efficient algorithms typically rely on the Weighted Regularized Least-Squares (WRLS) estimator. Because WRLS only provides poi…
Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systems
Andreas Look, Melih Kandemir, Barbara Rakitsch +1
Graph neural networks are often used to model interacting dynamical systems since they gracefully scale to systems with a varying and high number of agents. While there has been mu…