14 citations · 54 across the 21 of their papers we have counts for
21 papers
The Minimax Rate of Perturbed Second-Order Calibration
Kamil Ciosek, Banafsheh Rafiee, Sina Ghiassian +1
Second-order calibration error quantifies how closely a higher-order predictor's epistemic-uncertainty estimate matches the conditional variance of the label probability on its lev…
Measuring Uncertainty Calibration
Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian +6
We make two contributions to the problem of estimating the calibration error of a binary classifier from a finite dataset. First, we provide an upper bound for any classifier…
Hallucination Detection on a Budget: Efficient Bayesian Estimation of Semantic Entropy
Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian
Detecting whether an LLM hallucinates is an important research challenge. One promising way of doing so is to estimate the semantic entropy (Farquhar et al., 2024) of the distribut…
Learning in complex action spaces without policy gradients
Arash Tavakoli, Sina Ghiassian, Nemanja Rakićević
While conventional wisdom holds that policy gradient methods are better suited to complex action spaces than action-value methods, foundational work has shown that the two paradigm…
Soft Preference Optimization: Aligning Language Models to Expert Distributions
Arsalan Sharifnassab, Saber Salehkaleybar, Sina Ghiassian +2
We propose Soft Preference Optimization (SPO), a method for aligning generative models, such as Large Language Models (LLMs), with human preferences, without the need for a reward…
On the Importance of Uncertainty in Decision-Making with Large Language Models
Nicolò Felicioni, Lucas Maystre, Sina Ghiassian +1
We investigate the role of uncertainty in decision-making problems with natural language as input. For such tasks, using Large Language Models as agents has become the norm. Howeve…
In-context Exploration-Exploitation for Reinforcement Learning
Zhenwen Dai, Federico Tomasi, Sina Ghiassian
In-context learning is a promising approach for online policy learning of offline reinforcement learning (RL) methods, which can be achieved at inference time without gradient opti…
Auxiliary task discovery through generate-and-test
Banafsheh Rafiee, Sina Ghiassian, Jun Jin +3
In this paper, we explore an approach to auxiliary task discovery in reinforcement learning based on ideas from representation learning. Auxiliary tasks tend to improve data effici…
Importance Sampling Placement in Off-Policy Temporal-Difference Methods
Eric Graves, Sina Ghiassian
A central challenge to applying many off-policy reinforcement learning algorithms to real world problems is the variance introduced by importance sampling. In off-policy learning,…
An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment
Sina Ghiassian, Richard S. Sutton
Many off-policy prediction learning algorithms have been proposed in the past decade, but it remains unclear which algorithms learn faster than others. We empirically compare 11 of…
An Empirical Comparison of Off-policy Prediction Learning Algorithms on the Collision Task
Sina Ghiassian, Richard S. Sutton
Off-policy prediction -- learning the value function for one policy from data generated while following another policy -- is one of the most challenging subproblems in reinforcemen…
A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning
Andrew Patterson, Adam White, Martha White
Many reinforcement learning algorithms rely on value estimation, however, the most widely used algorithms -- namely temporal difference algorithms -- can diverge under both off-pol…
Does the Adam Optimizer Exacerbate Catastrophic Forgetting?
Dylan R. Ashley, Sina Ghiassian, Richard S. Sutton
Catastrophic forgetting remains a severe hindrance to the broad application of artificial neural networks (ANNs), however, it continues to be a poorly understood phenomenon. Despit…
From Eye-blinks to State Construction: Diagnostic Benchmarks for Online Representation Learning
Banafsheh Rafiee, Zaheer Abbas, Sina Ghiassian +4
We present three new diagnostic prediction problems inspired by classical-conditioning experiments to facilitate research in online prediction learning. Experiments in classical co…
Gradient Temporal-Difference Learning with Regularized Corrections
Sina Ghiassian, Andrew Patterson, Shivam Garg +3
It is still common to use Q-learning and temporal difference (TD) learning-even though they have divergence issues and sound Gradient TD alternatives exist-because divergence seems…
Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks
Sina Ghiassian, Banafsheh Rafiee, Yat Long Lo +1
Reinforcement learning systems require good representations to work well. For decades practical success in reinforcement learning was limited to small domains. Deep reinforcement l…
Overcoming Catastrophic Interference in Online Reinforcement Learning with Dynamic Self-Organizing Maps
Yat Long Lo, Sina Ghiassian
Using neural networks in the reinforcement learning (RL) framework has achieved notable successes. Yet, neural networks tend to forget what they learned in the past, especially whe…
Should All Temporal Difference Learning Use Emphasis?
Xiang Gu, Sina Ghiassian, Richard S. Sutton
Emphatic Temporal Difference (ETD) learning has recently been proposed as a convergent off-policy learning method. ETD was proposed mainly to address convergence issues of conventi…
Online Off-policy Prediction
Sina Ghiassian, Andrew Patterson, Martha White +2
This paper investigates the problem of online prediction learning, where learning proceeds continuously as the agent interacts with an environment. The predictions made by the agen…
Two geometric input transformation methods for fast online reinforcement learning with neural nets
Sina Ghiassian, Huizhen Yu, Banafsheh Rafiee +1
We apply neural nets with ReLU gates in online reinforcement learning. Our goal is to train these networks in an incremental manner, without the computationally expensive experienc…
A First Empirical Study of Emphatic Temporal Difference Learning
Sina Ghiassian, Banafsheh Rafiee, Richard S. Sutton
In this paper we present the first empirical study of the emphatic temporal-difference learning algorithm (ETD), comparing it with conventional temporal-difference learning, in par…