38 citations · 88 across the 15 of their papers we have counts for
16 papers
Efficient Offline Learning of Ranking Policies via Top- Policy Decomposition
Ren Kishimoto, Koichi Tanaka, Haruka Kiyohara +4
Many recommender systems such as for e-commerce and news platforms aim to provide users with rankings they are likely to interact with. Off-Policy Learning (OPL) of ranking policie…
Credit-assigned Policy Gradient for Early Stage Retrieval in Two-stage Ranking
Haruka Kiyohara, Mihaela Curmei, Ariel Evnine +7
Large-scale search, recommendation, and retrieval-augmented generation (RAG) systems typically employ a two-stage architecture: an early-stage ranker (ESR) generates a candidate se…
Safely Exploring Novel Actions in Recommender Systems via Deployment-Efficient Policy Learning
Haruka Kiyohara, Yusuke Narita, Yuta Saito +2
In many real recommender systems, novel items are added frequently over time. The importance of sufficiently presenting novel actions has widely been acknowledged for improving lon…
Prompt Optimization with Logged Bandit Data
Haruka Kiyohara, Daniel Yiming Cao, Yuta Saito +1
We study how to use naturally available user feedback, such as clicks, to optimize large language model (LLM) pipelines for generating personalized sentences using prompts. Naive a…
Policy Design for Two-sided Platforms with Participation Dynamics
Haruka Kiyohara, Fan Yao, Sarah Dean
In two-sided platforms (e.g., video streaming or e-commerce), viewers and providers engage in interactive dynamics: viewers benefit from increases in provider populations, while pr…
Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Bandits
Tatsuhiro Shimizu, Koichi Tanaka, Ren Kishimoto +3
We explore off-policy evaluation and learning (OPE/L) in contextual combinatorial bandits (CCB), where a policy selects a subset in the action space. For example, it might choose a…
Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstraction
Haruka Kiyohara, Masahiro Nomura, Yuta Saito
We study off-policy evaluation (OPE) in the problem of slate contextual bandits where a policy selects multi-dimensional actions known as slates. This problem is widespread in reco…
Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluation
Haruka Kiyohara, Ren Kishimoto, Kosuke Kawakami +3
Off-Policy Evaluation (OPE) aims to assess the effectiveness of counterfactual policies using only offline logged data and is often used to identify the top-k promising policies fo…
SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
Haruka Kiyohara, Ren Kishimoto, Kosuke Kawakami +3
This paper introduces SCOPE-RL, a comprehensive open-source Python software designed for offline reinforcement learning (offline RL), off-policy evaluation (OPE), and selection (OP…
Off-Policy Evaluation of Ranking Policies under Diverse User Behavior
Haruka Kiyohara, Masatoshi Uehara, Yusuke Narita +3
Ranking interfaces are everywhere in online platforms. There is thus an ever growing interest in their Off-Policy Evaluation (OPE), aiming towards an accurate performance evaluatio…
Policy-Adaptive Estimator Selection for Off-Policy Evaluation
Takuma Udagawa, Haruka Kiyohara, Yusuke Narita +2
Off-policy evaluation (OPE) aims to accurately evaluate the performance of counterfactual policies using only offline logged data. Although many estimators have been developed, the…
Future-Dependent Value-Based Off-Policy Evaluation in POMDPs
Masatoshi Uehara, Haruka Kiyohara, Andrew Bennett +5
We study off-policy evaluation (OPE) for partially observable MDPs (POMDPs) with general function approximation. Existing methods such as sequential importance sampling estimators…
Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model
Haruka Kiyohara, Yuta Saito, Tatsuya Matsuhiro +3
In real-world recommender systems and search engines, optimizing ranking decisions to present a ranked list of relevant items is critical. Off-policy evaluation (OPE) for ranking p…
Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation
Haruka Kiyohara, Kosuke Kawakami, Yuta Saito
In recommender systems (RecSys) and real-time bidding (RTB) for online advertisements, we often try to optimize sequential decision making using bandit and reinforcement learning (…
Evaluating the Robustness of Off-Policy Evaluation
Yuta Saito, Takuma Udagawa, Haruka Kiyohara +3
Off-policy Evaluation (OPE), or offline evaluation in general, evaluates the performance of hypothetical policies leveraging only offline log data. It is particularly useful in app…
Constrained Generalized Additive 2 Model with Consideration of High-Order Interactions
Akihisa Watanabe, Michiya Kuramata, Kaito Majima +3
In recent years, machine learning and AI have been introduced in many industrial fields. In fields such as finance, medicine, and autonomous driving, where the inference results of…