Publications (77)
Optimizing an Utility Function for Exploration / Exploitation Trade-off in Context-Aware Recommender System
Djallel Bouneffouf
In this paper, we develop a dynamic exploration/ exploitation (exr/exp) strategy for contextual recommender systems (CRS). Specifically, our methods can adaptively balance the two…
Towards User Profile Modelling in Recommender System
Djallel Bouneffouf
The notion of profile appeared in the 1970s decade, which was mainly due to the need to create custom applications that could be adapted to the user. In this paper, we treat the di…
Role of temporal inference in the recognition of textual inference
Djallel Bouneffouf
This project is a part of nature language processing and its aims to develop a system of recognition inference text-appointed TIMINF. This type of system can detect, given two port…
Evaluating the Prompt Steerability of Large Language Models
Erik Miehling, Michael Desmond, Karthikeyan Natesan Ramamurthy +5
Building pluralistic AI requires designing models that are able to be shaped to represent a wide range of value systems and cultures. Achieving this requires first being able to ev…
Utterance Classification with Logical Neural Network: Explainable AI for Mental Disorder Diagnosis
Yeldar Toleubay, Don Joven Agravante, Daiki Kimura +3
In response to the global challenge of mental health problems, we proposes a Logical Neural Network (LNN) based Neuro-Symbolic AI method for the diagnosis of mental disorders. Due…
Towards Healthy AI: Large Language Models Need Therapists Too
Baihan Lin, Djallel Bouneffouf, Guillermo Cecchi +1
Recent advances in large language models (LLMs) have led to the development of powerful AI chatbots capable of engaging in natural and human-like conversations. However, these chat…
COMPASS: Computational Mapping of Patient-Therapist Alliance Strategies with Language Modeling
Baihan Lin, Djallel Bouneffouf, Yulia Landa +3
The therapeutic working alliance is a critical predictor of psychotherapy success. Traditionally, working alliance assessment relies on questionnaires completed by both therapists…
Situation-Aware Approach to Improve Context-based Recommender System
Djallel Bouneffouf
In this paper, we introduce a novel situation aware approach to improve a context based recommender system. To build situation aware user profiles, we rely on evidence issued from…
Multi-armed Bandit Problem with Known Trend
Djallel Bouneffouf, Raphaël Feraud
We consider a variant of the multi-armed bandit model, which we call multi-armed bandit problem with known trend, where the gambler knows the shape of the reward function of each a…
Conversational Topic Recommendation in Counseling and Psychotherapy with Decision Transformer and Large Language Models
Aylin Gunal, Baihan Lin, Djallel Bouneffouf
Given the increasing demand for mental health assistance, artificial intelligence (AI), particularly large language models (LLMs), may be valuable for integration into automated cl…
Non-Stationary Bandits with Auto-Regressive Temporal Dependency
Qinyi Chen, Negin Golrezaei, Djallel Bouneffouf
Traditional multi-armed bandit (MAB) frameworks, predominantly examined under stochastic or adversarial settings, often overlook the temporal dynamics inherent in many real-world a…
Freshness-Aware Thompson Sampling
Djallel Bouneffouf
To follow the dynamicity of the user's content, researchers have recently started to model interactions between users and the Context-Aware Recommender Systems (CARS) as a bandit p…
Survey: Multi-Armed Bandits Meet Large Language Models
Djallel Bouneffouf, Raphael Feraud
Bandit algorithms and Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence, each addressing distinct yet complementary challenges in decision-maki…
Deep Annotation of Therapeutic Working Alliance in Psychotherapy
Baihan Lin, Guillermo Cecchi, Djallel Bouneffouf
The therapeutic working alliance is an important predictor of the outcome of the psychotherapy treatment. In practice, the working alliance is estimated from a set of scoring quest…
Predicting human decision making in psychological tasks with recurrent neural networks
Baihan Lin, Djallel Bouneffouf, Guillermo Cecchi
Unlike traditional time series, the action sequences of human decision making usually involve many cognitive processes such as beliefs, desires, intentions, and theory of mind, i.e…
Beyond Backprop: Online Alternating Minimization with Auxiliary Variables
Anna Choromanska, Benjamin Cowen, Sadhana Kumaravel +8
Despite significant recent advances in deep neural networks, training them remains a challenge due to the highly non-convex nature of the objective function. State-of-the-art metho…
Improving adaptation of ubiquitous recommander systems by using reinforcement learning and collaborative filtering
Djallel Bouneffouf
The wide development of mobile applications provides a considerable amount of data of all types (images, texts, sounds, videos, etc.). Thus, two main issues have to be considered:…
Interpolating Item and User Fairness in Multi-Sided Recommendations
Qinyi Chen, Jason Cheuk Nam Liang, Negin Golrezaei +1
Today's online platforms heavily lean on algorithmic recommendations for bolstering user engagement and driving revenue. However, these recommendations can impact multiple stakehol…
How can AI Automate End-to-End Data Science?
Charu Aggarwal, Djallel Bouneffouf, Horst Samulowitz +9
Data science is labor-intensive and human experts are scarce but heavily involved in every aspect of it. This makes data science time consuming and restricted to experts with the r…
The Impact of Situation Clustering in Contextual-Bandit Algorithm for Context-Aware Recommender Systems
Djallel Bouneffouf
Most existing approaches in Context-Aware Recommender Systems (CRS) focus on recommending relevant items to users taking into account contextual information, such as time, location…
TherapyView: Visualizing Therapy Sessions with Temporal Topic Modeling and AI-Generated Arts
Baihan Lin, Stefan Zecevic, Djallel Bouneffouf +1
We present the TherapyView, a demonstration system to help therapists visualize the dynamic contents of past treatment sessions, enabled by the state-of-the-art neural topic modeli…
Optimal Exploitation of Clustering and History Information in Multi-Armed Bandit
Djallel Bouneffouf, Srinivasan Parthasarathy, Horst Samulowitz +1
We consider the stochastic multi-armed bandit problem and the contextual bandit problem with historical observations and pre-clustered arms. The historical observations can contain…
Contextual Bandit with Adaptive Feature Extraction
Baihan Lin, Djallel Bouneffouf, Guillermo Cecchi +1
We consider an online decision making setting known as contextual bandit problem, and propose an approach for improving contextual bandit performance by using an adaptive feature e…
Split Q Learning: Reinforcement Learning with Two-Stream Rewards
Baihan Lin, Djallel Bouneffouf, Guillermo Cecchi
Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for a reinforcement learning problem, which extends the stan…
Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations
Swapnaja Achintalwar, Adriana Alvarado Garcia, Ateret Anaby-Tavor +35
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations. Due to several limiting factors surrounding LLMs (trai…
Exponentiated Gradient Exploration for Active Learning
Djallel Bouneffouf
Active learning strategies respond to the costly labelling task in a supervised classification by selecting the most useful unlabelled examples in training a predictive model. Many…
Neural Topic Modeling of Psychotherapy Sessions
Baihan Lin, Djallel Bouneffouf, Guillermo Cecchi +1
In this work, we compare different neural topic modeling methods in learning the topical propensities of different psychiatric conditions from the psychotherapy session transcripts…
R-UCB: a Contextual Bandit Algorithm for Risk-Aware Recommender Systems
Djallel Bouneffouf
Mobile Context-Aware Recommender Systems can be naturally modelled as an exploration/exploitation trade-off (exr/exp) problem, where the system has to choose between maximizing its…
The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models
Matthew Riemer, Erik Miehling, Miao Liu +2
Although parameter-efficient fine-tuning methods, such as LoRA, only modify a small subset of parameters, they can have a significant impact on the model. Our instruction-tuning ex…
Learning to Generate Image Source-Agnostic Universal Adversarial Perturbations
Pu Zhao, Parikshit Ram, Songtao Lu +4
Adversarial perturbations are critical for certifying the robustness of deep learning models. A universal adversarial perturbation (UAP) can simultaneously attack multiple images,…
Double-Linear Thompson Sampling for Context-Attentive Bandits
Djallel Bouneffouf, Raphaël Féraud, Sohini Upadhyay +2
In this paper, we analyze and extend an online learning framework known as Context-Attentive Bandit, motivated by various practical applications, from medical diagnosis to dialog s…
Contextual Bandit with Missing Rewards
Djallel Bouneffouf, Sohini Upadhyay, Yasaman Khazaeni
We consider a novel variant of the contextual bandit problem (i.e., the multi-armed bandit with side-information, or context, available to a decision-maker) where the reward associ…
Proposition d'une technique de gestion de projet dans les startups
Djallel Bouneffouf
This project is part of the development of mobile CRM. It aims to develop a management application client named NOMALYS. This application allows the commercial and business leaders…
A Survey on Compositional Generalization in Applications
Baihan Lin, Djallel Bouneffouf, Irina Rish
The field of compositional generalization is currently experiencing a renaissance in AI, as novel problem settings and algorithms motivated by various practical applications are be…
Hybrid Q-Learning Applied to Ubiquitous recommender system
Djallel Bouneffouf
Ubiquitous information access becomes more and more important nowadays and research is aimed at making it adapted to users. Our work consists in applying machine learning technique…
Assessing AI Utility: The Random Guesser Test for Sequential Decision-Making Systems
Shun Ide, Allison Blunt, Djallel Bouneffouf
We propose a general approach to quantitatively assessing the risk and vulnerability of artificial intelligence (AI) systems to biased decisions. The guiding principle of the propo…
Optimal Epidemic Control as a Contextual Combinatorial Bandit with Budget
Baihan Lin, Djallel Bouneffouf
In light of the COVID-19 pandemic, it is an open challenge and critical practical problem to find a optimal way to dynamically prescribe the best policies that balance both the gov…
Computing the Dirichlet-Multinomial Log-Likelihood Function
Djallel Bouneffouf
Dirichlet-multinomial (DMN) distribution is commonly used to model over-dispersion in count data. Precise and fast numerical computation of the DMN log-likelihood function is impor…
Scopes of Alignment
Kush R. Varshney, Zahra Ashktorab, Djallel Bouneffouf +2
Much of the research focus on AI alignment seeks to align large language models and other foundation models to the context-less and generic values of helpfulness, harmlessness, and…
A Neural Networks Committee for the Contextual Bandit Problem
Robin Allesiardo, Raphael Feraud, Djallel Bouneffouf
This paper presents a new contextual bandit algorithm, NeuralBandit, which does not need hypothesis on stationarity of contexts and rewards. Several neural networks are trained to…
Targeted Advertising on Social Networks Using Online Variational Tensor Regression
Tsuyoshi Idé, Keerthiram Murugesan, Djallel Bouneffouf +1
This paper is concerned with online targeted advertising on social networks. The main technical task we address is to estimate the activation probability for user pairs, which quan…
Psychotherapy AI Companion with Reinforcement Learning Recommendations and Interpretable Policy Dynamics
Baihan Lin, Guillermo Cecchi, Djallel Bouneffouf
We introduce a Reinforcement Learning Psychotherapy AI Companion that generates topic recommendations for therapists based on patient responses. The system uses Deep Reinforcement…
Scalable Recollections for Continual Lifelong Learning
Matthew Riemer, Tim Klinger, Djallel Bouneffouf +1
Given the recent success of Deep Learning applied to a variety of single tasks, it is natural to consider more human-realistic settings. Perhaps the most difficult of these setting…
Online Learning in Iterated Prisoner's Dilemma to Mimic Human Behavior
Baihan Lin, Djallel Bouneffouf, Guillermo Cecchi
As an important psychological and social experiment, the Iterated Prisoner's Dilemma (IPD) treats the choice to cooperate or defect as an atomic action. We propose to study the beh…
Enhancing Value Alignment of LLMs with Multi-agent system and Combinatorial Fusion
Yuanhong Wu, Djallel Bouneffouf, D. Frank Hsu
Aligning large language models (LLMs) with human values is a central challenge for ensuring trustworthy and safe deployment. While existing methods such as Reinforcement Learning f…
Incorporating Behavioral Constraints in Online AI Systems
Avinash Balakrishnan, Djallel Bouneffouf, Nicholas Mattei +1
AI systems that learn through reward feedback about the actions they take are increasingly deployed in domains that have significant impact on our daily life. However, in many case…
Working Alliance Transformer for Psychotherapy Dialogue Classification
Baihan Lin, Guillermo Cecchi, Djallel Bouneffouf
As a predictive measure of the treatment outcome in psychotherapy, the working alliance measures the agreement of the patient and the therapist in terms of their bond, task and goa…
Spectral Clustering using Eigenspectrum Shape Based Nystrom Sampling
Djallel Bouneffouf
Spectral clustering has shown a superior performance in analyzing the cluster structure. However, its computational complexity limits its application in analyzing large-scale data.…
Solving Constrained CASH Problems with ADMM
Parikshit Ram, Sijia Liu, Deepak Vijaykeerthi +5
The CASH problem has been widely studied in the context of automated configurations of machine learning (ML) pipelines and various solvers and toolkits are available. However, CASH…
Contextual Moral Value Alignment Through Context-Based Aggregation
Pierre Dognin, Jesus Rios, Ronny Luss +7
Developing value-aligned AI agents is a complex undertaking and an ongoing challenge in the field of AI. Specifically within the domain of Large Language Models (LLMs), the capabil…
Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits and RL
Baihan Lin, Guillermo Cecchi, Djallel Bouneffouf +2
Artificial behavioral agents are often evaluated based on their consistent behaviors and performance to take sequential actions in an environment to maximize some notion of cumulat…
Online learning with Corrupted context: Corrupted Contextual Bandits
Djallel Bouneffouf
We consider a novel variant of the contextual bandit problem (i.e., the multi-armed bandit with side-information, or context, available to a decision-maker) where the context used…
Agentic AI Needs a Systems Theory
Erik Miehling, Karthikeyan Natesan Ramamurthy, Kush R. Varshney +11
The endowment of AI with reasoning capabilities and some degree of agency is widely viewed as a path toward more capable and generalizable systems. Our position is that the current…
Reinforcement Learning with Algorithms from Probabilistic Structure Estimation
Jonathan P. Epperlein, Roman Overko, Sergiy Zhuk +4
Reinforcement learning (RL) algorithms aim to learn optimal decisions in unknown environments through experience of taking actions and observing the rewards gained. In some cases,…
Evolution of the user's content: An Overview of the state of the art
Djallel Bouneffouf
The evolution of the user's content still remains a problem for an accurate recommendation.This is why the current research aims to design Recommender Systems (RS) able to continua…
Alignment Studio: Aligning Large Language Models to Particular Contextual Regulations
Swapnaja Achintalwar, Ioana Baldini, Djallel Bouneffouf +16
The alignment of large language models is usually done by model providers to add or control behaviors that are common or universally understood across use cases and contexts. In co…
Survey on Applications of Neurosymbolic Artificial Intelligence
Djallel Bouneffouf, Charu C. Aggarwal
In recent years, the Neurosymbolic framework has attracted a lot of attention in various applications, from recommender systems and information retrieval to healthcare and finance.…
Recommandation mobile, sensible au contexte de contenus évolutifs: Contextuel-E-Greedy
Djallel Bouneffouf
We introduce in this paper an algorithm named Contextuel-E-Greedy that tackles the dynamicity of the user's content. It is based on dynamic exploration/exploitation tradeoff and ca…
A Survey on Practical Applications of Multi-Armed and Contextual Bandits
Djallel Bouneffouf, Irina Rish
In recent years, multi-armed bandit (MAB) framework has attracted a lot of attention in various applications, from recommender systems and information retrieval to healthcare and f…
Applying machine learning techniques to improve user acceptance on ubiquitous environement
Djallel Bouneffouf
Ubiquitous information access becomes more and more important nowadays and research is aimed at making it adapted to users. Our work consists in applying machine learning technique…
Proceedings of 1st Workshop on Advancing Artificial Intelligence through Theory of Mind
Mouad Abrini, Omri Abend, Dina Acklin +105
This volume includes a selection of papers presented at the Workshop on Advancing Artificial Intelligence through Theory of Mind held at AAAI 2025 in Philadelphia US on 3rd March 2…
Mitigating Misalignment Contagion by Steering with Implicit Traits
Maria Chang, Ronny Luss, Miao Liu +3
Language models (LMs) are increasingly used in high-stakes, multi-agent settings, where following instructions and maintaining value alignment are critical. Most alignment research…
A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry
Baihan Lin, Guillermo Cecchi, Djallel Bouneffouf +2
Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends st…
Mobile Recommender Systems Methods: An Overview
Djallel Bouneffouf
The information that mobiles can access becomes very wide nowadays, and the user is faced with a dilemma: there is an unlimited pool of information available to him but he is unabl…
Context-Based Information Retrieval in Risky Environment
Djallel Bouneffouf
Context-Based Information Retrieval is recently modelled as an exploration/ exploitation trade-off (exr/exp) problem, where the system has to choose between maximizing its expected…
Position: Theory of Mind Benchmarks are Broken for Large Language Models
Matthew Riemer, Zahra Ashktorab, Djallel Bouneffouf +4
Our paper argues that the majority of theory of mind benchmarks are broken because of their inability to directly test how large language models (LLMs) adapt to new partners. This…
Context Attentive Bandits: Contextual Bandit with Restricted Context
Djallel Bouneffouf, Irina Rish, Guillermo A. Cecchi +1
We consider a novel formulation of the multi-armed bandit model, which we call the contextual bandit with restricted context, where only a limited number of features can be accesse…
Hyper-parameter Tuning for the Contextual Bandit
Djallel Bouneffouf, Emmanuelle Claeys
We study here the problem of learning the exploration exploitation trade-off in the contextual bandit problem with linear reward function setting. In the traditional algorithms tha…
Bandit Models of Human Behavior: Reward Processing in Mental Disorders
Djallel Bouneffouf, Irina Rish, Guillermo A. Cecchi
Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for multi-armed bandit problem, which extends the standard T…
Ãtude des dimensions spécifiques du contexte dans un système de filtrage d'informations
Djallel Bouneffouf
In the context of business information systems, e-commerce and access to knowledge, the relevance of the information provided to use is a key fact to the success of information sys…
Contextual Value Alignment via Multilayer Combinatorial Fusion
Yuanhong Wu, Djallel Bouneffouf, D. Frank Hsu
Aligning large language models (LLMs) with human values remains a major challenge, especially for trustworthy AI. While existing approaches such as RLHF, CAI, and their variants ha…
Exponentiated Gradient LINUCB for Contextual Multi-Armed Bandits
Djallel Bouneffouf
We present Exponentiated Gradient LINUCB, an algorithm for con-textual multi-armed bandits. This algorithm uses Exponentiated Gradient to find the optimal exploration of the LINUCB…
Etat de l'art sur l'application des bandits multi-bras
Djallel Bouneffouf
The Multi-armed bandit offer the advantage to learn and exploit the already learnt knowledge at the same time. This capability allows this approach to be applied in different domai…
An ADMM Based Framework for AutoML Pipeline Configuration
Sijia Liu, Parikshit Ram, Deepak Vijaykeerthy +6
We study the AutoML problem of automatically configuring machine learning pipelines by jointly selecting algorithms and their appropriate hyper-parameters for all steps in supervis…
The Ultimate Test of Superintelligent AI Agents: Can an AI Balance Care and Control in Asymmetric Relationships?
Djallel Bouneffouf, Matthew Riemer, Kush Varshney
This paper introduces the Shepherd Test, a new conceptual test for assessing the moral and relational dimensions of superintelligent artificial agents. The test is inspired by huma…
SupervisorBot: NLP-Annotated Real-Time Recommendations of Psychotherapy Treatment Strategies with Deep Reinforcement Learning
Baihan Lin, Guillermo Cecchi, Djallel Bouneffouf
We propose a recommendation system that suggests treatment strategies to a therapist during the psychotherapy session in real-time. Our system uses a turn-level rating mechanism th…
Interpretable Multi-Objective Reinforcement Learning through Policy Orchestration
Ritesh Noothigattu, Djallel Bouneffouf, Nicholas Mattei +6
Autonomous cyber-physical agents and systems play an increasingly large role in our lives. To ensure that agents behave in ways aligned with the values of the societies in which th…