Publications (33)
Differentially Private Training of Mixture of Experts Models
Pierre Tholoniat, Huseyin A. Inan, Janardhan Kulkarni +1
This position paper investigates the integration of Differential Privacy (DP) in the training of Mixture of Experts (MoE) models within the field of natural language processing. As…
Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents
Yuxin Wang, Paul Thomas, Zhiwei Yu +5
Prior research on memory mechanism in RAG-based conversational system has emphasized how memory is stored and retrieved. However, far less is known about how memories with differen…
Computer-Using World Model
Yiming Guan, Rui Yu, John Zhang +15
Agents operating in complex software environments benefit from reasoning about the consequences of their actions, as even a single incorrect user interface (UI) operation can derai…
Project Florida: Federated Learning Made Easy
Daniel Madrigal Diaz, Andre Manoel, Jialei Chen +2
We present Project Florida, a system architecture and software development kit (SDK) enabling deployment of large-scale Federated Learning (FL) solutions across a heterogeneous dev…
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
Fatemehsadat Mireshghallah, Huseyin A. Inan, Marcello Hasegawa +3
Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy implications when training models on user content such…
Learning GUI Grounding with Spatial Reasoning from Visual Feedback
Yu Zhao, Wei-Ning Chen, Huseyin Atahan Inan +8
Graphical User Interface (GUI) grounding is commonly framed as a coordinate prediction task -- given a natural language instruction, generate on-screen coordinates for actions such…
Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity
Menglin Xia, Xuchao Zhang, Shantanu Dixit +6
Agent memory systems must accommodate continuously growing information while supporting efficient, context-aware retrieval for downstream tasks. Abstraction is essential for scalin…
Privacy Leakage in Text Classification: A Data Extraction Approach
Adel Elmahdy, Huseyin A. Inan, Robert Sim
Recent work has demonstrated the successful extraction of training data from generative language models. However, it is not evident whether such extraction is feasible in text clas…
On Privacy and Confidentiality of Communications in Organizational Graphs
Masoumeh Shafieinejad, Huseyin Inan, Marcello Hasegawa +1
Machine learned models trained on organizational communication data, such as emails in an enterprise, carry unique risks of breaching confidentiality, even if the model is intended…
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
Xiang Yue, Huseyin A. Inan, Xuechen Li +6
Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data. Generating syntheti…
Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning
Yae Jee Cho, Andre Manoel, Gauri Joshi +2
Federated learning (FL) enables edge-devices to collaboratively learn a model without disclosing their private data to a central aggregating server. Most existing FL algorithms req…
TinyTroupe: An LLM-powered Multiagent Persona Simulation Toolkit
Paulo Salem, Robert Sim, Christopher Olsen +3
Recent advances in Large Language Models (LLM) have led to a new class of autonomous agents, renewing and expanding interest in the area. LLM-powered Multiagent Systems (MAS) have…
Conversations with Documents. An Exploration of Document-Centered Assistance
Maartje ter Hoeve, Robert Sim, Elnaz Nouri +3
The role of conversational assistants has become more prevalent in helping people increase their productivity. Document-centered assistance, for example to help an individual quick…
Exploring How LLMs Capture and Represent Domain-Specific Knowledge
Mirian Hipolito Garcia, Camille Couturier, Daniel Madrigal Diaz +5
We study whether Large Language Models (LLMs) inherently capture domain-specific nuances in natural language. Our experiments probe the domain sensitivity of LLMs by examining thei…
Training Data Leakage Analysis in Language Models
Huseyin A. Inan, Osman Ramadan, Lukas Wutschitz +4
Recent advances in neural network based language models lead to successful deployments of such models, improving user experience in various applications. It has been demonstrated t…
Federated Multilingual Models for Medical Transcript Analysis
Andre Manoel, Mirian Hipolito Garcia, Tal Baumel +6
Federated Learning (FL) is a novel machine learning approach that allows the model trainer to access more data samples, by training the model across multiple decentralized data sou…
Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
Dujian Ding, Ankur Mallick, Chi Wang +5
Large language models (LLMs) excel in most NLP tasks but also require expensive cloud servers for deployment due to their size, while smaller models that can be deployed on lower c…
FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations
Mirian Hipolito Garcia, Andre Manoel, Daniel Madrigal Diaz +3
In this paper we introduce "Federated Learning Utilities and Tools for Experimentation" (FLUTE), a high-performance open-source platform for federated learning research and offline…
Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation
Xinyu Tang, Richard Shin, Huseyin A. Inan +6
We study the problem of in-context learning (ICL) with large language models (LLMs) on private datasets. This scenario poses privacy risks, as LLMs may leak or regurgitate the priv…
Analyzing Leakage of Personally Identifiable Information in Language Models
Nils Lukas, Ahmed Salem, Robert Sim +3
Language Models (LMs) have been shown to leak information about training data through sentence-level membership inference and reconstruction attacks. Understanding the risk of LMs…
Sandi: A System for Accountability and Applications in Direct Communication (Extended Abstract)
F. Betül Durak, Kim Laine, Simon Langowski +3
Reputation systems guide our decision making both in life and work: which restaurant to eat at, which vendor to buy from, which software dependencies to use, and who or what to tru…
Simulating Environments with Reasoning Models for Agent Training
Yuetai Li, Huseyin A Inan, Xiang Yue +6
LLM agents excel in compact environments requiring deep reasoning but remain brittle when operating in broader, more complex contexts that demand robustness across diverse tools an…
UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis
Fatemehsadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi +3
Global models are trained to be as generalizable as possible, with user invariance considered desirable since the models are shared across multitudes of users. As such, these model…
CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents
Wenjie Fu, Xiaoting Qin, Jue Zhang +5
Enterprise LLM agents can dramatically improve workplace productivity, but their core capability, retrieving and using internal context to act on a user's behalf, also creates new…
Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation
Chen Dun, Mirian Hipolito Garcia, Guoqing Zheng +3
Large Language Models (LLMs) have the ability to solve a variety of tasks, such as text summarization and mathematical questions, just out of the box, but they are often trained wi…
Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
Yehya Farhat, Hamza ElMokhtar Shili, Fangshuo Liao +7
Mixture-of-Experts (MoEs) achieve scalability by dynamically activating subsets of their components. Yet, understanding how expertise emerges through joint training of gating mecha…
Privately Aligning Language Models with Reinforcement Learning
Fan Wu, Huseyin A. Inan, Arturs Backurs +3
Positioned between pre-training and user deployment, aligning large language models (LLMs) through reinforcement learning (RL) has emerged as a prevailing strategy for training ins…
ACON: Optimizing Context Compression for Long-horizon LLM Agents
Minki Kang, Wei-Ning Chen, Dongge Han +5
Large language models (LLMs) are increasingly deployed as agents in dynamic real-world environments, where success depends on maintaining precise records of actions and observation…
VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild
Junlin Fang, Wenyu Chen, Reshmi Ghosh +7
Vision-language Models (VLMs) are essential for contextual understanding of both visual and textual information. However, their vulnerability to adversarially manipulated inputs pr…
Understanding Questions that Arise When Working with Business Documents
Farnaz Jahanbakhsh, Elnaz Nouri, Robert Sim +2
While digital assistants are increasingly used to help with various productivity tasks, less attention has been paid to employing them in the domain of business documents. To build…
Controllable Synthetic Clinical Note Generation with Privacy Guarantees
Tal Baumel, Andre Manoel, Daniel Jones +5
In the field of machine learning, domain-specific annotated data is an invaluable resource for training effective models. However, in the medical domain, this data often includes P…
Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
Guangchen Lan, Huseyin A. Inan, Sahar Abdelnabi +5
As the era of autonomous agents making decisions on behalf of users unfolds, ensuring contextual integrity (CI) -- what is the appropriate information to share while carrying out a…
TrojanPuzzle: Covertly Poisoning Code-Suggestion Models
Hojjat Aghakhani, Wei Dai, Andre Manoel +7
With tools like GitHub Copilot, automatic code suggestion is no longer a dream in software engineering. These tools, based on large language models, are typically trained on massiv…