32 citations · 39 across the 23 of their papers we have counts for
23 papers
Efficient Agent Evaluation via Diversity-Guided User Simulation
Itay Nakash, George Kour, Ateret Anaby-Tavor
Large language models (LLMs) are increasingly deployed as customer-facing agents, yet evaluating their reliability remains challenging due to stochastic, multi-turn interactions. C…
Near-Miss: Latent Policy Failure Detection in Agentic Workflows
Ella Rabinovich, David Boaz, Naama Zwerdling +1
Agentic systems for business process automation often require compliance with policies governing conditional updates to the system state. Evaluation of policy adherence in LLM-base…
CRISP: Complex Reasoning with Interpretable Step-based Plans
Matan Vetzler, Koren Lazar, Guy Uziel +3
Recent advancements in large language models (LLMs) underscore the need for stronger reasoning capabilities to solve complex problems effectively. While Chain-of-Thought (CoT) reas…
Towards Enforcing Company Policy Adherence in Agentic Workflows
Naama Zwerdling, David Boaz, Ella Rabinovich +3
Large Language Model (LLM) agents hold promise for a flexible and scalable alternative to traditional business process automation, but struggle to reliably follow complex company p…
Effective Red-Teaming of Policy-Adherent Agents
Itay Nakash, George Kour, Koren Lazar +3
Task-oriented LLM-based agents are increasingly used in domains with strict policies, such as refund eligibility or cancellation rules. The challenge lies in ensuring that the agen…
Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models
George Kour, Itay Nakash, Ateret Anaby-Tavor +1
As Large Language Models (LLMs) become deeply integrated into human life and increasingly influence decision-making, it's crucial to evaluate whether and to what extent they exhibi…
On the Robustness of Agentic Function Calling
Ella Rabinovich, Ateret Anaby-Tavor
Large Language Models (LLMs) are increasingly acting as autonomous agents, with function calling (FC) capabilities enabling them to invoke specific tools for tasks. While prior res…
Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In
Itay Nakash, George Kour, Guy Uziel +1
Following the advancement of large language models (LLMs), the development of LLM-based autonomous agents has become increasingly prevalent. As a result, the need to understand the…
Exploring Straightforward Conversational Red-Teaming
George Kour, Naama Zwerdling, Marcel Zalmanovici +3
Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks. Multi-turn conversations, where context influences the mod…
A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios
Samuel Ackerman, Ella Rabinovich, Eitan Farchi +1
We evaluate the robustness of several large language models on multiple datasets. Robustness here refers to the relative insensitivity of the model's answers to meaning-preserving…
From Zero to Hero: Cold-Start Anomaly Detection
Tal Reiss, George Kour, Naama Zwerdling +2
When first deploying an anomaly detection system, e.g., to detect out-of-scope queries in chatbots, there are no observed data, making data-driven approaches ineffective. Zero-shot…
Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations
Swapnaja Achintalwar, Adriana Alvarado Garcia, Ateret Anaby-Tavor +35
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations. Due to several limiting factors surrounding LLMs (trai…
SpeCrawler: Generating OpenAPI Specifications from API Documentation Using Large Language Models
Koren Lazar, Matan Vetzler, Guy Uziel +4
In the digital era, the widespread use of APIs is evident. However, scalable utilization of APIs poses a challenge due to structure divergence observed in online API documentation.…
What's the Plan? Evaluating and Developing Planning-Aware Techniques for Language Models
Eran Hirsch, Guy Uziel, Ateret Anaby-Tavor
Planning is a fundamental task in artificial intelligence that involves finding a sequence of actions that achieve a specified goal in a given environment. Large language models (L…
Unveiling Safety Vulnerabilities of Large Language Models
George Kour, Marcel Zalmanovici, Naama Zwerdling +5
As large language models become more prevalent, their possible harmful or inappropriate responses are a cause for concern. This paper introduces a unique dataset containing adversa…
Measuring the Measuring Tools: An Automatic Evaluation of Semantic Metrics for Text Corpora
George Kour, Samuel Ackerman, Orna Raz +3
The ability to compare the semantic similarity between text corpora is important in a variety of natural language processing applications. However, standard methods for evaluating…
Understanding the Properties of Generated Corpora
Naama Zwerdling, Segev Shlomov, Esther Goldbraich +6
Models for text generation have become focal for many research tasks and especially for the generation of sentence corpora. However, understanding the properties of an automaticall…
High-quality Conversational Systems
Samuel Ackerman, Ateret Anaby-Tavor, Eitan Farchi +7
Conversational systems or chatbots are an example of AI-Infused Applications (AIIA). Chatbots are especially important as they are often the first interaction of clients with a bus…
Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems
Ella Rabinovich, Matan Vetzler, David Boaz +3
The rapidly growing market demand for automatic dialogue agents capable of goal-oriented behavior has caused many tech-industry leaders to invest considerable efforts into task-ori…
Classifier Data Quality: A Geometric Complexity Based Method for Automated Baseline And Insights Generation
George Kour, Marcel Zalmanovici, Orna Raz +2
Testing Machine Learning (ML) models and AI-Infused Applications (AIIAs), or systems that contain ML models, is highly challenging. In addition to the challenges of testing classic…
Improved Goal Oriented Dialogue via Utterance Generation and Look Ahead
Eyal Ben-David, Boaz Carmeli, Ateret Anaby-Tavor
Goal oriented dialogue systems have become a prominent customer-care interaction channel for most businesses. However, not all interactions are smooth, and customer intent misunder…
We've had this conversation before: A Novel Approach to Measuring Dialog Similarity
Ofer Lavi, Ella Rabinovich, Segev Shlomov +3
Dialog is a core building block of human natural language interactions. It contains multi-party utterances used to convey information from one party to another in a dynamic and evo…
Not Enough Data? Deep Learning to the Rescue!
Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich +5
Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a…