26 citations · 46 across the 12 of their papers we have counts for
13 papers
PACIFIC: a framework for generating benchmarks to check Precise Automatically Checked Instruction Following In Code
Itay Dreyfuss, Antonio Abu Nassar, Samuel Ackerman +5
Large Language Model (LLM)-based code assistants have emerged as a powerful application of generative AI, demonstrating impressive capabilities in code generation and comprehension…
Evaluating perturbation robustness of generative systems that use COBOL code inputs
Samuel Ackerman, Wesam Ibraheem, Orna Raz +1
Systems incorporating large language models (LLMs) as a component are known to be sensitive (i.e., non-robust) to minor input variations that do not change the meaning of the input…
Exploring Straightforward Conversational Red-Teaming
George Kour, Naama Zwerdling, Marcel Zalmanovici +3
Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks. Multi-turn conversations, where context influences the mod…
Generating Unseen Code Tests In Infinitum
Marcel Zalmanovici, Orna Raz, Eitan Farchi +1
Large Language Models (LLMs) are used for many tasks, including those related to coding. An important aspect of being able to utilize LLMs is the ability to assess their fitness fo…
Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations
Swapnaja Achintalwar, Adriana Alvarado Garcia, Ateret Anaby-Tavor +35
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations. Due to several limiting factors surrounding LLMs (trai…
Unveiling Safety Vulnerabilities of Large Language Models
George Kour, Marcel Zalmanovici, Naama Zwerdling +5
As large language models become more prevalent, their possible harmful or inappropriate responses are a cause for concern. This paper introduces a unique dataset containing adversa…
High-quality Conversational Systems
Samuel Ackerman, Ateret Anaby-Tavor, Eitan Farchi +7
Conversational systems or chatbots are an example of AI-Infused Applications (AIIA). Chatbots are especially important as they are often the first interaction of clients with a bus…
Classifier Data Quality: A Geometric Complexity Based Method for Automated Baseline And Insights Generation
George Kour, Marcel Zalmanovici, Orna Raz +2
Testing Machine Learning (ML) models and AI-Infused Applications (AIIAs), or systems that contain ML models, is highly challenging. In addition to the challenges of testing classic…
Automatically detecting data drift in machine learning classifiers
Samuel Ackerman, Orna Raz, Marcel Zalmanovici +1
Classifiers and other statistics-based machine learning (ML) techniques generalize, or learn, based on various statistical properties of the training data. The assumption underlyin…
Density-based interpretable hypercube region partitioning for mixed numeric and categorical data
Samuel Ackerman, Eitan Farchi, Orna Raz +2
Consider a structured dataset of features, such as . A user may want to know where in the feature space obser…
FreaAI: Automated extraction of data slices to test machine learning models
Samuel Ackerman, Orna Raz, Marcel Zalmanovici
Machine learning (ML) solutions are prevalent. However, many challenges exist in making these solutions business-grade. One major challenge is to ensure that the ML solution provid…
Machine Learning Model Drift Detection Via Weak Data Slices
Samuel Ackerman, Parijat Dube, Eitan Farchi +2
Detecting drift in performance of Machine Learning (ML) models is an acknowledged challenge. For ML models to become an integral part of business applications it is essential to de…
Detection of data drift and outliers affecting machine learning model performance over time
Samuel Ackerman, Eitan Farchi, Orna Raz +2
A trained ML model is deployed on another `test' dataset where target feature values (labels) are unknown. Drift is distribution change between the training and deployment data, wh…