8 citations · 32 across the 31 of their papers we have counts for
10 papers · 1 filter
Automatic Generation of Benchmarks and Reliable LLM Judgment for Code Tasks
Eitan Farchi, Shmulik Froimovich, Rami Katan +1
LLMs can be used in a variety of code related tasks such as translating from one programming language to another, implementing natural language requirements and code summarization.…
A Practical Approach to Combinatorial Test Design
Eitan Farchi, Debbie Furman
Typical software has a huge input space. The number of inputs may be astronomical or even infinite. Thus, the task of validating that the software is correct seems hopeless. To dea…
Exploring Straightforward Conversational Red-Teaming
George Kour, Naama Zwerdling, Marcel Zalmanovici +3
Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks. Multi-turn conversations, where context influences the mod…
How Safe is Your Safety Metric? Automatic Concatenation Tests for Metric Reliability
Ora Nova Fandina, Leshem Choshen, Eitan Farchi +3
Consider a scenario where a harmfulness evaluation metric intended to filter unsafe responses from a Large Language Model. When applied to individual harmful prompt-response pairs,…
A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios
Samuel Ackerman, Ella Rabinovich, Eitan Farchi +1
We evaluate the robustness of several large language models on multiple datasets. Robustness here refers to the relative insensitivity of the model's answers to meaning-preserving…
Generating Unseen Code Tests In Infinitum
Marcel Zalmanovici, Orna Raz, Eitan Farchi +1
Large Language Models (LLMs) are used for many tasks, including those related to coding. An important aspect of being able to utilize LLMs is the ability to assess their fitness fo…