289 citations · 554 across the 28 of their papers we have counts for
30 papers
Comparison Visual Instruction Tuning
Wei Lin, Muhammad Jehanzeb Mirza, Sivan Doveh +4
Comparing two images in terms of Commonalities and Differences (CaD) is a fundamental human capability that forms the basis of advanced visual reasoning and interpretation. It is e…
NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning
Eli Schwartz, Leshem Choshen, Joseph Shtok +3
Language models struggle with handling numerical data and performing arithmetic operations. We hypothesize that this limitation can be partially attributed to non-intuitive textual…
Towards Multimodal In-Context Learning for Vision & Language Models
Sivan Doveh, Shaked Perek, M. Jehanzeb Mirza +5
State-of-the-art Vision-Language Models (VLMs) ground the vision and the language modality primarily via projecting the vision tokens from the encoder to language-like tokens, whic…
Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs
M. Jehanzeb Mirza, Leonid Karlinsky, Wei Lin +5
Prompt ensembling of Large Language Model (LLM) generated category-specific prompts has emerged as an effective method to enhance zero-shot recognition ability of Vision-Language M…
TAP: Targeted Prompting for Task Adaptive Generation of Textual Training Instances for Visual Classification
M. Jehanzeb Mirza, Leonid Karlinsky, Wei Lin +3
Vision and Language Models (VLMs), such as CLIP, have enabled visual recognition of a potentially unlimited set of categories described by text prompts. However, for the best visua…
LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collections
M. Jehanzeb Mirza, Leonid Karlinsky, Wei Lin +4
Recently, large-scale pre-trained Vision and Language (VL) models have set a new state-of-the-art (SOTA) in zero-shot visual classification enabling open-vocabulary recognition of…
On the Transferability of Visual Features in Generalized Zero-Shot Learning
Paola Cascante-Bonilla, Leonid Karlinsky, James Seale Smith +2
Generalized Zero-Shot Learning (GZSL) aims to train a classifier that can generalize to unseen classes, using a set of attributes as auxiliary information, and the visual features…
MAEDAY: MAE for few and zero shot AnomalY-Detection
Eli Schwartz, Assaf Arbelle, Leonid Karlinsky +4
We propose using Masked Auto-Encoder (MAE), a transformer model self-supervisedly trained on image inpainting, for anomaly detection (AD). Assuming anomalous regions are harder to…
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning
James Seale Smith, Paola Cascante-Bonilla, Assaf Arbelle +7
Recently, large-scale pre-trained Vision-and-Language (VL) foundation models have demonstrated remarkable capabilities in many zero-shot downstream tasks, achieving competitive res…
C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval
Andrew Rouditchenko, Yung-Sung Chuang, Nina Shvetsova +7
Multilingual text-video retrieval methods have improved significantly in recent years, but the performance for other languages lags behind English. We propose a Cross-Lingual Cross…
VL-Taboo: An Analysis of Attribute-based Zero-shot Capabilities of Vision-Language Models
Felix Vogel, Nina Shvetsova, Leonid Karlinsky +1
Vision-language models trained on large, randomly collected data had significant impact in many areas since they appeared. But as they show great performance in various fields, suc…
Unsupervised Domain Generalization by Learning a Bridge Across Domains
Sivan Harary, Eli Schwartz, Assaf Arbelle +11
The ability to generalize learned representations across significantly different visual domains, such as between real photos, clipart, paintings, and sketches, is a fundamental cap…
CHARTER: heatmap-based multi-type chart data extraction
Joseph Shtok, Sivan Harary, Ophir Azulai +3
The digital conversion of information stored in documents is a great source of knowledge. In contrast to the documents text, the conversion of the embedded documents graphics, such…
Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data
Ashraful Islam, Chun-Fu Chen, Rameswar Panda +3
Most existing works in few-shot learning rely on meta-learning the network on a large base dataset which is typically from the same domain as the target dataset. We tackle the prob…
Detector-Free Weakly Supervised Grounding by Separation
Assaf Arbelle, Sivan Doveh, Amit Alfassy +14
Nowadays, there is an abundance of data involving images and surrounding free-form text weakly corresponding to those images. Weakly Supervised phrase-Grounding (WSG) deals with th…
A Broad Study on the Transferability of Visual Representations with Contrastive Learning
Ashraful Islam, Chun-Fu Chen, Rameswar Panda +3
Tremendous progress has been made in visual representation learning, notably with the recent success of self-supervised contrastive learning methods. Supervised contrastive learnin…
AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
Yue Meng, Rameswar Panda, Chung-Ching Lin +5
Temporal modelling is the key for efficient video action recognition. While understanding temporal information can improve recognition accuracy for dynamic actions, removing tempor…
A Maximal Correlation Approach to Imposing Fairness in Machine Learning
Joshua Lee, Yuheng Bu, Prasanna Sattigeri +4
As machine learning algorithms grow in popularity and diversify to many industries, ethical and legal concerns regarding their fairness have become increasingly relevant. We explor…
Fine-grained Angular Contrastive Learning with Coarse Labels
Guy Bukchin, Eli Schwartz, Kate Saenko +4
Few-shot learning methods offer pre-training techniques optimized for easier later adaptation of the model to new classes (unseen during training) using one or a few examples. This…
OnlineAugment: Online Data Augmentation with Less Domain Knowledge
Zhiqiang Tang, Yunhe Gao, Leonid Karlinsky +3
Data augmentation is one of the most important tools in training modern deep neural networks. Recently, great advances have been made in searching for optimal augmentation policies…
AR-Net: Adaptive Frame Resolution for Efficient Action Recognition
Yue Meng, Chung-Ching Lin, Rameswar Panda +5
Action recognition is an open and challenging problem in computer vision. While current state-of-the-art models offer excellent recognition results, their computational expense lim…
TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classification
Moshe Lichtenstein, Prasanna Sattigeri, Rogerio Feris +2
The field of Few-Shot Learning (FSL), or learning from very few (typically or ) examples per novel class (unseen during training), has received a lot of attention and signif…
StarNet: towards Weakly Supervised Few-Shot Object Detection
Leonid Karlinsky, Joseph Shtok, Amit Alfassy +8
Few-shot detection and classification have advanced significantly in recent years. Yet, detection approaches require strong annotation (bounding boxes) both for pre-training and fo…
A Broader Study of Cross-Domain Few-Shot Learning
Yunhui Guo, Noel C. Codella, Leonid Karlinsky +5
Recent progress on few-shot learning largely relies on annotated data for meta-learning: base classes sampled from the same domain as the novel classes. However, in many applicatio…