Rethinking Cooking State Recognition with Vision Transformers

Abstract

To ensure proper knowledge representation of the kitchen environment, it is vital for kitchen robots to recognize the states of the food items that are being cooked. Although the domain of object detection and recognition has been extensively studied, the task of object state classification has remained relatively unexplored. The high intra-class similarity of ingredients during different states of cooking makes the task even more challenging. Researchers have proposed adopting Deep Learning based strategies in recent times, however, they are yet to achieve high performance. In this study, we utilized the self-attention mechanism of the Vision Transformer (ViT) architecture for the Cooking State Recognition task. The proposed approach encapsulates the globally salient features from images, while also exploiting the weights learned from a larger dataset. This global attention allows the model to withstand the similarities between samples of different cooking objects, while the employment of transfer learning helps to overcome the lack of inductive bias by utilizing pretrained weights. To improve recognition accuracy, several augmentation techniques have been employed as well. Evaluation of our proposed framework on the `Cooking State Recognition Challenge Dataset' has achieved an accuracy of 94.3%, which significantly outperforms the state-of-the-art.

arXiv commentAccepted in 25th ICCIT (6 pages, 5 Figures, 5 Tables)

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Transformers in Vision: A Survey
• Albumentations: fast and flexible image augmentations
• Less is More: Lighter and Faster Deep Neural Architecture for Tomato Leaf Disease Classification
• Identifying Object States in Cooking-Related Images
• Two Decades of Bengali Handwritten Digit Recognition: A Survey
• Cooking State Recognition from Images Using Inception Architecture

Cited by in corpus (2)

• Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL
• AQUA20: A Benchmark Dataset for Underwater Species Classification under Challenging Conditions