A Central Asian Food Dataset for Personalized Dietary Interventions, Extended Abstract

This manuscript is an extended abstract of our previously published article, A Central Asian Food Dataset for Personalized Dietary Interventions, which appeared in the Nutrients, MDPI in 2023 [1].

Food computation from visual data has gained prominence due to computer vision advancements and increased smartphone, and social media usage [2]. These platforms provide access to food-related information, which can be utilized for various tasks, including medical, gastronomic, and agronomic research. Deep learning-based food image recognition systems have been developed for applications in dietary assessment, smart restaurants, food safety inspection, and agriculture. Automatic food image recognition can improve the accuracy of nutritional records and assist visually impaired individuals [2].

Existing food classification datasets mostly include Western, European, Chinese, and other Asian cuisines [10, 6]. Examples of such datasets are presented in Table I. However, FoodAI is not open source, and Food2K is not publicly available. Food1K, a food recognition challenge dataset, has been released with 400,000 images and 1,000 food classes.

Most food datasets predominantly contain Western and Asian dishes, lacking specific national dishes like those in Central Asia. To address this, we aim to develop a unique food recognition system for our region, considering local preferences, specialties, and cuisines. In this paper, we describe the development of our dataset, food recognition models, and their performance and conclude with a summary of our findings.

This paper introduces the Central Asian Food Dataset (CAFD), consisting of 16,499 images across 42 classes representing popular Central Asian cuisine. We ensured the dataset’s high quality through extensive data cleaning, iterative annotation, and multiple inspections. The CAFD can also be a food image representation learning benchmark.

We followed a five-step process to create a diverse and high-quality dataset. First, we listed popular Central Asian food items. Second, we scraped images from search engines and social media using a Python script with Selenium. We extracted images from recipe videos using Roboflow [11] to increase underrepresented classes. Third, we removed duplicates using the HashImage Python library. Fourth, two annotators created bounding boxes for each food item using Roboflow software. Fifth, we cropped the food items based on bounding box coordinates and stored them in separate directories by class. The final dataset has an imbalanced number of images per class, ranging from 99 to 922.

Image classification is a computer vision task that extracts a single descriptor from an image. State-of-the-art models are based on CNNs and have improved due to large datasets. Transfer learning is often used when sufficient training data is unavailable, as it leverages knowledge from pre-trained models to solve similar problems in different domains [12]. In this work, we applied transfer learning to food classification using model weights pre-trained on ImageNet, a large dataset with over 14 million images [13].

We selected 10 models of different architectures and complexity to evaluate their performance on the CAFD. These models include VGG-16, Squeezenet1, and five models with ResNet architecture [14, 15, 16, 17, 18]. DenseNet-121 and EfficientNet-b4 have similar architectures to ResNets but introduce different scaling methods [19, 20].

Then we trained the models on the Food1K dataset and tested the combination of CAFD and Food1K. We carefully split the datasets into training, validation, and test sets to avoid bias and data leakage. Table II shows the number of images in each set for three different datasets.

Also, we performed transfer learning on Pytorch using pre-trained models on ImageNet. Models were trained for 40 epochs with a learning rate of 0.001, batch size of 64, and a categorical cross-entropy loss. The input size of images varied depending on the model. We used Top-5 accuracy and Top-1 accuracy as evaluation metrics and precision, recall, and $F_{1}$-score metrics to identify and analyze the best and worst-classified food classes.

Table III summarizes the classification models’ results. All models performed better on the CAFD than on Food1K and CAFD+Food1K, indicating the accuracy and cleanness of the CAFD. VGG-16 achieved 86.03% Top-1 and 98.33% Top-5 accuracies on the CAFD, while Squeezenet1 had a lower performance. ResNet architectures achieved around 88% Top-1 and 98% Top-5 accuracy on the CAFD, with accuracy increasing as network depth increases. Wide ResNet-50 improved accuracy compared to ResNet50, and EfficientNet-b4 achieved the best results on Food1K and CAFD+Food1K.

Tables IV and V list the 10 best and worst detected CAFD classes by ResNet152 and EfficientNet-b4. Large classes with distinct features performed best, while fine-grained or similar-looking classes, presented in Figure 2, caused confusion and deteriorated model performance.

The Central Asian Food Dataset (CAFD) offers a unique advantage in automating and improving dietary assessment accuracy. It has potential applications in creating or modifying recipes, helping restaurants and food service providers plan menus, optimizing food production, and combating fraudulent food practices. It can be used to improve food quality, develop new recipes and personalized dietary plans, optimize production processes, increase food safety, and integrate with other food recognition systems.

Comprising 16,499 images of 42 food classes, the CAFD demonstrates the effectiveness of computer vision models for food recognition. Our models achieved a Top-5 accuracy of 98.59% and 98.01% for the CAFD and CAFD+Food1K, respectively. The dataset, source code, and pre-trained models are available on GitHub ¹¹1https://github.com/IS2AI/Central-Asian-Food-Dataset repository.

Future work includes exploring different neural network architectures, data augmentation methods, and utilizing the CAFD for other dietary-related tasks. We also plan to develop food scene recognition datasets with multiple food items per image and extend the current food categories based on additional food classes.

MYC and HAV conceived and designed the study. AK, HAV, and MYC contributed to defining the research scope and objectives. AK and AB collected and prepared the dataset and trained the models. AB created a pipeline for processing images in Roboflow. HAV provided guidelines for the project experiments. AK performed the final check of the dataset and finalized the experimental results. PI of the project: MYC. AK, MYC,and AB wrote the article, and all the authors contributed to the manuscript revision and approved the submitted version.