Explainable deep learning ensemble for food image analysis on edge devices

Food recognition systems recently garnered much research attention in the relevant field due to their ability to obtain objective measurements for dietary intake. This feature contributes to the management of various chronic conditions. Challenges such as inter and intraclass variations alongside th...

Full description

Saved in:
Bibliographic Details
Main Authors: Tahir, Ghalib Ahmed, Loo, Chu Kiong
Format: Article
Published: Elsevier 2021
Subjects:
Online Access:http://eprints.um.edu.my/26092/
https://doi.org/10.1016/j.compbiomed.2021.104972
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.um.eprints.26092
record_format eprints
spelling my.um.eprints.260922021-12-29T04:22:42Z http://eprints.um.edu.my/26092/ Explainable deep learning ensemble for food image analysis on edge devices Tahir, Ghalib Ahmed Loo, Chu Kiong QA75 Electronic computers. Computer science Food recognition systems recently garnered much research attention in the relevant field due to their ability to obtain objective measurements for dietary intake. This feature contributes to the management of various chronic conditions. Challenges such as inter and intraclass variations alongside the practical applications of smart glasses, wearable cameras, and mobile devices require resource-efficient food recognition models with high classification performance. Furthermore, explainable AI is also crucial in health-related domains as it characterizes model performance, enhancing its transparency and objectivity. Our proposed architecture attempts to address these challenges by drawing on the strengths of the transfer learning technique upon initializing MobiletNetV3 with weights from a pre-trained model of ImageNet. The MobileNetV3 achieves superior performance using the squeeze and excitation strategy, providing unequal weight to different input channels and contrasting equal weights in other variants. Despite being fast and efficient, there is a high possibility for it to be stuck in the local optima like other deep neural networks, reducing the desired classification performance of the model. Thus, we overcome this issue by applying the snapshot ensemble approach as it enables the M model in a single training process without any increase in the required training time. As a result, each snapshot in the ensemble visits different local minima before converging to the final solution which enhances recognition performance. On overcoming the challenge of explainability, we argue that explanations cannot be monolithic, since each stakeholder perceive the results’, explanations based on different objectives and aims. Thus, we proposed a user-centered explainable artificial intelligence (AI) framework to increase the trust of the involved parties by inferencing and rationalizing the results according to needs and user profile. Our framework is comprehensive in terms of a dietary assessment app as it detects Food/Non-Food, food categories, and ingredients. Experimental results on the standard food benchmarks and newly contributed Malaysian food dataset for ingredient detection demonstrated superior performance on an integrated set of measures over other methodologies. © 2021 Elsevier Ltd Elsevier 2021 Article PeerReviewed Tahir, Ghalib Ahmed and Loo, Chu Kiong (2021) Explainable deep learning ensemble for food image analysis on edge devices. Computers in Biology and Medicine, 139. p. 104972. ISSN 0010-4825 https://doi.org/10.1016/j.compbiomed.2021.104972 doi:10.1016/j.compbiomed.2021.104972
institution Universiti Malaya
building UM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Malaya
content_source UM Research Repository
url_provider http://eprints.um.edu.my/
topic QA75 Electronic computers. Computer science
spellingShingle QA75 Electronic computers. Computer science
Tahir, Ghalib Ahmed
Loo, Chu Kiong
Explainable deep learning ensemble for food image analysis on edge devices
description Food recognition systems recently garnered much research attention in the relevant field due to their ability to obtain objective measurements for dietary intake. This feature contributes to the management of various chronic conditions. Challenges such as inter and intraclass variations alongside the practical applications of smart glasses, wearable cameras, and mobile devices require resource-efficient food recognition models with high classification performance. Furthermore, explainable AI is also crucial in health-related domains as it characterizes model performance, enhancing its transparency and objectivity. Our proposed architecture attempts to address these challenges by drawing on the strengths of the transfer learning technique upon initializing MobiletNetV3 with weights from a pre-trained model of ImageNet. The MobileNetV3 achieves superior performance using the squeeze and excitation strategy, providing unequal weight to different input channels and contrasting equal weights in other variants. Despite being fast and efficient, there is a high possibility for it to be stuck in the local optima like other deep neural networks, reducing the desired classification performance of the model. Thus, we overcome this issue by applying the snapshot ensemble approach as it enables the M model in a single training process without any increase in the required training time. As a result, each snapshot in the ensemble visits different local minima before converging to the final solution which enhances recognition performance. On overcoming the challenge of explainability, we argue that explanations cannot be monolithic, since each stakeholder perceive the results’, explanations based on different objectives and aims. Thus, we proposed a user-centered explainable artificial intelligence (AI) framework to increase the trust of the involved parties by inferencing and rationalizing the results according to needs and user profile. Our framework is comprehensive in terms of a dietary assessment app as it detects Food/Non-Food, food categories, and ingredients. Experimental results on the standard food benchmarks and newly contributed Malaysian food dataset for ingredient detection demonstrated superior performance on an integrated set of measures over other methodologies. © 2021 Elsevier Ltd
format Article
author Tahir, Ghalib Ahmed
Loo, Chu Kiong
author_facet Tahir, Ghalib Ahmed
Loo, Chu Kiong
author_sort Tahir, Ghalib Ahmed
title Explainable deep learning ensemble for food image analysis on edge devices
title_short Explainable deep learning ensemble for food image analysis on edge devices
title_full Explainable deep learning ensemble for food image analysis on edge devices
title_fullStr Explainable deep learning ensemble for food image analysis on edge devices
title_full_unstemmed Explainable deep learning ensemble for food image analysis on edge devices
title_sort explainable deep learning ensemble for food image analysis on edge devices
publisher Elsevier
publishDate 2021
url http://eprints.um.edu.my/26092/
https://doi.org/10.1016/j.compbiomed.2021.104972
_version_ 1720980441585942528
score 13.160551