AI-based digital image dietary assessment methods compared to humans and ground truth: a systematic review

Abstract

Objective

Human error estimating food intake is a major source of bias in nutrition research. Artificial intelligence (AI) methods may reduce bias, but the overall accuracy of AI estimates is unknown. This study was a systematic review of peer-reviewed journal articles comparing fully automated AI-based (e.g. deep learning) methods of dietary assessment from digital images to human assessors and ground truth (e.g. doubly labelled water).

Materials and Methods

Literature was searched through May 2023 in four electronic databases plus reference mining. Eligible articles reported AI estimated volume, energy, or nutrients. Independent investigators screened articles and extracted data. Potential sources of bias were documented in absence of an applicable risk of bias assessment tool.

Results

Database and hand searches identified 14,059 unique publications; fifty-two papers (studies) published from 2010 to 2023 were retained. For food detection and classification, 79% of papers used a convolutional neural network. Common ground truth sources were calculation using nutrient tables (51%) and weighed food (27%). Included papers varied widely in food image databases and results reported, so meta-analytic synthesis could not be conducted. Relative errors were extracted or calculated from 69% of papers. Average overall relative errors (AI vs. ground truth) ranged from 0.10% to 38.3% for calories and 0.09% to 33% for volume, suggesting similar performance. Ranges of relative error were lower when images had single/simple foods.

Conclusions

Relative errors for volume and calorie estimations suggest that AI methods align with – and have the potential to exceed – accuracy of human estimations. However, variability in food image databases and results reported prevented meta-analytic synthesis. The field can advance by testing AI architectures on a limited number of large-scale food image and nutrition databases that the field determines to be adequate for training and testing and by reporting accuracy of at least absolute and relative error for volume or calorie estimations.

KEY MESSAGES

• These results suggest that AI methods are in line with – and have the potential to exceed – accuracy of human estimations of nutrient content based on digital food images.

• Variability in food image databases used and results reported prevented meta-analytic synthesis.

• The field can advance by testing AI architectures on a limited number of large-scale food image and nutrition databases that the field determines to be accurate and by reporting accuracy of at least absolute and relative error for volume or calorie estimations.

• Overall, the tools currently available need more development before deployment as stand-alone dietary assessment methods in nutrition research or clinical practice.

Keywords:

• Artificial intelligence
• machine learning
• nutrition assessment
• nutrition surveys
• food images

Acknowledgements

An anonymous reviewer provided valuable feedback on edits that improved readability and applicability for nutrition researchers.

Authors contributions

EH, ES, KCC, and XP contributed equally to the design of the research; KCC and XP contributed to the acquisition of the data; MC, KCC and ES analyzed and interpreted the data; ES and KCC drafted the manuscript; SK and KP contributed to study conceptualization and writing. All authors critically revised the manuscript, and all authors read and approved the final manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author, ES, upon reasonable request.

Additional information

Funding

Funding for this study was provided through NIH grant R21CA250024. The sponsor had no role in the study design; the collection, analysis, and interpretation of data; the writing of the manuscript; or the decision where to submit the paper for publication.