AI-based digital image dietary assessment methods compared to humans and ground truth: a systematic review

References

• Limketkai BN, Mauldin K, Manitius N, et al. The age of artificial intelligence: use of digital technology in clinical nutrition. Curr Surg Rep. 2021;9(7):1. doi:10.1007/s40137-021-00297-3.
   Web of Science ®Google Scholar
• Côté M, Lamarche B. Artificial intelligence in nutrition research: perspectives on current and future applications. Appl Physiol Nutr Metab. 2021;47(1):1–8. doi:10.1139/apnm-2021-0448.
   PubMed Web of Science ®Google Scholar
• Zhao X, Xu X, Li X, et al. Emerging trends of technology-based dietary assessment: a perspective study. Eur J Clin Nutr. 2021;75(4):582–16. doi:10.1038/s41430-020-00779-0.
   PubMed Web of Science ®Google Scholar
• Oliveira Chaves L, Gomes Domingos AL, Louzada Fernandes D, et al. Applicability of machine learning techniques in food intake assessment: a systematic review. Crit Rev Food Sci Nutr. 2023;63(7):902–919. doi:10.1080/10408398.2021.1956425.
   PubMed Web of Science ®Google Scholar
• Gemming L, Utter J, Mhurchu CN. Image-assisted dietary assessment: a systematic review of the evidence. J Acad Nutr Diet. 2015;115(1):64–77. doi:10.1016/j.jand.2014.09.015.
   PubMed Web of Science ®Google Scholar
• Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56. doi:10.1038/s41591-018-0300-7.
   PubMed Web of Science ®Google Scholar
• Lu Y, Stathopoulou T, Vasiloglou MF, et al. An artificial intelligence-based system to assess nutrient intake for hospitalised patients. IEEE Trans Multimedia. 2021;23:1136–1147. doi:10.1109/TMM.2020.2993948.
   Web of Science ®Google Scholar
• Shermila PJ, Milton A. Estimation of protein from the images of health drink powders. J Food Sci Technol. 2020;57(5):1887–1895. doi:10.1007/s13197-019-04224-4.
   PubMed Web of Science ®Google Scholar
• Fang S, Shao Z, Kerr DA, et al. An end-to-end image-based automatic food energy estimation technique based on learned energy distribution images: protocol and methodology. Nutrients. 2019;11(4):877. doi:10.3390/nu11040877.
   PubMed Web of Science ®Google Scholar
• Vasiloglou MF, Mougiakakou S, Aubry E, et al. A comparative study on carbohydrate estimation: goCARB vs. Dietitians. Nutrients. 2018;10(6):741. doi:10.3390/nu10060741.
   PubMed Web of Science ®Google Scholar
• Todd LE, Wells NM, Wilkins JL, et al. Digital food image analysis as a measure of children’s fruit and vegetable consumption in the elementary school cafeteria: a description and critique. J Hunger Environ Nutr. 2017;12(4):516–528. doi:10.1080/19320248.2016.1275996.
   Web of Science ®Google Scholar
• Burke LE, Wang J, Sevick MA. Self-monitoring in weight loss: a systematic review of the literature. J Am Diet Assoc. 2011;111(1):92–102. doi:10.1016/j.jada.2010.10.008.
   PubMedGoogle Scholar
• Subar AF, Freedman LS, Tooze JA, et al. Addressing current criticism regarding the value of self-report dietary data. J Nutr. 2015;145(12):2639–2645. doi:10.3945/jn.115.219634.
   PubMed Web of Science ®Google Scholar
• Lissner L, Troiano RP, Midthune D, et al. OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI. Int J Obes. 2007;31(6):956–961. doi:10.1038/sj.ijo.0803527.
   Web of Science ®Google Scholar
• Wehling H, Lusher J. People with a body mass index ≥ 30 under-report their dietary intake: a systematic review. J Health Psychol. 2019;24(14):2042–2059. doi:10.1177/1359105317714318.
   PubMed Web of Science ®Google Scholar
• Boushey CJ, Spoden M, Zhu FM, et al. New mobile methods for dietary assessment: review of image-assisted and image-based dietary assessment methods. Proc Nutr Soc. 2017;76(3):283–294. doi:10.1017/S0029665116002913.
   PubMed Web of Science ®Google Scholar
• Lieffers JRL, Hanning RM. Dietary assessment and self-monitoring: with nutrition applications for mobile devices. Can J Diet Pract Res. 2012;73(3):e253–e260. doi:10.3148/73.3.2012.e253.
   PubMedGoogle Scholar
• Wang J, Sereika SM, Chasens ER, et al. Effect of adherence to self-monitoring of diet and physical activity on weight loss in a technology-supported behavioral intervention. Patient Prefer Adherence. 2012;6:221–226. doi:10.2147/PPA.S28889.
   PubMed Web of Science ®Google Scholar
• Yang Y, Jia W, Bucher T, et al. Image-based food portion size estimation using a smartphone without a fiducial marker. Public Health Nutr. 2019;22(7):1180–1192.
   PubMed Web of Science ®Google Scholar
• Chen H-C, Jia W, Yue Y, et al. Model-based measurement of food portion size for image-based dietary assessment using 3D/2D registration. Meas Sci Technol. 2013;24(10):105701. doi:10.1088/0957-0233/24/10/105701.
   Web of Science ®Google Scholar
• Huang J, Ding H, McBride S, et al. Use of smartphones to estimate carbohydrates in foods for diabetes management. Stud Health Technol Inform. 2015;214:121–127.
   PubMedGoogle Scholar
• Ege T, Yanai K. Simultaneous estimation of dish locations and calories with multi-task learning. IEICE Trans Inf & Syst. 2019;E102.D(7):1240–1246. doi:10.1587/transinf.2018CEP0004.
   Google Scholar
• Ege T, Yanai K. Image-based food calorie estimation using recipe information. IEICE Trans Inf & Syst. 2018;E101.D(5):1333–1341. doi:10.1587/transinf.2017MVP0027.
   Google Scholar
• Wang W, Min W, Li T, et al. A review on vision-based analysis for automatic dietary assessment. Trends Food Sci Technol. 2022;122:223–237. doi:10.1016/j.tifs.2022.02.017.
   Web of Science ®Google Scholar
• Höchsmann C, Martin CK. Review of the validity and feasibility of image-assisted methods for dietary assessment. Int J Obes. 2020;44(12):2358–2371. doi:10.1038/s41366-020-00693-2.
   Web of Science ®Google Scholar
• Doulah A, McCrory MA, Higgins JA, et al. A systematic review of technology-driven methodologies for estimation of energy intake. IEEE Access. 2019;7:49653–49668. doi:10.1109/access.2019.2910308.
   PubMed Web of Science ®Google Scholar
• Dalakleidi KV, Papadelli M, Kapolos I, et al. Applying image-based food recognition systems on dietary assessment: a systematic review. Adv Nutr. 2022;13(6):2590–2619. doi:10.1093/advances/nmac078.
   PubMed Web of Science ®Google Scholar
• Tahir GA, Loo CK. A comprehensive survey of image-based food recognition and volume estimation methods for dietary assessment. Healthcare. 2021 2021;9(12):1676. doi:10.3390/healthcare9121676.
   Web of Science ®Google Scholar
• Kaur R, Kumar R, Gupta M. Deep neural network for food image classification and nutrient identification: a systematic review. Rev Endocr Metab Disord. 2023;24(4):633–653. doi:10.1007/s11154-023-09795-4.
   PubMed Web of Science ®Google Scholar
• Huawei Mate 20. 20 Pro unveiled: more AI, better cameras, wireless charging. Egypt Today. 2022.
   Google Scholar
• Farooqui A. Samsung’s Bixby Assistant Can Count Food Calories. Ubergizmo. 2022. https://www.ubergizmo.com/2018/01/samsungs-bixby-assistant-can-count-food-calories/
   Google Scholar
• Finding what works in health care: standards for systematic reviews. 2011.
   Google Scholar
• Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated 2023 Aug). Cochrane; 2023. Available from: www.training.cochrane.org/handbook.
   Google Scholar
• Moher D, Liberati A, Tetzlaff J, PRISMA Group, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi:10.1371/journal.pmed.1000097.
   PubMed Web of Science ®Google Scholar
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. 2021;10(1):89. doi:10.1186/s13643-021-01626-4.
   PubMed Web of Science ®Google Scholar
• Zhu F, Bosch M, Woo I, et al. The use of mobile devices in aiding dietary assessment and evaluation. IEEE J Sel Top Signal Process. 2010;4(4):756–766. doi:10.1109/JSTSP.2010.2051471.
   PubMed Web of Science ®Google Scholar
• Kong F, Tan J. DietCam: automatic dietary assessment with mobile camera phones. Pervasive Mob Comput. 2012;8(1):147–163. doi:10.1016/j.pmcj.2011.07.003.
   Web of Science ®Google Scholar
• Lee CD, Chae J, Schap TE, et al. Comparison of known food weights with image-based portion-size automated estimation and adolescents’ self-reported portion size. J Diabetes Sci Technol. 2012;6(2):428–434. doi:10.1177/193229681200600231.
   PubMedGoogle Scholar
• Pouladzadeh P, Shirmohammadi S, Al-Maghrabi R. Measuring calorie and nutrition from food image. IEEE Trans Instrum Meas. 2014;63(8):1947–1956. doi:10.1109/TIM.2014.2303533.
   Web of Science ®Google Scholar
• Siswantoro J, Prabuwono AS, Abdullah A, et al. Monte carlo method with heuristic adjustment for irregularly shaped food product volume measurement. Sci World J. 2014;2014:1–10. doi:10.1155/2014/683048.
   Google Scholar
• Anthimopoulos M, Dehais J, Shevchik S, et al. Computer vision-based carbohydrate estimation for type 1 patients with diabetes using smartphones. J Diabetes Sci Technol. 2015;9(3):507–515. doi:10.1177/1932296815580159.
   PubMedGoogle Scholar
• Rhyner D, Loher H, Dehais J, et al. Carbohydrate estimation by a mobile phone-based system versus self-estimations of individuals with type 1 diabetes mellitus: a comparative study. J Med Internet Res. 2016;18(5):e5567. doi:10.2196/jmir.5567.
   Web of Science ®Google Scholar
• Dehais J, Anthimopoulos M, Shevchik S, et al. Two-view 3D reconstruction for food volume estimation. IEEE Trans Multimedia. 2017;19(5):1090–1099. doi:10.1109/TMM.2016.2642792.
   Web of Science ®Google Scholar
• Hassannejad H, Matrella G, Ciampolini P, et al. A new approach to image-based estimation of food volume. Algorithms. 2017;10(2):66. doi:10.3390/a10020066.
   Google Scholar
• Minija SJ, Emmanuel WRS. Neural network classifier and multiple hypothesis image segmentation for dietary assessment using calorie calculator. Imag Sci J. 2017;65(7):379–392. doi:10.1080/13682199.2017.1356610.
   Web of Science ®Google Scholar
• Emmanuel WS, Minija SJ. Fuzzy clustering and whale-based neural network to food recognition and calorie estimation for daily dietary assessment. Sādhanā. 2018;43(5):1–19. doi:10.1007/s12046-018-0865-3.
   Web of Science ®Google Scholar
• Lo FPW, Sun Y, Qiu J, et al. Food volume estimation based on deep learning view synthesis from a single depth map. Nutrients. 2018;10(12):2005. doi:10.3390/nu10122005.
   PubMed Web of Science ®Google Scholar
• Subhi MA, Ali SHM, Ismail AG, et al. Food volume estimation based on stereo image analysis. IEEE Instrum Meas Mag. 2018;21(6):36–43. doi:10.1109/MIM.2018.8573592.
   Web of Science ®Google Scholar
• Anzawa M, Amano S, Yamakata Y, et al. Recognition of multiple food items in a single photo for use in a buffet-style restaurant. IEICE Trans Inf Syst. 2019;E102.D(2):410–414. doi:10.1587/transinf.2018EDL8183.
   Google Scholar
• Makhsous S, Mohammad HM, Schenk JM, et al. A novel mobile structured light system in food 3D reconstruction and volume estimation. Sensors. 2019;19(3):564. doi:10.3390/s19030564.
   PubMed Web of Science ®Google Scholar
• Situju SF, Takimoto H, Sato S, et al. Food constituent estimation for lifestyle disease prevention by multi-task CNN. Appl Artif Intel. 2019;33(8):732–746. doi:10.1080/08839514.2019.1602318.
   Web of Science ®Google Scholar
• Herzig D, Nakas CT, Stalder J, et al. Volumetric food quantification using computer vision on a depth-sensing smartphone: preclinical study. JMIR Mhealth Uhealth. 2020;8(3):e15294. doi:10.2196/15294.
   PubMed Web of Science ®Google Scholar
• Jiji GW, Rajesh A. Food sustenance estimation using food image. Int J Image Grap. 2020;20(04):2050034. doi:10.1142/S0219467820500345.
   Google Scholar
• Lo FPW, Sun Y, Qiu J, et al. Point2Volume: a vision-based dietary assessment approach using view synthesis. IEEE Trans Ind Inf. 2020;16(1):577–586. doi:10.1109/TII.2019.2942831.
   Web of Science ®Google Scholar
• Lu Y, Stathopoulou T, Vasiloglou MF, et al. goFOODTM: an artificial intelligence system for dietary assessment. Sensors. 2020;20(15):4283. doi:10.3390/s20154283.
   PubMed Web of Science ®Google Scholar
• Makhsous S, Bharadwaj M, Atkinson BE, et al. DietSensor: automatic dietary intake measurement using mobile 3d scanning sensor for diabetic patients. Sensors. 2020;20(12):3380. doi:10.3390/s20123380.
   PubMed Web of Science ®Google Scholar
• Pillai SM, Mohideen K. Food calorie measurement and classification of food images. Inter J Pharmaceut Res. 2020; doi:10.31838/ijpr/2020.12.04.262.
   Google Scholar
• Chotwanvirat P, Hnoohom N, Rojroongwasinkul N, et al. Feasibility study of an automated carbohydrate estimation system using thai food images in comparison with estimation by dietitians. Front Nutr. 2021;8:732449. doi:10.3389/fnut.2021.732449.
   PubMed Web of Science ®Google Scholar
• Kumar RD, Julie EG, Robinson YH, et al. Recognition of food type and calorie estimation using neural network. J Supercomput. 2021;77(8):8172–8193. doi:10.1007/s11227-021-03622-w.
   Web of Science ®Google Scholar
• Ma P, Lau CP, Yu N, et al. Image-based nutrient estimation for chinese dishes using deep learning. Food Res Int. 2021;147:110437. doi:10.1016/j.foodres.2021.110437.
   PubMed Web of Science ®Google Scholar
• Papathanail I, Brühlmann J, Vasiloglou MF, et al. Evaluation of a novel artificial intelligence system to monitor and assess energy and macronutrient intake in hospitalised older patients. Nutrients. 2021;13(12):4539. doi:10.3390/nu13124539.
   PubMed Web of Science ®Google Scholar
• Yang Z, Yu H, Cao S, et al. Human-Mimetic estimation of food volume from a single-View RGB image using an AI system. Electronics. 2021;10(13):1556. doi:10.3390/electronics10131556.
   PubMed Web of Science ®Google Scholar
• Yuan D, Hu X, Zhang H, et al. An automatic electronic instrument for accurate measurements of food volume and density. Public Health Nutr. 2021;24(6):1248–1255. doi:10.1017/S136898002000275X.
   PubMed Web of Science ®Google Scholar
• Dai Y, Park S, Lee K. Utilizing mask R-CNN for Solid-Volume food instance segmentation and calorie estimation. Appl Sci. 2022;12(21):10938. doi:10.3390/app122110938.
   Google Scholar
• Kadam P, Pandya S, Phansalkar S, et al. FVEstimator: a novel food volume estimator wellness model for calorie measurement and healthy living. Measurement. 2022;198:111294. doi:10.1016/j.measurement.2022.111294.
   Web of Science ®Google Scholar
• Li H, Yang G. Dietary nutritional information autonomous perception method based on machine vision in smart homes. Entropy. 2022;24(7):868. doi:10.3390/e24070868.
   PubMed Web of Science ®Google Scholar
• Ma P, Lau CP, Yu N, et al. Application of deep learning for image-based chinese market food nutrients estimation. Food Chem. 2022;373(Pt B):130994. doi:10.1016/j.foodchem.2021.130994.
   PubMedGoogle Scholar
• Pfisterer KJ, Amelard R, Chung AG, et al. Automated food intake tracking requires depth-refined semantic segmentation to rectify visual-volume discordance in long-term care homes. Sci Rep. 2022;12(1):83. doi:10.1038/s41598-021-03972-8.
   PubMed Web of Science ®Google Scholar
• Prakash ASJ, Sriramya P. Accuracy analysis for image classification and identification of nutritional values using convolutional neural networks in comparison with logistic regression model. J Pharmac Negative Results. 2022;
   Web of Science ®Google Scholar
• Sasaki Y, Sato K, Kobayashi S, et al. Nutrient and food group prediction as orchestrated by an automated image recognition system in a smartphone app (CALO mama): validation study. JMIR Form Res. 2022;6(1):e31875. doi:10.2196/31875.
   PubMedGoogle Scholar
• Minija SJ, Sam Emmanuel WR. Imperialist competitive algorithm-based deep belief network for food recognition and calorie estimation. Evol Intel. 2022;15(2):955–970. doi:10.1007/s12065-019-00265-y.
   Web of Science ®Google Scholar
• Tagi M, Tajiri M, Hamada Y, et al. Accuracy of an artificial intelligence–based model for estimating leftover liquid food in hospitals: validation study. JMIR Form Res. 2022;6(5):e35991. doi:10.2196/35991.
   PubMedGoogle Scholar
• Haque RU, Khan RH, Shihavuddin ASM, et al. Lightweight and Parameter-Optimized Real-Time food calorie estimation from images using CNN-Based approach. Appl Sci. 2022;12(19):9733. doi:10.3390/app12199733.
   Google Scholar
• Zhang Q, He C, Qin W, et al. Eliminate the hardware: mobile terminals-oriented food recognition and weight estimation system. Front Nutr. 2022;9:965801. doi:10.3389/fnut.2022.965801.
   PubMed Web of Science ®Google Scholar
• Nadeem M, Shen H, Choy L, et al. Smart diet diary: real-Time mobile application for food recognition. ASI. 2023;6(2):53. doi:10.3390/asi6020053.
   Google Scholar
• Shao W, Min W, Hou S, et al. Vision-based food nutrition estimation via RGB-D fusion network. Food Chem. 2023;424:136309. doi:10.1016/j.foodchem.2023.136309.
   PubMed Web of Science ®Google Scholar
• Zheng X, Liu C, Gong Y, et al. Food volume estimation by multi-layer superpixel. Math Biosci Eng. 2023;20(4):6294–6311. doi:10.3934/mbe.2023271.
   PubMed Web of Science ®Google Scholar
• LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–444. doi:10.1038/nature14539.
   PubMed Web of Science ®Google Scholar
• Gervis JE, Hennessy E, Shonkoff ET, et al. Weighed plate waste can accurately measure children’s energy consumption from food in Quick-Service restaurants. J Nutr. 2019;150(2):404–410. doi:10.1093/jn/nxz222.
   Web of Science ®Google Scholar
• Urban LE, McCrory MA, Dallal GE, et al. Accuracy of stated energy contents of restaurant foods. Jama. 2011;306(3):287–293. doi:10.1001/jama.2011.993.
   PubMed Web of Science ®Google Scholar
• Thames Q, Karpur A, Norris W, et al. Nutrition5k: towards automatic nutritional understanding of generic food.
   Google Scholar
• Bray H. MIT, Harvard scientists find AI can recognize race from X-rays—and nobody knows how. Boston Globe. 2022. https://www.bostonglobe.com/2022/05/13/business/mit-harvard-scientists-find-ai-can-recognize-race-x-rays-nobody-knows-how/#:∼:text=The%20study%20found%20that%20an, race%20with%2090%20percent%20accuracy
   Google Scholar
• Yu K-H, Beam AL, Kohane IS. Artificial intelligence in healthcare. Nat Biomed Eng. 2018;2(10):719–731. doi:10.1038/s41551-018-0305-z.
   PubMed Web of Science ®Google Scholar
• FitzGerald C, Hurst S. Implicit bias in healthcare professionals: a systematic review. BMC Med Ethics. 2017;18(1):19. doi:10.1186/s12910-017-0179-8.
   PubMed Web of Science ®Google Scholar
• Marcelin JR, Siraj DS, Victor R, et al. The impact of unconscious bias in healthcare: how to recognize and mitigate it. J Infect Dis. 2019;220(220 Suppl 2):S62–S73. doi:10.1093/infdis/jiz214.
   PubMed Web of Science ®Google Scholar
• Amann J, Blasimme A, Vayena E, et al. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20(1):310. doi:10.1186/s12911-020-01332-6.
   PubMed Web of Science ®Google Scholar