Objectively Measuring Pain Using Facial Expression: Is the Technology Finally Ready?

References

• Tracey I , MantyhPW . The cerebral signature for pain perception and its modulation . Neuron55 ( 3 ), 377 – 391 ( 2007 ).
   PubMed Web of Science ®Google Scholar
• Prkachin KM . Facial pain expression . Pain Manag.1 ( 4 ), 367 – 376 ( 2011 ).
   PubMedGoogle Scholar
• Hadjistavropoulos T , HerrK , PrkachinKMet al. Pain assessment in elderly adults with dementia . Lancet Neurol.13 ( 12 ), 1216 – 1227 ( 2014 ).
   PubMed Web of Science ®Google Scholar
• Kaseweter KA , DrweckiBB , PrkachinKM . Racial differences in pain treatment and empathy in a Canadian sample . Pain Res. Manag.17 ( 6 ), 381 – 384 ( 2012 ).
   PubMed Web of Science ®Google Scholar
• De Ruddere L , GoubertL , PrkachinKM , Louis StevensMA , Van RyckeghemDML , CrombezG . When you dislike patients, pain is taken less seriously . Pain152 ( 10 ), 2342 – 2347 ( 2011 ).
   PubMed Web of Science ®Google Scholar
• Kappesser J , WilliamsAC , PrkachinKM . Testing two accounts of pain underestimation . Pain124 ( 1–2 ), 109 – 116 ( 2006 ).
   PubMed Web of Science ®Google Scholar
• Schultz A , MurphyE , MortonJ , StempelA , Messenger-RiouxC , BennettK . Preverbal, Early Verbal Pediatric Pain Scale (PEPPS): development and early psychometric testing . J. Pediatr. Nurs.14 ( 1 ), 19 – 27 ( 1999 ).
   PubMedGoogle Scholar
• H⊘len JC , SaltvedtI , FayersPM , HjermstadMJ , LogeJH , KaasaS . Doloplus-2, a valid tool for behavioural pain assessment?BMC Geriatr.7 , 29 ( 2007 ).
   PubMedGoogle Scholar
• Abbey J , PillerN , De BellisAet al. The Abbey pain scale: a 1-minnumerical indicator for people with end-stage dementia . Int. J. Palliat. Nurs.10 ( 1 ), 6 – 13 ( 2004 ).
   PubMedGoogle Scholar
• Gélinas C , FillionL , PuntilloKA , ViensC , FortierM . Validation of the critical-care pain observation tool in adult patients . Am. J. Crit. Care15 ( 4 ), 420 – 427 ( 2006 ).
   PubMed Web of Science ®Google Scholar
• Crellin D , SullivanTP , BablFE , O’SullivanR , HutchinsonA . Analysis of the validation of existing behavioral pain and distress scales for use in the procedural setting . Paediatr. Anaesth.17 ( 8 ), 720 – 733 ( 2007 ).
   PubMed Web of Science ®Google Scholar
• Sheu E , VerslootJ , NaderR , KerrD , CraigKD . Pain in the elderly: validity of facial expression components of observational measures . Clin. J. Pain27 ( 7 ), 593 – 601 ( 2011 ).
   PubMed Web of Science ®Google Scholar
• Schiavenato M . Facial expression and pain assessment in the pediatric patient: the primal face of pain . J. Spec. Pediatr. Nurs.13 ( 2 ), 89 – 97 ( 2008 ).
   PubMed Web of Science ®Google Scholar
• Schiavenato M , ByersJF , ScovannerPet al. Neonatal pain facial expression: evaluating the primal face of pain . Pain138 ( 2 ), 460 – 471 ( 2008 ).
   PubMed Web of Science ®Google Scholar
• Chang J , VerslootJ , FashlerSR , McCrystalKN , CraigKD . Pain assessment in children: validity of facial expression items in observational pain scales . Clin. J. Pain31 ( 3 ), 189 – 197 ( 2014 ).
   Web of Science ®Google Scholar
• Bartlett MS , HagerJC , EkmanP , SejnowskiTJ . Measuring facial expressions by computer image analysis . Psychophysiology36 ( 2 ), 253 – 263 ( 1999 ).
   PubMed Web of Science ®Google Scholar
• Pantic M , RothkrantzLJM . Machine understanding of facial expression of pain . J. Behav. Brain Sci.25 ( 4 ), 1 – 3 ( 2002 ).
   Web of Science ®Google Scholar
• Fasel B , LuettinJ . Automatic facial expression analysis: a survey . Pattern Recognit.36 ( 1 ), 259 – 275 ( 2003 ).
   Web of Science ®Google Scholar
• Brahnam S , ChuangCF , ShihFY , SlackMR . Machine recognition and representation of neonatal facial displays of acute pain . Artif. Intell. Med.36 ( 3 ), 211 – 222 ( 2006 ).
   PubMed Web of Science ®Google Scholar
• Ashraf AB , LuceyS , CohnJFet al. The painful face – pain expression recognition using active appearance models . Image Vis. Comput.27 ( 12 ), 1788 – 1796 ( 2009 ).
   PubMed Web of Science ®Google Scholar
• Sikka K , AhmedAA , DiazDet al. Automated assessment of children’s postoperative pain using computer vision . Pediatrics136 ( 1 ), 1 – 8 ( 2015 ).
   PubMed Web of Science ®Google Scholar
• Pantic M , PatrasI . Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences . IEEE Trans. Syst. Man Cybern. B Cybern.36 ( 2 ), 433 – 449 ( 2006 ).
   PubMed Web of Science ®Google Scholar
• Littlewort G , WhitehillJ , WuTet al. The computer expression recognition toolbox (CERT) . Presented at : 2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, FG 2011.CA, USA , 21–25March 2011 .
   Google Scholar
• Kaltwang S , RudovicO , PanticM . Continuous pain intensity estimation from facial expressions . In : Advances in Visual Computing. ISVC 2012. Lecture Notes in Computer Science.BebisG , BoyleR , ParvinBet al. Eds ). Springer , Berlin, Germany , 368 – 377 ( 2012 ).
   Google Scholar
• Wolf K , RaedlerT , HenkeKet al. The face of pain – a pilot study to validate the measurement of facial pain expression with an improved electromyogram method . Pain Res. Manag.10 ( 1 ), 15 – 19 ( 2005 ).
   PubMedGoogle Scholar
• Ekman P , FriesenWV . Facial Action Coding System: a technique for the measurement of facial movement . Environ. Psych. Nonverb. Behavior1 ( 1 ), 56 – 75 ( 1976 ).
   Web of Science ®Google Scholar
• Craig KD , PatrickCJ . Facial expression during induced pain . J. Pers. Soc. Psychol.48 ( 4 ), 1080 – 1091 ( 1985 ).
   PubMed Web of Science ®Google Scholar
• Peters JWB , KootHM , GrunauREet al. Neonatal Facial Coding System for assessing postoperative pain in infants: item reduction is valid and feasible . Clin. J. Pain19 ( 6 ), 353 – 363 ( 2003 ).
   PubMed Web of Science ®Google Scholar
• Grunau RVE , CraigKD . Pain expression in neonates: facial action and cry . Pain28 ( 3 ), 395 – 410 ( 1987 ).
   PubMed Web of Science ®Google Scholar
• Prkachin KM , MercerSR . Pain expression in patients with shoulder pathology: validity, properties and relationship to sickness impact . Pain39 ( 3 ), 257 – 265 ( 1989 ).
   PubMed Web of Science ®Google Scholar
• Prkachin KM . The consistency of facial expressions of pain: a comparison across modalities . Pain51 ( 3 ), 297 – 306 ( 1992 ).
   PubMed Web of Science ®Google Scholar
• Prkachin KM , SolomonPE . The structure, reliability and validity of pain expression: evidence from patients with shoulder pain . Pain139 ( 2 ), 267 – 274 ( 2008 ).
   PubMed Web of Science ®Google Scholar
• Kunz M , MyliusV , SchepelmannK , LautenbacherS . Impact of age on the facial expression of pain . J. Psychosom. Res.64 ( 3 ), 311 – 318 ( 2008 ).
   PubMed Web of Science ®Google Scholar
• Kunz M , ScharmannS , HemmeterU , SchepelmannK , LautenbacherS . The facial expression of pain in patients with dementia . Pain133 ( 1–3 ), 221 – 228 ( 2007 ).
   PubMed Web of Science ®Google Scholar
• Rahu MA , GrapMJ , CohnJF , MunroCL , LyonDE , SesslerCN . Facial expression as an indicator of pain in critically ill intubated adults during endotracheal suctioning . Am. J. Crit. Care22 ( 5 ), 412 – 422 ( 2013 ).
   PubMed Web of Science ®Google Scholar
• Kohler CG , TurnerT , StolarNMet al. Differences in facial expressions of four universal emotions . Psychiatry Res.128 ( 3 ), 235 – 244 ( 2004 ).
   PubMed Web of Science ®Google Scholar
• Kunz M , MyliusV , SchepelmannK , LautenbacherS . On the relationship between self-report and facial expression of pain . J. Pain5 ( 7 ), 368 – 376 ( 2004 ).
   PubMed Web of Science ®Google Scholar
• LeResche L , DworkinSF , WilsonL , EhrlichKJ . Effect of temporomandibular disorder pain duration on facial expressions and verbal report of pain . Pain51 ( 3 ), 289 – 295 ( 1992 ).
   PubMed Web of Science ®Google Scholar
• Hsu K-T , ShumanSK , HamamotoDT , HodgesJS , FeldtKS . The application of facial expressions to the assessment of orofacial pain in cognitively impaired older adults . J. Am. Dent. Assoc.138 ( 7 ), 963 – 969 ( 2007 ).
   PubMed Web of Science ®Google Scholar
• Kunz M , GruberA , LautenbacherS . Sex differences in facial encoding of pain . J. Pain7 ( 12 ), 915 – 928 ( 2006 ).
   PubMed Web of Science ®Google Scholar
• Jiang B , ValstarM , MartinezB , PanticM . A dynamic appearance descriptor approach to facial actions temporal modeling . IEEE Trans. Cybern.44 ( 2 ), 161 – 174 ( 2014 ).
   PubMed Web of Science ®Google Scholar
• Bartlett MS , LittlewortG , LainscsekC , FaselI , MovellanJ . Machine learning methods for fully automatic recognition of facial expressions and facial actions . Presented at : 2004 IEEE International Conference on Systems, Man and Cybernetics (IEE Cat. No.04CH37583).The Hague, The Netherlands , 10–13October 2004 .
   Google Scholar
• Bartlett MS , LittlewortG , FrankM , LainscsekC , FaselI , MovellanJ . Fully automatic facial action recognition in spontaneous behavior . Presented at : The 7th International Conference on Automatic Face and Gesture Recognition (FGR06).Southampton, UK , 10–12April 2006 .
   Google Scholar
• Cohn JF , ZlochowerAJ , LienJ , KanadeT . Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding . In : What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS).EkmanP , RosenbergEL ( Eds ). Oxford University Press , Oxford, UK ( 2012 ).
   Google Scholar
• Pantic M , PatrasI . Detecting facial actions and their temporal segments in nearly frontal-view face image sequences . 2005 IEEE Int. Conf. Syst. Man. Cybern.4 , 3358 – 3363 ( 2005 ).
   Google Scholar
• Heiderich TM , LeslieAT , GuinsburgR . Neonatal procedural pain can be assessed by computer software that has good sensitivity and specificity to detect facial movements . Acta Paediatr.104 ( 2 ), e63 – e69 ( 2015 ).
   PubMed Web of Science ®Google Scholar
• Dimberg U . Facial electromyography and emotional reactions . Psychophysiology27 ( 5 ), 481 – 494 ( 1990 ).
   PubMed Web of Science ®Google Scholar
• Wolf K , MassR , IngenbleekT , KieferF , NaberD , WiedemannK . The facial pattern of disgust, appetence, excited joy and relaxed joy: an improved facial EMG study . Scand. J. Psychol.46 ( 5 ), 403 – 409 ( 2005 ).
   PubMed Web of Science ®Google Scholar
• Huang CN , ChenCH , ChungHY . The review of applications and measurements in facial electromyography . J. Med. Biol. Eng.25 ( 1 ), 15 – 20 ( 2005 ).
   Google Scholar
• De Luca CJ . The use of surface electromyography in biomechanics . J. Appl. Biomech.13 ( 2 ), 135 – 163 ( 1997 ).
   Web of Science ®Google Scholar
• Aung MSH , KaltwangS , Romera-ParedesBet al. The automatic detection of chronic pain-related expression: requirements, challenges and the multimodal EmoPain dataset . IEEE Trans. Affect. Comput.7 ( 4 ), 435 – 451 ( 2016 ).
   PubMed Web of Science ®Google Scholar
• Hamedi M , SallehSH , TingC , AstarakiM , NoorA . Robust facial expression recognition for MuCI: a comprehensive neuromuscular signal analysis . IEEE Trans. Affect. Comput. ( 99 ), 1 ( 2016 ).
   Google Scholar
• Pasin L , NardelliP , PintaudiMet al. Closed-loop delivery systems versus manually controlled administration of total IV anesthesia: a meta-analysis of randomized clinical trials . Anesth. Analg.124 ( 2 ), 456 – 464 ( 2017 ).
   PubMed Web of Science ®Google Scholar
• Hemmerling TM , ArbeidE , WehbeMet al. Evaluation of a novel closed-loop total intravenous anaesthesia drug delivery system: a randomized controlled trial . Br. J. Anaesth.110 ( 6 ), 1031 – 1039 ( 2013 )
   PubMed Web of Science ®Google Scholar
• Fatoorechi M , ParkinsonJ , PranceRJ , PranceH , SethAK , SchwartzmanDJ . A comparative study of electrical potential sensors and Ag/AgCl electrodes for characterising spontaneous and event related electroencephalagram signals . J. Neurosci. Methods251 , 7 – 16 ( 2015 ).
   PubMed Web of Science ®Google Scholar
• McGhee JT , HamediM , FatoorechiMet al. Towards a novel biometric facial input for emotion recognition and assistive technology for virtual reality . Presented at : 11th International Conference on Disability, Virtual Reality and Associated.CA, USA , 20–22September 2016 .
   Google Scholar