A review of computer vision for semi-autonomous control of assistive robotic manipulators (ARMs)

References

• Marasinghe KM. Assistive technologies in reducing caregiver burden among informal caregivers of older adults: a systematic review. Disabil Rehabil Assist Technol. 2016;11:353–360.
   Google Scholar
• Bedaf S, Marti P, Amirabdollahian F, et al. A multi-perspective evaluation of a service robot for seniors: the voice of different stakeholders. Disabil Rehabil Assist Technol. 2017;0:1–8.
   Google Scholar
• Romer G, Stuyt H, Peters A. Cost-savings and economic benefits due to the assistive robotic manipulator (arm). 9th International Conference on Rehabilitation Robotics, 2005 ICORR; 2005 June 28–July 1. Chicago, IL, USA.
   Google Scholar
• Maheu V, Archambault PS, Frappier J, et al. Evaluation of the jaco robotic arm: clinico-economic study for powered wheelchair users with upper-extremity disabilities. IEEE Int Conf Rehabil Robot. 2011;2011:1–5.
   Google Scholar
• Prior SD. An electric wheelchair mounted robotic arm a survey of potential users. J Med Eng Technol. 1990;14:143–154.
   Google Scholar
• KINOVA. Robotic arms series; 2018 [Cited 2019 May 20]. Available from: https://www.kinovarobotics.com/en/products/robotic-arm-series
   Google Scholar
• Exact Dynamics. iARM; 2018. Available from: http://www.exactdynamics.nl/site/?page=iarm. (visited on 05/20/2019)
   Google Scholar
• Kim DJ, Lovelett R, Behal A. An empirical study with simulated adl tasks using a vision-guided assistive robot arm. In: 2009 IEEE International Conference on Rehabilitation Robotics; 2009 June 23–26; Kyoto, Japan. p. 504–509.
   Google Scholar
• Jiang H, Wachs JP, Duerstock BS. Integrated vision-based robotic arm interface for operators with upper limb mobility impairments. IEEE Int Conf Rehabil Robot. 2013;2013:6650447.
   Google Scholar
• Mülling K, Venkatraman A, Valois J, et al. Autonomy infused teleoperation with application to brain computer interface controlled manipulation. Autonomous Robots. August 2017.
   Google Scholar
• Loconsole C, Stroppa F, Bevilacqua V, et al. A robust real-time 3d tracking approach for assisted object grasping. In: Auvray M, Duriez C, editors. Haptics: neuroscience, devices, modeling, and applications. Berlin, Heidelberg: Springer Berlin Heidelberg; 2014. p. 400–408.
   Google Scholar
• Oguntosin VW, Mori Y, Kim H, et al. Design and validation of exoskeleton actuated by soft modules toward neurorehabilitation vision-based control for precise reaching motion of upper limb. Front Neurosci. 2017;11:352.
   Google Scholar
• Chung CS, Wang H, Cooper RA. Functional assessment and performance evaluation for assistive robotic manipulators: literature review. J Spinal Cord Med. 2013;36:273–289.
   Google Scholar
• Kim DJ, Hazlett-Knudsen R, Culver-Godfrey H, et al. How autonomy impacts performance and satisfaction: results from a study with spinal cord injured subjects using an assistive robot. IEEE Trans Syst, Man, Cybern A Syst Hum. 2012;42:2–14.
   Google Scholar
• Parasuraman R, Sheridan TB, Wickens CD. A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern A Syst Hum. 2000;30:286–297.
   Google Scholar
• Pitzer B, Styer M, Bersch C, et al. Towards perceptual shared autonomy for robotic mobile manipulation. In: 2011 IEEE International Conference on Robotics and Automation; 2011 May 9–13; Shanghai, China. p. 6245–6251.
   Google Scholar
• Remazeilles A, Leroux C, Chalubert G. Sam: A robotic butler for handicapped people. In: RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication; 2008 Aug 1–3; Munich, Germany. p. 315–321.
   Google Scholar
• Kim DJ, Wang Z, Behal A. Motion segmentation and control design for ucf-manus;an intelligent assistive robotic manipulator. IEEE/ASME Trans Mechatron. 2012;17:936–948.
   Google Scholar
• Elarbi-Boudihir M, Al-Shalfan KA. Eye-in-hand/eye-to-hand configuration for a wmra control based on visual servoing. In: 2013 IEEE 11th International Workshop of Electronics, Control, Measurement, Signals and their application to Mechatronics; 2013 June 24–26; Toulouse, France. p. 1–6.
   Google Scholar
• Quintero CP, Ramirez O, Jägersand M. Vibi: Assistive vision-based interface for robot manipulation. In: 2015 IEEE International Conference on Robotics and Automation (ICRA); 2015 May 26–30; Seattle, WA, USA. p. 4458–4463.
   Google Scholar
• Jiang H, Zhang T, Wachs JP, et al. Enhanced control of a wheelchair-mounted robotic manipulator using 3-d vision and multimodal interaction. Comput Vis Image Underst. 2016;149:21–31.
   Google Scholar
• Gualtieri M, Kuczynski J, Shultz AM, et al. Open world assistive grasping using laser selection. In: IEEE International Conference on Robotics and Automation (ICRA); IEEE; 2017 May 29–June 3; Singapore, Singapore. p. 4052–4057.
   Google Scholar
• Zhang Z, Huang Y, Chen S, et al. An intention-driven semi-autonomous intelligent robotic system for drinking. Front Neurorobot 2017;11:48.
   Google Scholar
• Arrichiello F, Lillo PD, Vito DD, et al. Assistive robot operated via p300-based brain computer interface. In: 2017 IEEE International Conference on Robotics and Automation (ICRA); 2017 May 29–June 3; Singapore, Singapore. p. 6032–6037.
   Google Scholar
• Zeng H, Wang Y, Wu C, et al. Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback. Front Neurorobot. 2017;11:60.
   Google Scholar
• Microsoft. Kinect Sensor; 2018. Available from: https://msdn.microsoft.com/en-_us/library/hh438998.aspx (visited on 05/20/2019)
   Google Scholar
• Frisoli A, Salsedo F, Bergamasco M, et al. A force-feedback exoskeleton for upper-limb rehabilitation in virtual reality. Applied Bionics Biomech. 2009;6:115–126.
   Google Scholar
• Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun Acm. 1981;24:381–395.
   Google Scholar
• Bay H, Ess A, Tuytelaars T, et al. Speeded-up robust features (surf). Comput Vis Image Underst. 2008;110:346–359.
   Google Scholar
• Dragan AD, Srinivasa SS. A policy-blending formalism for shared control. Int J Robot Res. 2013;32:790–805.
   Google Scholar
• Downey JE, Weiss JM, Muelling K, et al. Blending of brain-machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping. J NeuroEng Rehabil. 2016;13:28.
   Google Scholar
• Schultz C, Gaurav S, Monfort M, et al. Goal-predictive robotic teleoperation from noisy sensors. In: IEEE International Conference on Robotics and Automation (ICRA); IEEE; 2017 May 29–June 3; Singapore, Singapore.
   Google Scholar
• Myers A, Teo CL, Fermüller C, et al. Affordance detection of tool parts from geometric features. In: IEEE International Conference on Robotics and Automation (ICRA); 2015 May 26–30; Seattle, WA, USA. p. 1374–1381.
   Google Scholar
• Csikszentmihalyi M. Flow and the foundations of positive psychology: the collected works of mihaly csikszentmihalyi. Dordrecht: Springer; 2014.
   Google Scholar
• Domae Y, Okuda H, Taguchi Y, et al. Fast graspability evaluation on single depth maps for bin picking with general grippers. In: IEEE International Conference on Robotics and Automation (ICRA); IEEE; 2014 May 31–June 7; Hong Kong, China. p. 1997–2004.
   Google Scholar
• Klingbeil E, Rao D, Carpenter B, et al. Grasping with application to an autonomous checkout robot. In: IEEE International Conference on Robotics and Automation (ICRA); IEEE; 2011 May 9–13; Shanghai, China. p. 2837–2844.
   Google Scholar
• Redmon J, Angelova A. Real-time grasp detection using convolutional neural networks. In: IEEE International Conference on Robotics and Automation (ICRA); IEEE; 2015 May 26–30; Seattle, WA, USA. p. 1316–1322.
   Google Scholar
• Lenz I, Lee H, Saxena A. Deep learning for detecting robotic grasps. Int J Robot Res. 2015;34:705–724.
   Google Scholar
• Miller AT, Knoop S, Christensen HI, et al. Automatic grasp planning using shape primitives. In: 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422); Vol. 2; 2003 Sept 14–19; Taipei, Taiwan, Taiwan. p. 1824–1829.
   Google Scholar
• Huebner K, Ruthotto S, Kragic D. Minimum volume bounding box decomposition for shape approximation in robot grasping. In: 2008 IEEE International Conference on Robotics and Automation; 19–23 May 2008; Pasadena, CA, USA. p. 1628–1633.
   Google Scholar
• Ciocarlie M, Hsiao K, Jones EG, et al. Towards reliable grasping and manipulation in household environments. In: Khatib O., Kumar V., Sukhatme G, editors. Experimental robotics. Springer Tracts in Advanced Robotics, vol 79. Berlin, Heidelberg: Springer; 2014.
   Google Scholar
• Herzog A, Pastor P, Kalakrishnan M, et al. Template-based learning of grasp selection. In: 2012 IEEE International Conference on Robotics and Automation; 2012 May 14–18; Saint Paul, MN, USA. p. 2379–2384.
   Google Scholar
• Krainin M, Henry P, Ren X, et al. Manipulator and object tracking for in-hand 3d object modeling. Int J Rob Res. 2011;30:1311–1327.
   Google Scholar
• Gualtieri M, ten Pas A, Saenko K, et al. High precision grasp pose detection in dense clutter. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 9–14 Oct; 2016. Daejeon, South Korea. p. 598–605.
   Google Scholar
• Klingensmith M, Sirinivasa SS, Kaess M. Articulated robot motion for simultaneous localization and mapping (ARM-SLAM). IEEE Robot Autom Lett. 2016;1:1156–1163.
   Google Scholar
• Zhang Y, Funkhouser T. Deep depth completion of a single rgb-d image. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT. 2018. p. 175–185.
   Google Scholar
• PPMDTec. CamBoard Pico Flexx; 2018 [Cited 2019 May 20]. Available from: https://pmdtec.com/picofamily/
   Google Scholar