Neuromuscular control of the point to point and oscillatory movements of a sagittal arm with the actor–critic reinforcement learning method

References

• Albus JS A new approach to manipulator control: the cerebellar model articulation control (CMAC) J. Dyn. Syst. Meas. Contr. Transactions ASME Series G 3 220 227 1975
   Google Scholar
• Althoefer , L , Krekelberg , B , Husmeire , D and Seneviratne , L . 2001 . Reinforcement learning in a rule-based navigator for robot manipulators . Neurocomputing , 37 : 51 – 70 .
   Google Scholar
• Barto , AF , Sutton , S and Anderson , W . 1983 . Neuronlike adaptive elements that can solve difficult systems . IEEE Trans. Sys. Man Cyber. , 13 : 834 – 846 .
   Web of Science ®Google Scholar
• Dariush , B , Parnianpour , M and Hemami , H . 1998 . Stability and a control strategy of a multi-link musculoskeletal model with applications in FES . IEEE Trans. Biomed. Eng. , 45 : 3 – 14 .
   PubMed Web of Science ®Google Scholar
• Dinneen , JA and Hemami , H . 1993 . Stability and movement of a one-link neuromusculoskeletal sagittal arm . IEEE Trans. Biomed. Eng. , 40 : 541 – 548 .
   PubMed Web of Science ®Google Scholar
• Doya , K , Kimura , H and Miyamura , A . 2001 . Motor control: neural models and system theory . Int. J. Appl. Math. Comput. Sci. , 11 : 101 – 128 .
   Google Scholar
• Gielen , CC and Houk , C . 1987 . A model of the motor servo: incorporating nonlinear spindle receptor and muscle mechanical properties . Biol. Cyber. , 57 : 217 – 231 .
   PubMed Web of Science ®Google Scholar
• Golkhou , V , Parnianpour , M and Lucas , C . 2004 . The role of multisensor data fusion in neuromuscular control of a sagittal arm with a pair of muscles using actor–critic reinforcement learning method . Technol. Health Care , 12 : 425 – 438 .
   PubMedGoogle Scholar
• Golkhou , V , Lucas , C and Parnianpour , M . 2004 . Application of actor–critic reinforcement learning method for control of a sagittal arm during oscillatory movement . Biomed. Eng. Appl. Basis Commun. , 16 : 305 – 312 .
   Google Scholar
• Holm , S , Indahl , A and Solomonow , M . 2002 . Sensorimotor control of the spine . J. Electromyogr. Kinesiol. , 12 : 209 – 234 .
   Google Scholar
• Houk JC Adams JL Barto G A model of how the basal ganglia generate and use neural signals that predict reinforcement Models of Information Processing in the Basal Ganglia Cambridge, MA, USA MIT Press 1994 249 270
   Google Scholar
• Iqbal , K , Hemami , H and Simon , S . 1993 . Stability and control of a frontal four-link biped system . IEEE Trans. Biomed. Eng. , 40 : 1007 – 1018 .
   PubMedGoogle Scholar
• Izawa J Kondo T Ito K Biological robot arm motion through reinforcement learning Int. Conf. Rob. Aut. Washington DC 3398 3403 2002
   Google Scholar
• Izawa J Kondo T Ito K Motor learning using reinforcement learning with neural internal model Proc. IEEE Int. Conf. Rob. Aut. (ICRA'03) Taipei, Taiwan, (in press), www http://www.ito.dis.titech.ac.jp/~izawa/index.html 2003
   Google Scholar
• Jalics , L , Parnianpour , M , Barin , K and Hemami , H . 1999 . Rhythmic movement of a pair of one-link arms: coordination by intermitted control . Comput. Meth. Biomech. Biomed. Eng. , 1 : 29 – 43 .
   Google Scholar
• Jordan MI Wolpert M Computational motor control The Cognitive Neurosciences, MIT Press 1999 601 620
   Google Scholar
• Kawato , M , Furukawa , K and Suzuki , R . 1987 . A hierarchical neural network model for control and learning of voluntary movement . Biol. Cyber. , 57 : 169 – 185 .
   PubMed Web of Science ®Google Scholar
• Kawato , M , Uno , Y , Isobe , M and Suzuki , R . 1988 . Hierarchical neural network model for voluntary movement with applications to robotics . IEEE Contr. Syst. Mag. , 8 : 8 – 16 .
   Google Scholar
• Kim J Hemami H Coordinated three-dimensional motion of the head and torso by dynamic neural networks IEEE Trans. Syst. Man Cyber. part B 653 666 1998
   Google Scholar
• Kim J Neural excitation training, and control of biological system Ph. D. Dissertation Columbus, Ohio Department of Electrical Engineering, The Ohio State University 1995
   Google Scholar
• Kim , J and Hemami , H . 1995 . Control of a one-link arm by burst signal generators . Biol. Cyber. , 73 : 37 – 47 .
   PubMed Web of Science ®Google Scholar
• Kretchmar RM A synthesis of reinforcement learning and robust control theory Ph. D. Dissertation Department of Computer Engineering, The Colorado State University 2000
   Google Scholar
• Martin , P and Millan , R . 2000 . Robot arm reaching through neural inversions and reinforcement learning . Robot. Auton. Syst. , 31 : 227 – 246 .
   Web of Science ®Google Scholar
• Pandy , M , Anderson , F and Hull , D . 1992 . A parameter optimization approach for the optimal control of large scale, musculoskeletal systems . J. Biomech. , 114 : 450 – 460 .
   Google Scholar
• Parnianpour , M , Wang , JC , Shirazi-Adl , A , Khayatian , B and Lafferriere , G . 1999 . A computational method for simulation of trunk motion: toward a theoretical based quantitative assessment of trunk performance . Biomed. Eng. Appl. Basis Commun. , 11 : 27 – 38 .
   Google Scholar
• Ross , EC , Parnianpour , M and Martin , D . 1993 . The effects of resistance level on muscle coordination patterns and movement profile during trunk extension . Spine , 18 : 1829 – 1838 .
   PubMed Web of Science ®Google Scholar
• Smith RL Intelligent motion control with an artificial cerebellum Ph. D. Dissertation New Zealand Department of Electrical Engineering, University of Auckland 1998
   Google Scholar
• Solomonow , M , Baratta , RV , Zhou , BH , Burger , E , Zieske , A and Gedalia , A . 2003 . Muscular dysfunction elicited by creep of lumbar viscoelactic tissue . J. Electromyogr. Kinesiol. , 13 : 381 – 396 .
   PubMed Web of Science ®Google Scholar
• Sparto , PJ , Parnianpour , M and Reinsel , E . 1997 . Spectral and temporal responses of trunk extensor EMG to an isometric endurance test . Spine , 22 : 418 – 425 .
   PubMed Web of Science ®Google Scholar
• Suttan RS Barto AF Reinforcement Learning: An Introduction, MIT Press 1998
   Google Scholar
• Sutton , RS . 1996 . Generalization in reinforcement learning: successful examples using sparse coarse coding . Advances in Neural Information Processing Systems , 8 : 1038 – 1044 .
   Google Scholar
• Tham , CK . 1995 . Reinforcement learning of multiple tasks using a hierarchical CMAC architecture . Robot. Auton. Syst. , 15 : 247 – 274 .
   Web of Science ®Google Scholar
• Ulusoy , I , Parnianpour , M , Berme , N and Simon , R . 2001 . A neural network system with reinforcement learning to control a dynamic arm model . Biomed. Eng. Appl. Basis Commun. , 13 : 117 – 123 .
   Google Scholar
• Van Dieen , JH , Selen , LP and Cholewicki , J . 2003 . Trunk muscle activation in low-back pain patients, an analysis of the literature . J. Electromyogr. Kinesiol. , 13 : 333 – 351 .
   PubMed Web of Science ®Google Scholar
• Wang , JL , Parnianpour , M , Shirazi-Adl , A and Engin , E . 2000 . Finite-element analysis of a lumbar motion segment in combined compression and sagittal flexion, effect of loading rate . Spine , 25 : 310 – 318 .
   PubMed Web of Science ®Google Scholar
• Winters J Concepts in neuromuscular modeling Three Dimensional Analysis of Human Movement Allard P Stokes I Blanchi J Champaign, IL Human Kinetics chapter 12 1995 257 292
   Google Scholar
• Wolpert , DM . 1997 . Computational approaches in motor control . Trends Cogn. Sci. , 1 : 206 – 209 .
   Google Scholar
• Wolpert , DM , Miall , RC and Kawato , M . 1998 . Internal models in the cerebellum . Trends Cognitive Sci. , 2 : 338 – 347 .
   PubMed Web of Science ®Google Scholar
• Zajac , F . 1989 . Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control . Crit. Rev. Biomed. Eng. , 17 : 359 – 411 .
   PubMed Web of Science ®Google Scholar