New combinatorial direction stochastic approximation algorithms

References

• Bertsekas , D. P. and Tsitsiklis , J. N. 2003 . Gradient convergence in gradient methods with errors . SIAM J. Optim , 10 : 627 – 642 . (doi:10.1137/S1052623497331063)
   Web of Science ®Google Scholar
• Blum , J. R. 1954 . Multidimensional stochastic approximation methods . Ann. Math. Statist , 25 : 737 – 744 . (doi:10.1214/aoms/1177728659)
   Google Scholar
• Dai , Y. H. and Yuan , Y. 1999 . Convergence of three-term conjugate gradient methods . Math. Numer. Sinica , 21 : 355 – 362 .
   Google Scholar
• Dai , Y. H. and Yuan , Y. 2000 . Nonlinear Conjugate Gradient Methods , China : Shanghai Scientific and Technical Publishers . in Chinese.
   Google Scholar
• Delyon , B. and Juditsky , A. 1993 . Accelerated stochastic approximation . SIAM J. Optim , 3 : 868 – 881 . (doi:10.1137/0803045)
   Google Scholar
• Fabian , V. 1968 . On asymptotic normality in stochastic approximation . Ann. Math. Statist , 39 : 1327 – 1332 . (doi:10.1214/aoms/1177698258)
   Google Scholar
• Fu , M. C. 1994 . Optimization via simulation: A review . Ann. Oper. Res , 53 : 199 – 247 . (doi:10.1007/BF02136830)
   Google Scholar
• Fu , M. C. and Healy , K. J. 1997 . Techniques for optimization via simulation: An experimental study on an (s, S) inventory system . IIE Trans , 29 : 191 – 199 .
   Web of Science ®Google Scholar
• Kesten , H. 1958 . Accelerated stochastic approximation . Ann. Math. Statist , 29 : 41 – 59 . (doi:10.1214/aoms/1177706705)
   Google Scholar
• Kiefer , J. and Wolfowitz , J. 1952 . Stochastic estimation of the modulus of a regression function . Ann. Math. Statist , 23 : 462 – 466 . (doi:10.1214/aoms/1177729392)
   Google Scholar
• Kushner , H. J. and Clark , D. S. 1978 . Stochastic Approximation for Constrained and Unconstrained Systems , Berlin : Springer .
   Google Scholar
• Moré , J. J. , Garbow , B. S. and Hillstrom , K. E. 1981 . Testing unconstrained optimization software . ACM Trans. Math. Softw , 7 : 17 – 41 . (doi:10.1145/355934.355936)
   Web of Science ®Google Scholar
• Nevel'son , M. B. and Khas'minskij , R. Z. 1973 . Stochastic Approximation and Recursive Estimation , Providence, RI : American Mathematical Society .
   Google Scholar
• Poljak , B. T. and Tsypkin , Y. Z. 1973 . Pseudogradient adaptation and training algorithms . Automat. Remote Control , 12 : 83 – 94 .
   Google Scholar
• Robbins , H. and Monro , S. 1951 . A stochastic approximation method . Ann. Math. Statist , 22 : 400 – 407 . (doi:10.1214/aoms/1177729586)
   Google Scholar
• Schraudolph , N. N. and Graepel , T. Combining conjugate direction methods with stochastic approximation of gradients . Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics . Key West , FL. pp. 7 – 13 .
   Google Scholar
• Spall , J. C. 1992 . Multivariate stochastic approximation using a simultanoues perturbation gradient approximation . IEEE Trans. Automat. Control , 37 : 332 – 341 . (doi:10.1109/9.119632)
   Web of Science ®Google Scholar
• Spall , J. C. 2000 . Adaptive stochastic approximation by the simultaneous perturbation method . IEEE Trans. Automat. Control , 45 : 1839 – 1853 . (doi:10.1109/TAC.2000.880982)
   Web of Science ®Google Scholar
• Spall , J. C. 2003 . Introduction to Stochastic Search and Optimization , New York : Wiley .
   Google Scholar
• Wasan , M. 1970 . Stochastic Approximation , Cambridge, MA : Cambridge University Press .
   Google Scholar
• Xu , Z. 2010 . A combined direction stochastic approximation algorithm . Optim. Lett , 4 : 117 – 129 . (doi:10.1007/s11590-009-0139-4)
   Web of Science ®Google Scholar
• Xu , Z. and Dai , Y. H. 2008 . A stochastic approximation frame algorithm with adaptive directions . Numer. Math. Theory Methods Appl , 1 : 460 – 474 .
   Web of Science ®Google Scholar