The extended Ritz method for functional optimization: overview and applications to single-person and team optimal decision problems

References

• Alessandri , A. , Baglietto , M. , Parisini , T. and Zoppoli , R. 1999 . A neural state estimator with bounded errors for nonlinear systems . IEEE Trans. Automat. Control , 44 : 2028 – 2042 .
   Web of Science ®Google Scholar
• Alessandri , A. , Cuneo , M. , Pagnan , S. and Sanguineti , M. 2007 . A recursive algorithm for nonlinear least–squares problems . Comput. Optim. Appl. , 38 : 195 – 216 .
   Google Scholar
• Alessandri , A. , Parisini , T. , Sanguineti , M. and Zoppoli , R. Neural strategies for nonlinear optimal filtering . Proceedings of IEEE International Conference on Systems Engineering . pp. 44 – 49 . Kobe, , Japan
   Google Scholar
• Alessandri , A. and Sanguineti , M. 2005 . Optimization of approximating networks for optimal fault diagnosis . Optim. Methods Softw. , 20 : 241 – 266 .
   Web of Science ®Google Scholar
• Alessandri , A. , Sanguineti , M. and Maggiore , M. 2002 . Optimization-based learning with bounded error for feedforward neural networks . IEEE Trans. Neural Net. , 13 : 261 – 273 .
   PubMed Web of Science ®Google Scholar
• Apostolopoulos , G. , Williams , D. , Kamat , S. , Guerin , R. , Orda , A. and Przygienda , T. 1999 . QoS routing mechanisms and OSPF extensions . RFC 2676 , August
   Google Scholar
• Attouch , H. 1984 . Variational Convergence for Functions and Operators , London, , England : Pitman Publishing .
   Google Scholar
• Baglietto , M. , Battistelli , G. , Bolla , R. , Bruschi , R. , Davoli , F. and Zoppoli , R. A neural network solution to QoS-IP team-optimal dynamic routing . Proceedings of Joint 44th IEEE Conference on Decision and Control and European Control Conference . pp. 7452 – 7459 . Seville, , Spain
   Google Scholar
• Baglietto , M. , Cervellera , C. , Sanguineti , M. and Zoppoli , R. 2006 . “ Water reservoirs management under uncertainty by approximating networks and learning from data ” . In Topics on System Analysis and Integrated Water Resource Management , Edited by: Castelletti , A. and Soncini Sessa , R. Vol. Chapter 6 , 117 – 139 . Elsevier Science Ltd .
   Google Scholar
• Baglietto , M. , Parisini , T. and Zoppoli , R. 2001 . Numerical solutions to the Witsenhausen counterexample by approximating networks . IEEE Trans. Automat. Control , 46 : 1471 – 1477 .
   Google Scholar
• Baglietto , M. , Parisini , T. and Zoppoli , R. 2001 . Distributed-information neural control. The case of dynamic routing in traffic networks . IEEE Trans. Neural Net. , 12 : 485 – 502 .
   PubMedGoogle Scholar
• Barron , A. R. 1993 . Universal approximation bounds for superpositions of a sigmoidal function . IEEE Trans. Infom. Theory , 39 : 930 – 945 .
   Web of Science ®Google Scholar
• Beard , R. W. and Mclain , T. W. 1998 . Successive Galerkin approximation algorithms for nonlinear optimal and robust control . Int. J. Control , 71 : 717 – 743 .
   Web of Science ®Google Scholar
• Bellman , R. 1957 . Dynamic Programming , Princeton, NJ : Princeton University Press .
   Google Scholar
• Breiman , L. 1993 . Hinging hyperplanes for regression, classification, and function approximation . IEEE Trans. Inform. Theory , 39 : 993 – 1013 .
   Google Scholar
• Chen , V. C.P. , Ruppert , D. and Shoemaker , C. A. 1999 . Applying experimental design and regression splines to high-dimensional continuous-state stochastic dynamic programming . Oper. Res. , 47 : 38 – 53 .
   Web of Science ®Google Scholar
• DeVore , R. A. , Howard , R. and Micchelli , C. 1989 . Optimal nonlinear approximation . Manuscipta Math. , 63 : 469 – 478 .
   Google Scholar
• Felgenhauer , U. On Ritz type discretizations for optimal control problems . Proceeding of 18th IFIP-ICZ Conference, Research Notes in Mathematics . Vol. 386 , pp. 91 – 99 . Chapman-Hall .
   Google Scholar
• Foufoula-Georgiou , E. and Kitanidis , P. K. 1988 . Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems . Water Resour. Res. , 24 : 1345 – 1359 .
   Web of Science ®Google Scholar
• Gelfand , I. M. and Fomin , S. V. 1963 . Calculus of Variations , Englewood Cliffs, NJ : Prentice Hall .
   Google Scholar
• Girosi , F. , Jones , M. and Poggio , T. 1995 . Regularization theory and neural networks architectures . Neural Comput. , 7 : 219 – 269 .
   Web of Science ®Google Scholar
• Giulini , S. and Sanguineti , M. Approximation schemes for functional optimization problems . J. Optim. Theory Appl. to appear ,
   Google Scholar
• Gnecco , G. and Sanguineti , M. 2008 . Accuracy of suboptimal solutions to kernel principal component analysis . Comput. Optim. Appl , DOI:10.1007/s10589-007-9108-y
   Google Scholar
• Gnecco , G. and Sanguineti , M. 2008 . The weight-decay technique in learning from data: An optimization point of view . Comp. Manag. Sci , DOI:10.1007/s10287-008-0072-5
   Google Scholar
• Gnecco , G. and Sanguineti , M. 2008 . Approximation error bounds via Rademacher complexity . Appl. Math. Sci.2 , : 153 – 176 .
   Google Scholar
• Gnecco , G. and Sanguineti , M. Estimates of the approximation error using Rademacher complexity: learning vector-valued functions . J. Inequal. Appl. to appear ,
   Google Scholar
• Grippo , L. 2002 . Convergent on-line algorithms for supervised learning in neural networks . IEEE Trans. Neural Net. , 11 : 1284 – 1299 .
   Google Scholar
• Hager , W. W. 1975 . The Ritz–Trefftz method for state and control constrained optimal control problems . SIAM J. Numer. Anal. , 12 : 854 – 867 .
   Web of Science ®Google Scholar
• Iftar , A. and Davison , E. J. 1998 . A decentralized discrete–time controller for dynamic routing . Int. J. Control , 69 : 599 – 632 .
   Web of Science ®Google Scholar
• Johnson , S. A. , Stedinger , J. R. , Shoemaker , C. , Li , Y. and Tejada-Guibert , J. A. 1993 . Numerical solution of continuous-state dynamic programs using linear and spline interpolation . Oper. Res. , 41 : 484 – 500 .
   Web of Science ®Google Scholar
• Jones , L. K. 1992 . A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training . Ann. Stat. , 20 : 608 – 613 .
   Web of Science ®Google Scholar
• Kainen , P. C. , K˙urková , V. and Sanguineti , V. M. 2003 . Minimization of error functionals over variable-basis functions . SIAM J. Optim. , 14 : 732 – 742 .
   Google Scholar
• Kainen , P. C. , K˙urková , V. and Sanguineti , V. M. 2005 . Rates of approximate minimization of error functionals over Boolean variable-basis functions . J. Mathe. Model. Algorithms , 4 : 355 – 368 .
   Google Scholar
• Kainen , P. C. , K˙urková , V. and Sanguineti , V. M. Complexity of Gaussian radial-basis networks approximating smooth functions . J. Complexity, to appear ,
   Google Scholar
• Kainen , P. C. , K˙urková , V. and Vogt , A. 1999 . Approximation by neural networks is not continuous . Neurocomputing , 29 : 47 – 56 .
   Google Scholar
• Kainen , P. C. , K˙urková , V. and Vogt , A. 2000 . Geometry and topology of continuous best and near best approximations . J. Approx. Theory , 105 : 252 – 262 .
   Google Scholar
• K˙urková , V. 1997 . “ Dimension-independent rates of approximation by neural networks ” . In Computer-Intensive Methods in Control and Signal Processing: Curse of Dimensionality , Edited by: Warwick , K. and Kárný , M. 261 – 270 . Boston : Birkhäuser .
   Google Scholar
• K˙urková , V. , Kainen , P. C. and Kreinovich , V. 1997 . Estimates of the number of hidden units and variation with respect to half-spaces . Neural Net. , 10 : 1061 – 1068 .
   Google Scholar
• K˙urková , V. and Sanguineti , M. 2001 . Bounds on rates of variable–basis and neural–network approximation . IEEE Trans. Inform. Theory , 47 : 2659 – 2665 .
   Web of Science ®Google Scholar
• K˙urková , V. and Sanguineti , M. 2002 . Comparison of worst-case errors in linear and neural network approximation . IEEE Trans. Inform. Theory , 48 : 264 – 275 .
   Web of Science ®Google Scholar
• K˙urková , V. and Sanguineti , M. 2005 . Learning with generalization capability by kernel methods of bounded complexity . J. Complexity , 21 : 350 – 367 .
   Google Scholar
• K˙urková , V. and Sanguineti , M. 2005 . Error estimates for approximate optimization by the extended Ritz method . SIAM J. Optim. , 15 : 461 – 487 .
   Google Scholar
• K˙urková , V. and Sanguineti , M. Approximate minimization of the regularized expected error over kernel models . Math. Oper. Res. to appear ,
   Google Scholar
• K˙urková , V. , Savický , P. and Hlávácková , K. 1998 . Representations and rates of approximation of real–valued Boolean functions by neural networks . Neural Netw. , 11 : 651 – 659 .
   Google Scholar
• Kushner , H. J. and Yin , G. G. 1997 . Stochastic Approximation Algorithms and Applications , New York : Springer Verlag .
   Google Scholar
• Maiorov , V. and Pinkus , A. 1999 . Lower bounds for approximation by MLP neural networks . Neurocomputing , 25 : 81 – 91 .
   Web of Science ®Google Scholar
• Marschak , J. 1955 . Elements for a theory of teams . Manag. Sci. , 1 : 127 – 137 .
   Web of Science ®Google Scholar
• Marschak , J. and Radner , R. 1972 . Economic Theory of Teams , Yale University Press .
   Google Scholar
• Niyogi , P. and Girosi , F. 1996 . On the relationship between generalization error, hypothesis complexity, and sample complexity for radial basis functions . Neural Comput. , 8 : 819 – 842 .
   Google Scholar
• Papadimitriou , C. H. and Tsitsiklis , J. N. 1986 . Intractable problems in control theory . SIAM J. on Control Optim. , 24 : 639 – 654 .
   Web of Science ®Google Scholar
• Parisini , T. and Zoppoli , R. 1994 . Neural networks for feedback feedforward nonlinear control systems . IEEE Trans. Neural Net. , 5 : 436 – 449 .
   PubMedGoogle Scholar
• Parisini , T. and Zoppoli , R. 1998 . Neural approximations for infinite-horizon optimal control of nonlinear stochastic systems . IEEE Trans. Neural Netw. , 9 : 1388 – 1408 .
   PubMed Web of Science ®Google Scholar
• Philbrick , C. R. Jr. and Kitanidis , P. K. 1999 . Improved dynamic programming methods for optimal control of lumped-parameter stochastic systems . Oper. Res. , 49 : 398 – 412 .
   Google Scholar
• Pinkus , A. 1997 . “ Approximation by ridge functions ” . In Surface Fitting and Multiresolution Methods , Edited by: Le Méhauté , A. , Rabut , C. and Schumaker , L. L. 1 – 14 . Nashville, TN : Vanderbilt University Press .
   Google Scholar
• Pinkus , A. 1999 . Approximation theory of the MLP model in neural networks . Acta Numer. , 8 : 143 – 195 .
   Google Scholar
• Radner , R. 1962 . Team decision problems, Ann. Mathe. Stat. , 33 : 857 – 881 .
   Google Scholar
• Ritz , W. 1909 . ¨Uber Eine Neue Methode zur L¨osung Gewisser Variationsprobleme der Mathematischen Physik . J. für die reine und angewandte Mathematik , 135 : 1 – 61 .
   Google Scholar
• Sanguineti , M. 2008 . Universal approximation by ridge computational models: a survey . Open Appl. Math. J. , 2 : 31 – 58 .
   Google Scholar
• Segall , A. 1977 . The modeling of adaptive routing in data–communication networks . IEEE Trans. Commun. , 25 : 85 – 95 .
   Google Scholar
• Singh , A. , Mittal , G. and Nandi , S. K. 2000 . QoS and traffic engineering: MPLS, diffserv and constraint based routing , May Available at http://keskus.hut.fi/tutkimus/ironet/ns2/qosr.pdf
   Google Scholar
• Sirisena , H. R. and Chou , F. S. 1979 . Convergence of the control parametrization Ritz method for nonlinear optimal control problems . J. Optim. Theory Appl. , 29 : 369 – 382 .
   Web of Science ®Google Scholar
• Witsenhausen , H. S. 1968 . A counterexample in stochastic optimum control . SIAM J. Control. , 6 : 131 – 147 .
   Web of Science ®Google Scholar
• Yakowitz , S. 1982 . Dynamic programming applications in water resources . Water Resour. Res. , 18 : 673 – 696 .
   Web of Science ®Google Scholar
• Zoppoli , R. and Parisini , T. 1992 . “ Learning techniques and neural networks for the solution of N–stage nonlinear nonquadratic optimal control problems ” . In Systems, Models and Feedback: Theory and Applications , Edited by: Isidori , A. and Tarn , T. J. 193 – 210 . Boston : Birkhäuser .
   Google Scholar
• Zoppoli , R. , Sanguineti , M. and Parisini , T. 2002 . Approximating networks and the extended Ritz method for the solution of functional optimization problems . J. Optim. Theory Appl. , 112 : 403 – 439 .
   Web of Science ®Google Scholar
• The Network Simulator 2. Available at http://www.isi.edu/nsnam/ns/
   Google Scholar