Model-Free control performance improvement using virtual reference feedback tuning and reinforcement Q-learning

References

• Basin, M.V., & Ramirez, P.C.R. (2014). A supertwisting algorithm for systems of dimension more than one. IEEE Transactions on Industrial Electronics, 61(11), 6472–6480.
   Web of Science ®Google Scholar
• Bazanella, A.S., Campestrini, L., & Eckhard, D. (2012). Data-Driven controller design: The H₂ approach. Berlin: Springer-Verlag.
   Google Scholar
• Bazanella, A.S., & Neuhaus, T. (2014). Tuning nonlinear controllers with the virtual reference approach. Proceedings of 19^th World Congress of the International Federation of Automatic Control, Cape Town, South Africa, 10269–10274.
   Google Scholar
• Blažič, S., Matko, D., & Škrjanc, I. (2010). Adaptive law with a new leakage term. IET Control Theory & Applications, 4(9), 1533–1542.
   Web of Science ®Google Scholar
• Campestrini, L., Eckhard, D., Gevers, M., & Bazanella, A. (2011). Virtual reference feedback tuning for non-minimum phase plants. Automatica, 47(8), 1778–1784.
   Web of Science ®Google Scholar
• Campi, M.C., Lecchini, A., & Savaresi, S.M. (2002). Virtual reference feedback tuning: A direct method for the design of feedback controllers. Automatica, 38(8), 1337–1346.
   Web of Science ®Google Scholar
• Campi, M.C., & Savaresi, S.M. (2006). Direct nonlinear control design: The Virtual Reference Feedback Tuning (VRFT) approach. IEEE Transactions on Automatic Control, 51(1), 14–27.
   Web of Science ®Google Scholar
• Eckhard, D., & Bazanella, A.S. (2012). Robust convergence of the steepest descent method for data-based control. International Journal of Systems Science, 43(10), 1969–1975.
   Web of Science ®Google Scholar
• Ernst, D., Geurts, P., & Wehenkel, L. (2005). Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6(4), 503–556.
   Google Scholar
• Erokhin, V., & Volkov, V. (2015). Recovering images, registered by device with inexact point-spread function, using Tikhonov's regularized least squares method. International Journal of Artificial Intelligence, 13(1), 123–134.
   Google Scholar
• Esparza, A., Sala, A., & Albertos, P. (2011). Neural networks in virtual reference tuning. Engineering Applications of Artificial Intelligence, 24(6), 983–995.
   Web of Science ®Google Scholar
• Filatovas, E., Podkopaev, D., & Kurasova, O. (2015). A visualization technique for accessing solution pool in interactive methods of multiobjective optimization. International Journal of Computers, Communication & Control, 10(4), 508—519.
   Web of Science ®Google Scholar
• Filip, F.G. (2008). Decision support and control for large-scale complex systems. Annual Reviews in Control, 32(1), 61–70.
   Web of Science ®Google Scholar
• Fliess, M., & Join, C. (2013). Model-free control. International Journal of Control, 86(12), 2228–2252.
   Web of Science ®Google Scholar
• Formentin, S., Savaresi, S.M., & Del Re, L. (2014). Direct multivariable controller tuning for internal combustion engine test benches. Control Engineering Practice, 29(8), 115–122.
   Google Scholar
• Giagkiozis, I., Purshouse, R.C., & Fleming, P.J. (2015). An overview of population-based algorithms for multi-objective optimisation. International Journal of Systems Science, 46(9), 1572–1599.
   Web of Science ®Google Scholar
• Haber, R.E., & Alique, J.R. (2007). Fuzzy logic-based torque control system for milling process optimization. IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, 37(5), 941–950.
   Web of Science ®Google Scholar
• Hafner, R., & Riedmiller, M. (2005). Neural fitted Q iteration - first experiences with a data efficient neural reinforcement learning method. In J. Gama, R. Camacho, P. Brazdil, A. Jorge, and L. Torgo (Eds.), Machine learning: ECML 2005 (Lecture Notes in Computer Science, vol. 3720, pp. 317–328). Berlin: Springer-Verlag.
   Google Scholar
• Hafner, R., & Riedmiller, M. (2007). Neural reinforcement learning controllers for a real robot application. Proceedings of 2007 IEEE Conference on Robotics and Automation, Rome, Italy, 2098–2103.
   Google Scholar
• Hafner, R., & Riedmiller, M. (2011). Reinforcement learning in feedback control. Challenges and benchmarks from technical process control. Machine Learning, 84(1), 137–169.
   Web of Science ®Google Scholar
• Hjalmarsson, H. (2002). Iterative feedback tuning – an overview. International Journal of Adaptive Control and Signal Processing, 16(5), 373–395.
   Web of Science ®Google Scholar
• Hou, Z., & Jin, S. (2011). Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems. IEEE Transactions on Neural Networks, 22(12), 2173–2188.
   PubMed Web of Science ®Google Scholar
• Inteco, Ltd. (2007). Two rotor aerodynamical system, user's manual. Krakow: Inteco Ltd.
   Google Scholar
• Lewis, F., & Vamvoudakis, K.G. (2011). Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 41(1), 14–25.
   PubMed Web of Science ®Google Scholar
• Lewis, F., Vrabie, D., & Vamvoudakis, K.G. (2012). Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Systems Magazine, 32(6), 76–105.
   Web of Science ®Google Scholar
• Liu, Y., Chen, H.-W., & Lu, J.-Q. (2014). Data-based controllability analysis of discrete-time linear time-delay systems. International Journal of Systems Science, 45(11), 2411–2417.
   Web of Science ®Google Scholar
• Liu, D., Javaherian, H., Kovalenko, O., & Huang, T. (2008). Adaptive critic learning techniques for engine torque and air-fuel ratio control. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 38(4), 988–993.
   PubMed Web of Science ®Google Scholar
• Moraes Rêgo, P.H., da Fonseca Neto, J.V., & Ferreira, E.M. (2015). Convergence of the standard RLS method and UDUT factorisation of covariance matrix for solving the algebraic Riccati equation of the DLQR via heuristic approximate dynamic programming. International Journal of Systems Science, 46(11), 2006–2028.
   Web of Science ®Google Scholar
• Passenbrunner, T.E., Formentin, S., Savaresi, S.M., & Del Re, L. (2014). Direct multivariable controller tuning for internal combustion engine test benches. Control Engineering Practice, 29(8), 115–122.
   Web of Science ®Google Scholar
• Precup, R.-E., David, R.-C., Petriu, E.M., Preitl, S., & Radac, M.-B. (2011). Gravitational search algorithms in fuzzy control systems tuning. Proceedings of 18^th World Congress of the International Federation of Automatic Control, Milano, Italy, 13624–13629.
   Google Scholar
• Precup, R.-E., David, R.-C., Petriu, E.M., Preitl, S., & Radac, M.-B. (2013). Fuzzy logic-based adaptive gravitational search algorithm for optimal tuning of fuzzy-controlled servo systems. IET Control Theory & Applications, 7(1), 99–107.
   Web of Science ®Google Scholar
• Precup, R.-E., David, R.-C., Petriu, E.M., Preitl, S., & Radac, M.-B. (2014). Novel adaptive charged system search algorithm for optimal tuning of fuzzy controllers. Expert Systems with Applications, 41(4), 1168–1175.
   Web of Science ®Google Scholar
• Precup, R.-E., Radac, M.-B., Tomescu, M.L., Petriu, E.M., & Preitl, S. (2013). Stable and convergent iterative feedback tuning of fuzzy controllers for discrete-time SISO systems. Expert Systems with Applications, 40(1), 188–199.
   Web of Science ®Google Scholar
• Previdi, F., Schauer, T., Savaresi, S.M., & Hunt, K.J. (2004). Data-driven control design for neuroprotheses: A Virtual Reference Feedback Tuning (VRFT) approach. IEEE Transactions on Control Systems Technology, 12(1), 176–182.
   Web of Science ®Google Scholar
• Prokhorov, D.V., & Wunsch, D.C. (1997). Adaptive critics design. IEEE Transactions on Neural Networks, 8(5), 997–1007.
   PubMed Web of Science ®Google Scholar
• Qin, C., Zhang, H., Luo, Y., & Wang, B. (2014). Finite horizon optimal control of non-linear discrete-time switched systems using adaptive dynamic programming with ϵ-error bound. International Journal of Systems Science, 45(8), 1683–1693.
   Web of Science ®Google Scholar
• Radac, M.-B., Precup, R.-E., & Petriu, E.M. (2015). Model-free primitive-based iterative learning control approach to trajectory tracking of MIMO systems with experimental validation. IEEE Transactions on Neural Networks and Learning Systems, 26(11), 2925–2938.
   PubMed Web of Science ®Google Scholar
• Radac, M.-B., Precup, R.-E., Petriu, E.M., Preitl, S., & Dragos, C.-A. (2013). Data-driven reference trajectory tracking algorithm and experimental validation. IEEE Transactions on Industrial Informatics, 9(4), 2327–2336.
   Web of Science ®Google Scholar
• Spall, J.C., & Cristion, J.A. (1998). Model-free control of nonlinear stochastic systems with discrete-time measurements. IEEE Transactions on Automatic Control, 43(9), 1198–1210.
   Web of Science ®Google Scholar
• Sutton, R.S., & Barto, A.G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
   Google Scholar
• Tikk, D., Johanyák, Z.C., Kovács, S. & Wong, K.W. (2011). Fuzzy rule interpolation and extrapolation techniques: Criteria and evaluation guidelines. Journal of Advanced Computational Intelligence and Intelligent Informatics, 15(3), 254–263.
   Google Scholar
• van Heusden, K., Karimi, A., & Bonvin, D. (2011). Data-driven model reference control with asymptotically guaranteed stability. International Journal of Adaptive Control and Signal Processing, 25(4), 331–351.
   Web of Science ®Google Scholar
• Walsh, T.J., Nouri, A., Li, L., & Littman, M.L. (2007). Planning and learning in environments with delayed feedback. In J.N. Kok et al. (Eds.), Machine learning: ECML 2007 (Lecture Notes in Computer Science, vol. 4701, pp. 442–453). Berlin: Springer-Verlag.
   Google Scholar
• Wang, Z., & Liu, D. (2011). Data-based controllability and observability analysis of linear discrete-time systems. IEEE Transactions on Neural Networks, 22(12), 2388–2392.
   PubMed Web of Science ®Google Scholar
• Wang, F.-Y., Zhang, H., & Liu, D. (2009). Adaptive dynamic programming: an introduction. IEEE Computational Intelligence Magazine, 4(2), 39–47.
   Web of Science ®Google Scholar
• Watkins, C, & Dayan, P. (1992). Q-learning. Machine Learning, 8(3–4), 279–292.
   Web of Science ®Google Scholar
• Werbos, P.J. (1992). Approximate dynamic programming for real-time control and neural modeling. In D.A. White and D.A. Sofge (Eds.), Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches (ch. 13, pp. 493–525). New York, NY: Van Nostrand Reinhold.
   Google Scholar
• Xu, D., Jiang, B., & Shi, P. (2014). A novel model-free adaptive control design for multivariable industrial processes. IEEE Transactions on Industrial Electronics, 61(11), 6391–6398.
   Web of Science ®Google Scholar
• Yan, P., Liu, D., Wang, D., & Ma, H. (2016). Data-driven controller design for general MIMO nonlinear systems via virtual reference feedback tuning and neural networks. Neurocomputing, 171(1), 815–825.
   Web of Science ®Google Scholar
• Yin, S., Li, X., Gao, H., & Kaynak, O. (2015). Data-based techniques focused on modern industry: An overview. IEEE Transactions on Industrial Electronics, 62(1), 657–667.
   Web of Science ®Google Scholar
• Zhai, D., Zhang, Q.-L., & Liu, G.-Y. (2014). Data-driven criteria synthesis of system with two-degree-of-freedom controller. International Journal of Systems Science, 45(11), 2275–2281.
   Web of Science ®Google Scholar
• Zhang, Y., Ding, S.X., Yang, Y., & Li, L. (2015). Data-driven design of two-degree-of-freedom controllers using reinforcement learning techniques. IET Control Theory and Applications, 9(7), 1011–1021.
   Web of Science ®Google Scholar
• Zhang, R.K., Hou, Z.S., Ji, H.H., & Yin, C.K. (2016). Adaptive iterative learning control for a class of non-linearly parameterised systems with input saturations. International Journal of Systems Science, 47(5), 1084–1094.
   Web of Science ®Google Scholar
• Zhang, J., Zhang, H., Wang B., & Cai, T. (2016). Nearly data-based optimal control for linear discrete model-free systems with delays via reinforcement learning. International Journal of Systems Science, 47(7), 1563–1573.
   Web of Science ®Google Scholar