Safe and efficient imitation learning by clarification of experienced latent space

References

• Tsurumine Y, Cui Y, Uchibe E, et al. Deep reinforcement learning with smooth policy update: application to robotic cloth manipulation. Rob Auton Syst. 2019;112:72–83.
   Web of Science ®Google Scholar
• Itadera S, Kobayashi T, Nakanishi J, et al. Towards physical interaction-based sequential mobility assistance using latent generative model of movement state. Adv Robot. 2021;35(1):64–79.
   Web of Science ®Google Scholar
• James S, Ma Z, Arrojo DR, et al. Rlbench: the robot learning benchmark & learning environment. IEEE Robot Autom Lett. 2020;5(2):3019–3026.
   Web of Science ®Google Scholar
• Sutton RS, Barto AG. Reinforcement learning: an introduction. Cambridge (MA): MIT Press; 2018.
   Google Scholar
• Johannink T, Bahl S, Nair A, et al. Residual reinforcement learning for robot control. In: International Conference on Robotics and Automation. IEEE; 2019. p. 6023–6029.
   Google Scholar
• Schaal S, Ijspeert A, Billard A. Computational approaches to motor learning by imitation. Philos Trans R Soc Lond Ser B Biolog Sci. 2003;358(1431):537–547.
   PubMed Web of Science ®Google Scholar
• Bain M, Sammut C. A framework for behavioural cloning. In: Machine Intelligence; 1995. p. 103–129.
   Google Scholar
• Bojarski M, Del Testa D, Dworakowski D, et al. End to end learning for self-driving cars. arXiv preprint arXiv:160407316. 2016.
   Google Scholar
• Farry KA, Walker ID, Baraniuk RG. Myoelectric teleoperation of a complex robotic hand. IEEE Trans Rob Autom. 1996;12(5):775–788.
   Google Scholar
• O'Doherty JE, Lebedev MA, Ifft PJ, et al. Active tactile exploration using a brain–machine–brain interface. Nature. 2011;479(7372):228–231.
   PubMed Web of Science ®Google Scholar
• Torabi F, Warnell G, Stone P. Behavioral cloning from observation. In: International Joint Conference on Artificial Intelligence. 2018. p. 4950–4957.
   Google Scholar
• Tobias B, Hidehito F, Taisuke K. Behavioral cloning from observation with bi-directional dynamics model. In: IEEE/SICE International Symposium on System Integration. 2021.
   Google Scholar
• Kingma DP, Welling M. Auto-encoding variational Bayes. In: International Conference on Learning Representations. 2014.
   Google Scholar
• Higgins I, Matthey L, Pal A, et al. Beta-vae: learning basic visual concepts with a constrained variational framework. In: International Conference on Learning Representations. 2017.
   Google Scholar
• Ng AY, Russell SJ. Algorithms for inverse reinforcement learning. In: International Conference on Machine Learning. 2000. p. 663–670.
   Google Scholar
• Ho J, Ermon S. Generative adversarial imitation learning. arXiv preprint arXiv:160603476. 2016.
   Google Scholar
• Edwards A, Sahni H, Schroecker Y, et al. Imitating latent policies from observation. In: International Conference on Machine Learning. PMLR; 2019. p. 1755–1763.
   Google Scholar
• Torabi F, Warnell G, Stone P. Generative adversarial imitation from observation. arXiv preprint arXiv:180706158. 2018.
   Google Scholar
• Singh S, Silakari S. An ensemble approach for feature selection of cyber attack dataset. arXiv preprint arXiv:09121014. 2009.
   Google Scholar
• Peddabachigari S, Abraham A, Grosan C, et al. Modeling intrusion detection system using hybrid intelligent systems. J Netw Comput Appl. 2007;30(1):114–132. Network and Information Security: A Computational Intelligence Approach; Available from: http://www.sciencedirect.com/science/article/pii/S1084804505000445
   Web of Science ®Google Scholar
• Ruff L, Vandermeulen RA, Franks BJ, et al. Rethinking assumptions in deep anomaly detection. arXiv preprint arXiv:200600339. 2020.
   Google Scholar
• Sun Y, Wong AK, Kamel MS. Classification of imbalanced data: a review. Int J Pattern Recogn Art Intell. 2009;23(04):687–719.
   Web of Science ®Google Scholar
• Münz G, Li S, Carle G. Traffic anomaly detection using k-means clustering. In: GI/ITG Workshop MMBnet. 2007. p. 13–14.
   Google Scholar
• Amer M, Goldstein M, Abdennadher S. Enhancing one-class support vector machines for unsupervised anomaly detection. In: ACM SIGKDD Workshop on Outlier Detection and Description; 2013. p. 8–15.
   Google Scholar
• Borghesi A, Bartolini A, Lombardi M, et al. Anomaly detection using autoencoders in high performance computing systems. In: AAAI Conference on Artificial Intelligence. Vol. 33; 2019. p. 9428–9433.
   Google Scholar
• An J, Cho S. Variational autoencoder based anomaly detection using reconstruction probability. Spec Lectur on IE. 2015;2(1):1–18.
   Google Scholar
• Liu W, Li R, Zheng M, et al. Towards visually explaining variational autoencoders. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. p. 8642–8651.
   Google Scholar
• Pol AA, Berger V, Germain C, et al. Anomaly detection with conditional variational autoencoders. In: IEEE International Conference on Machine Learning And Applications; IEEE; 2019. p. 1651–1657.
   Google Scholar
• Goldstein M, Seiichi U. A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data. PLoS ONE. 2016 04;11(4):1–31. Available from: https://doi.org/10.1371/journal.pone.0152173
   Web of Science ®Google Scholar
• Nalisnick E, Matsukawa A, Teh YW, et al. Do deep generative models know what they don't know? arXiv preprint arXiv:181009136. 2018.
   Google Scholar
• Kirichenko P, Izmailov P, Wilson AG. Why normalizing flows fail to detect out-of-distribution data. arXiv preprint arXiv:200608545. 2020.
   Google Scholar
• Nalisnick E, Matsukawa A, Teh YW, et al. Detecting out-of-distribution inputs to deep generative models using typicality. arXiv preprint arXiv:190602994. 2019.
   Google Scholar
• Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science. 2006;313:504–507.
   PubMed Web of Science ®Google Scholar
• Kobyzev I, Prince S, Brubaker M. Normalizing flows: an introduction and review of current methods. IEEE Trans Pattern Anal Mach Intell. 2020.
   PubMed Web of Science ®Google Scholar
• Dinh L, Sohl-Dickstein J, Bengio S. Density estimation using real nvp. arXiv preprint arXiv:160508803. 2016.
   Google Scholar
• Reynolds DA. Gaussian mixture models. Encycl Biomet. 2009;741:659–663.
   Google Scholar
• Bhalodia R, Lee I, Elhabian S. dpvaes: fixing sample generation for regularized vaes. arXiv preprint arXiv:191110506. 2019.
   Google Scholar
• Brockman G, Cheung V, Pettersson L, et al. Openai gym. arXiv preprint arXiv:160601540. 2016.
   Google Scholar
• Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms. arXiv preprint arXiv:170706347. 2017.
   Google Scholar
• Kingma DP, Ba JAdam. A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014.
   Google Scholar
• Kotani A, Tellex S. Teaching robots to draw. In: International Conference on Robotics and Automation. 2019. p. 4797–4803.
   Google Scholar
• Eysenbach B, Gu S, Ibarz J, et al. Leave no trace: learning to reset for safe and autonomous reinforcement learning. arXiv preprint arXiv:171106782. 2017.
   Google Scholar
• Thananjeyan B, Balakrishna A, Nair S, et al. Recovery rl: safe reinforcement learning with learned recovery zones. arXiv preprint arXiv:201015920. 2020.
   Google Scholar
• Kirkpatrick J, Pascanu R, Rabinowitz N, et al. Overcoming catastrophic forgetting in neural networks. Proc Nat Acad Sci. 2017;114(13):3521–3526.
   PubMed Web of Science ®Google Scholar
• Codevilla F, Miiller M, López A, et al. End-to-end driving via conditional imitation learning. In: IEEE International Conference on Robotics and Automation. IEEE; 2018. p. 1–9.
   Google Scholar
• Liang X, Wang T, Yang L, et al. Cirl: controllable imitative reinforcement learning for vision-based self-driving. In: European Conference on Computer Vision; 2018. p. 584–599.
   Google Scholar