Deep reinforcement learning in production systems: a systematic literature review

Figures & data

Figure 1. Agent–environment interaction; Sutton and Barto (2017).

Figure 2. Eight step approach to conduct an SLR.

Table 1. Taxonomy framework of the SLR.

Characteristic	Categories
(1) Focus	Research outcomes	Research methods	Theories	Applications
(2) Goal	Integration	Criticism		Central issues
(3) Perspective	Neutral representation		Espousal of position
(4) Coverage	Exhaustive	Exhaustive and selective	Representative	Central/pivotal
(5) Organisation	Historical	Conceptual		Methodological
(6) Audience	Specialised scholars	General scholars	Practitioners/politicians	General public

Table 2. Defined keywords for the SLR.

Algorithmic keywords		General keywords		Specific keywords
$\left[\begin{array}{ll}\mathrm{D}\mathrm{e}\mathrm{e}\mathrm{p}\mathrm{R}{\mathrm{L}}^{\mathrm{a}}\mathrm{O}\mathrm{R}& \\ & \mathrm{A}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{O}\mathrm{R}\\ \mathrm{R}{\mathrm{L}}^{\mathrm{a}}\mathrm{A}\mathrm{N}\mathrm{D}& \mathrm{D}\mathrm{e}\mathrm{e}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{O}\mathrm{R}\\ & \mathrm{M}\mathrm{a}\mathrm{c}\mathrm{h}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{g}\end{array}\right]$	AND	$\left[\begin{array}{l}\mathrm{A}\mathrm{s}\mathrm{s}\mathrm{e}\mathrm{m}\mathrm{b}\mathrm{l}\mathrm{y}\mathrm{O}\mathrm{R}\\ \mathrm{A}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{O}\mathrm{R}\\ \mathrm{I}\mathrm{n}\mathrm{d}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{y}\mathrm{O}\mathrm{R}\\ \mathrm{M}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{f}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{O}\mathrm{R}\\ \mathrm{P}\mathrm{r}\mathrm{o}\mathrm{d}\mathrm{u}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\end{array}\right]$	OR	$\left[\begin{array}{l}\mathrm{L}\mathrm{o}\mathrm{g}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{s}\mathrm{O}\mathrm{R}\\ \mathrm{M}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{O}\mathrm{R}\\ \mathrm{P}\mathrm{r}\mathrm{o}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{O}\mathrm{R}\\ \mathrm{Q}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{O}\mathrm{R}\\ \mathrm{R}\mathrm{e}\mathrm{a}\mathrm{l}-\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{O}\mathrm{R}\\ \mathrm{T}\mathrm{o}\mathrm{o}\mathrm{l}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\end{array}\right]$

^aRL, Reinforcement learning.

Figure 3. Conducted review process.

Figure 4. Analysis of yearly deep RL publications, 2021 includes Jan./Feb. (a) Yearly and accumulated deep RL publications (b) Yearly deep and non-deep RL publications.

Figure 5. Number of publications per outlet; 2010–2021.

Figure 6. Number of publications allocated to the production disciplines.

Table 3. Summary of deep RL applications in process control.

	Application	Algorithm	Superiority	Source
1	Batch process	DDPG	Superior	Xu, Xie, and Shi (2020)
2	Brine injection process	Actor-critic	Superior	Andersen et al. (2019)
3	Liquid moulding process	DQN	Superior	Szarski and Chauhan (2021)
4	Chemical microdroplet reactions	Actor-critic	Superior	Zhou, Li, and Zare (2017)
5	Colour fading	Actor-critic	Superior	He et al. (2020)
6	Continuously stirred tank reactor	Actor-critic	–	Pandian and Noel (2018)
7	Continuously stirred tank reactor	DQN	Comparable	Powell, Machalek, and Quah (2020)
8	Continuously stirred tank reactor	Actor-critic	Comparable	Quah, Machalek, and Powell (2020)
9	Interacting tank liquid level control	Actor-critic	Superior	Noel and Pandian (2014)
10	Double dome draping	Actor-critic	–	Zimmerling, Poppe, and Kärger (2020)
11	General discrete-time processes	Actor-critic	–	Spielberg et al. (2019)
12	General discrete-time processes	Actor-critic	–	Spielberg, Gopaluni, and Loewen (2017)
13	Goethite iron-removal process	DDPG	–	Chen et al. (2020)
14	Hematite iron ore processing	DQN/PG	–	Li et al. (2020)
15	Laser welding	DQN	–	Günther et al. (2016)
16	Laser welding	Value based	–	Jin and Hongming Gao (2019)
17	Laser welding	Value based	–	Masinelli et al. (2020)
18	Laser welding	Actor-critic	Superior	Zou and Lan (2020)
19	Metal sheet deep drawing	ANN-PSO/PPO	–	Dornheim, Link, and Gumbsch (2020)
20	Non-lin. semi-batch polymerisation	DQN/DPG/REI	Comparable	Ma et al. (2019)
21	One-stage mineral grinding	Actor-Critic	–	Lu et al. (2016)
22	Optical lens manufacturing	PPO	Superior	Guo et al. (2019)
23	Propylene oxide batch polymerisation	DQN	Superior	Yoo et al. (2021)
24	Rot. chemical mechanical polishing	DDPG	Superior	Yu and Guo (2020)
25	Single-cell flotation process	DDPG	Superior	Jiang et al. (2018)
26	Single-cell flotation process	DDPG	–	Jiang et al. (2019)
27	Single-circuit ball mill grinding	DRO	Superior	Guo, Wang, and Zhang (2019)
28	Tempered glass manufacturing	Actor-critic	–	Mazgualdi et al. (2021)
29	Well surveillance	Actor-critic	Superior	Tewari, Liu, and Papageorgiou (2020)

Table 4. Summary of deep RL applications in production scheduling, dispatching, and (intra-) logistics.

	Scheduling
	Application	Algorithm	Superiority	Source
30	Chemical scheduling	A2C	Superior	Hubbs et al. (2020)
31	Cloud manufacturing	DQN	Superior	Dong et al. (2020)
32	Cloud manufacturing	PG	Superior	Zhu et al. (2020)
33	Dynamic scheduling	DQN	–	Zhou, Zhang, and Horn (2020)
34	Dynamic scheduling	DQN	Superior	Hu et al. (2020)
35	Flow shop scheduling	Reinforce	Superior	Wu et al. (2020)
36	Job-shop scheduling	DDPG	Superior	Liu, Chang, and Tseng (2020)
37	Job-shop scheduling	PPO	Superior	Park et al. (2021)
38	Job-shop scheduling	DQN	–	Baer et al. (2020)
39	Job-shop scheduling	–	–	Baer et al. (2019)
40	Job-shop scheduling	DQN	Superior	Zhou et al. (2021)
41	Job-shop scheduling	DDDQN	Superior	Han and Yang (2020)
42	Job-shop scheduling	(M)DQN	Superior	Lin et al. (2019)
43	Lot scheduling	PPO	Superior	Rummukainen and Nurminen (2019)
44	Mold scheduling	DQN	Superior	Lee, Cho, and Lee (2020)
45	Multichip production	DQN	Superior	Park et al. (2020)
46	Packaging line scheduling	DQN	Superior	Chen et al. (2019)
47	Paint job scheduling	Double DQN	Superior	Leng et al. (2020)
48	Parallel, re-entrant production	DQN	Comparable	Shi et al. (2020)
49	Rescheduling	DQN	Superior	Palombarini and Martinez (2018)
50	Rescheduling	DQN	Superior	Palombarini and Martínez (2019)
51	Single machine scheduling	DQN	Comparable	Xie, Zhang, and Rose (2019)
	Dispatching
	Application	Algorithm	Superiority	Source
52	General job-shop	DQN	Comparable	Marc-André and Fohlmeister (2020)
53	General job-shop	double DQN	Superior	Luo (2020)
54	General job-shop	DQN	Comparable	Heger and Voß (2020)
55	General job-shop	Reinforce	Superior	Zheng, Gupta, and Serita (2020)
56	Mining materials flow	PG	Superior	Kumar, Dimitrakopoulos, and Maulen (2020)
57	Wafer fabrication	DQN	Comparable	Waschneck et al. (2018a)
58	Wafer fabrication	TRPO	Comparable	Kuhnle et al. (2020)
59	Wafer fabrication	DQN	Comparable	Waschneck et al. (2018b)
60	Wafer fabrication	DQN	Superior	Stricker et al. (2018)
61	Wafer fabrication	DQN	Superior	Altenmüller et al. (2020)
62	WIP bounding	DQN	Superior	Tomé and Azevedo (2019)
	(Intra-) logistics
	Application	Algorithm	Superiority	Source
63	AGV scheduling	DQN	Superior	Feldkamp, Bergmann, and Strassburger (2020)
64	AGV scheduling	DQN	Superior	Hu et al. (2020)
65	AMR dispatching	TD3	Superior	Malus, Kozjek, and Vrabič (2020)
66	Item betching into trays	PPO	–	Hildebrand, Andersen, and Bøgh (2020)
67	QoS service composition model	duelingDQN	Superior	Liang et al. (2021)
68	Syringe filling process	doubleDQN	Superior	Xia et al. (2020)
69	Three-grid sortation system	DQN	–	Kim et al. (2020)

Table 5. Summary of deep RL applications in assembly and robotics.

	Assembly
	Application	Use-case	Algorithm	Source
70	Sequence planning	Building block model	DQN	Watanabe and Inada (2020)
71	Sequence planning	Lift desk assembly	As AlphaGoZero	Yu, Huang, and Chang (2020)
72	Sequence planning	Seven parts assembly process	DQN	Zhao et al. (2019)
73	High precision insertion	Circuit breaker housing	DQN	Li et al. (2019)
74	High precision insertion	Gear set assembly	Model-based	Luo et al. (2019)
75	High precision insertion	Peg-in-hole	DQN	Inoue et al. (2017)
76	High precision insertion	Peg-in-hole	DQN, DDPG	Li et al. (2019)
77	High precision insertion	Peg-in-hole	SAC	Zhao et al. (2020)
78	Insertion task	Peg-in-hole (contact-rich, deform.)	AC	Luo et al. (2018)
79	Insertion task	Double peg-in-hole	DDPG	Xu et al. (2019)
80	Insertion task	Double peg-in-hole (contact-rich)	DDPG	Hoppe et al. (2019)
81	Insertion task	Peg-in-hole	DDPG	Wang et al. (2020)
82	Insertion task	Peg-in-hole	DDPG	Lämmle et al. (2020)
83	Insertion task	Peg-in-hole (cuboid)	SAC	Beltran-Hernandez et al. (2020)
84	Insertion task	Peg-in-hole (square)	DDPG	Kim, Ahn, and Song (2020)
85	Insertion task	Ring-insertion	SAC	Beltran-Hernandez et al. (2020)
86	Plug insertion tasks	Model-E connector	SAC, TD3	Schoettler et al. (2020)
87	Shoe tongue assembly	Soft fabric shoe tongues	DQN	Tsai et al. (2020)
	Robotics
	Application	Use-case	Algorithm	Source
88	Intelligent gripping	Find optimal grasp position	PPO	Park et al. (2020)
89	Motion planning	Real-time CNC traj. smoothing	DQN, DDPG	Li et al. (2020)
90	Motion planning	Bézier curve trajectory smoothing	DDPG	Scheiderer, Thun, and Meisen (2019)
91	Visual control	Low-cost servo control	Sarsa, DQN	Miljković et al. (2013)

Table 6. Summary of deep RL applications in maintenance, energy management, and (process) design.

	Maintenance
	Application	Use-case	Algorithm	Source
92	Condition-based maintenance	Minimise cost rates in multiple stages	Dou.DQN	Zhang and Si (2020)
93	Machine fault diagnosis	Gear root crack analysis	DQN	Dai et al. (2020)
94	Machine fault diagnosis	Rolling bearing fault	DQN	Ding et al. (2019)
95	Oportunistic maintenance	Minimise prod./ maint. interference	PPO	Kuhnle, Jakubik, and Lanza (2019)
96	Real-time prev. maintenance	Minimise long-run costs in serial prod.	Dou.DQN	Huang, Chang, and Arinez (2020)
97	Selective maintenace	Maximise maint. mission success	AC	Liu, Chen, and Jiang (2020)
98	Self-diagnosis and self-repair	Optimise self-repair in prod. lines	DQN	Epureanu, Aydin Nassehi, and Koren (2020)
99	Sensor-driven maintenance	Calc. remaining useful (turbofan)	DQN	Skordilis and Moghaddass (2020)
	Energy management
	Application	Use-case	Algorithm	Source
100	Energy system balancing	Tank level scheduling	DQN	Zhang et al. (2020)
101	Multi-agent energy optimisation	CPS energy coordination	AC	Bakakeu et al. (2020)
102	Network resource management	Energy efficient RF/VLC network	DQN	Yang et al. (2020)
103	PCB order acceptance	Real-time order acceptance decisions	DQN	Leng et al. (2021)
104	Production-energy schedule opt.	Single machine optimisation	DQN	Bakakeu et al. (2018)
105	Production-energy schedule opt.	Lithium-ion battery assembly	DDPG	Lu et al. (2020)
106	Production-energy schedule opt.	Steel powder manufacturing process	AC	Huang et al. (2019)
107	Sustainable joint energy control	Two machine, one buffer system	DQN	Hu et al. (2019)
108	Clamping position optimisation	Milling machine	SAC	Samsonov et al. (2020)
109	Computer-aided process planning	Constrained machining	AC	Wu et al. (2021)
110	Integrated circuit design	Global IC routing	DQN	Liao et al. (2020)
111	Notch filter design	Industrial servo systems	DDPG	Oh et al. (2020)
112	Rectangular item placement	2D Strip Packing	DQN	Zhu and Xiang Dong Li (2020)
113	SaaS remote training	Heavy plate rolling	SAC	Scheiderer et al. (2020)
114	Tool path design	Geometric impeller optimisation	DDPG	Zhou et al. (2020)

Table 7. Summary of deep RL applications in quality control and further applications.

	Quality control
	Application	Use-case	Algorithm	Source
115	In-situ quality moniitoring	Subsurface dynamics analysis	sim. AlphaGO	Wasmer et al. (2019)
116	Quality inspection	Path teaching and adaption	AC	Brito et al. (2020)
	Further applications
	Application	Use-case	Algorithm	Source
117	Novel PLC learning/acting arch.	Real-time framework	–	Schmidt, Schellroth, and Riedel (2020)
118	Select opt. demand forecast model	Semiconductor components	DQN	Chien, Lin, and Lin (2020)
119	Multi-task policy generalisation	Mfg. system with various tasks	DQN	Wang et al. (2019)
120	Investigation of malicious behaviours	Function-/performance attacks	DQN	Liu et al. (2021)

Table 8. Summary of the key findings from the review analysis.

		Most frequent	Most frequent	Simulation-only	Superiority
Production domain	#Publications	algorithm	neural netw.	share	(#benchmarks)
Process control	29	AC	FFNN	86%	81% (16)
Scheduling	22	DQN	FFNN	100%	89% (19)
Dispatching	11	DQN	FFNN	100%	55% (11)
(Intra-) Logistics	7	DQN	FFNN	86%	100% (5)
Assembly	18	DQN/DDPG	FFNN	17%	100% (4)
Robotics	4	DQN/DDPG	FFNN	75%	– (0)
Maintenance	8	DQN	FFNN	75%	100% (3)
Energy Management	8	DQN	FFNN	100%	100% (4)
(Process) Design	7	(S)AC	FFNN	71%	100% (4)
Quality Control	2	AC/AlphaGo	FFNN	0%	– (0)
Others	4	DQN	FFNN	75%	100% (2)

Figure 7. Quantitative analysis of applied algorithms and testing environments.

Sutton, Richard S., and Andrew G. Barto. 2017. Reinforcement Learning: An Introduction. 2nd ed. Adaptive computation and machine learning series. Cambridge: The MIT Press.

 Google Scholar

Xu, Xinghai, Huimin Xie, and Jia Shi. 2020. “Iterative Learning Control (ILC) Guided Reinforcement Learning Control (RLC) Scheme for Batch Processes.” In 2020 IEEE 9th Data Driven Control and Learning Systems Conference (DDCLS), Liuzhou, Juni 19–21.

 Google Scholar

Andersen, Rasmus E., Steffen Madsen, Alexander B. K. Barlo, Sebastian B. Johansen, Morten Nør, Rasmus S. Andersen, and Simon Bøgh. 2019. “Self-Learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection.” Procedia Manufacturing 38: 171–177. doi:https://doi.org/10.1016/j.promfg.2020.01.023.

 Google Scholar

Szarski, Martin, and Sunita Chauhan. 2021. “Composite Temperature Profile and Tooling Optimization Via Deep Reinforcement Learning.” Composites Part A: Applied Science and Manufacturing 142: 106–235. doi:https://doi.org/10.1016/j.compositesa.2020.106235.

 Web of Science ®Google Scholar

Zhou, Zhenpeng, Xiaocheng Li, and Richard N. Zare. 2017. “Optimizing Chemical Reactions with Deep Reinforcement Learning.” ACS Central Science 3 (12): 1337–1344. doi:https://doi.org/10.1021/acscentsci.7b00492.

 PubMed Web of Science ®Google Scholar

He, Zhenglei, Kim-Phuc Tran, Sebastien Thomassey, Xianyi Zeng, Jie Xu, and Changhai Yi. 2020. “A Deep Reinforcement Learning Based Multi-Criteria Decision Support System for Optimizing Textile Chemical Process.” Computers in Industry 125: 103373. doi:https://doi.org/10.1016/j.compind.2020.103373.

 Web of Science ®Google Scholar

Pandian, B. Jaganatha, and Mathew M. Noel. 2018. “Tracking Control of a Continuous Stirred Tank Reactor Using Direct and Tuned Reinforcement Learning Based Controllers.” Chemical Product and Process Modeling 13 (3): 1–10. doi:https://doi.org/10.1515/cppm-2017-0040.

 Web of Science ®Google Scholar

Powell, By Kody M., Derek Machalek, and Titus Quah. 2020. “Real-Time Optimization Using Reinforcement Learning.” Computers & Chemical Engineering 143: 107077. doi:https://doi.org/10.1016/j.compchemeng.2020.107077.

 Web of Science ®Google Scholar

Quah, Titus, Derek Machalek, and Kody M. Powell. 2020. “Comparing Reinforcement Learning Methods for Real-Time Optimization of a Chemical Process.” Processes 8 (11): 1497. doi:https://doi.org/10.3390/pr8111497.

 Web of Science ®Google Scholar

Noel, Mathew Mithra, and B. Jaganatha Pandian. 2014. “Control of a Nonlinear Liquid Level System Using a New Artificial Neural Network Based Reinforcement Learning Approach.” Applied Soft Computing 23: 444–451. doi:https://doi.org/10.1016/j.asoc.2014.06.037.

 Web of Science ®Google Scholar

Zimmerling, Clemens, Christian Poppe, and Luise Kärger. 2020. “Estimating Optimum Process Parameters in Textile Draping of Variable Part Geometries – A Reinforcement Learning Approach.” Procedia Manufacturing 47: 847–854. doi:https://doi.org/10.1016/j.promfg.2020.04.263.

 Google Scholar

Spielberg, Steven, Aditya Tulsyan, Nathan P. Lawrence, Philip D. Loewen, and R. Bhushan Gopaluni. 2019. “Toward Self-Driving Processes: A Deep Reinforcement Learning Approach to Control.” AIChE Journal 65 (10): 16–689. doi:https://doi.org/10.1002/aic.16689.

 Web of Science ®Google Scholar

Spielberg, S., R. B. Gopaluni, and P. D. Loewen. 2017. “Deep Reinforcement Learning Approaches for Process Control.” In 2017 6th International Symposium on Advanced Control of Industrial Processes, Taipei, Taiwan, May 28–31.

 Google Scholar

Chen, Ning, Shuhan Luo, Jiayang Dai, Biao Luo, and Weihua Gui. 2020. “Optimal Control of Iron-Removal Systems Based on Off-Policy Reinforcement Learning.” IEEE Access 8: 149730–149740. doi:https://doi.org/10.1109/ACCESS.2020.3015801.

 Web of Science ®Google Scholar

Li, Jinna, Jinliang Ding, Tianyou Chai, and Frank L. Lewis. 2020. “Nonzero-Sum Game Reinforcement Learning for Performance Optimization in Large-Scale Industrial Processes.” IEEE Transactions on Cybernetics 50 (9): 4132–4145. doi:https://doi.org/10.1109/TCYB.2019.2950262.

 PubMed Web of Science ®Google Scholar

Günther, Johannes, Patrick M. Pilarski, Gerhard Helfrich, Hao Shen, and Klaus Diepold. 2016. “Intelligent Laser Welding Through Representation, Prediction, and Control Learning: An Architecture with Deep Neural Networks and Reinforcement Learning.” Mechatronics 34: 1–11. doi:https://doi.org/10.1016/j.mechatronics.2015.09.004.

 Web of Science ®Google Scholar

Jin, Zeshi, and Haichao Li Hongming Gao. 2019. “An Intelligent Weld Control Strategy Based on Reinforcement Learning Approach.” The International Journal of Advanced Manufacturing Technology100 (9-12): 2163–2175. doi:https://doi.org/10.1007/s00170-018-2864-2.

 Web of Science ®Google Scholar

Masinelli, Giulio, Tri Le-Quang, Silvio Zanoli, Kilian Wasmer, and Sergey A. Shevchik. 2020. “Adaptive Laser Welding Control: A Reinforcement Learning Approach.” IEEE Access 8: 103803–103814. doi:https://doi.org/10.1109/ACCESS.2020.2998052.

 Web of Science ®Google Scholar

Zou, Yanbiao, and Rui Lan. 2020. “An End-to-End Calibration Method for Welding Robot Laser Vision Systems With Deep Reinforcement Learning.” IEEE Transactions on Instrumentation and Measurement69 (7): 4270–4280. doi:https://doi.org/10.1109/TIM.2019.2942533.

 Web of Science ®Google Scholar

Dornheim, Johannes, Norbert Link, and Peter Gumbsch. 2020. “Model-Free Adaptive Optimal Control of Episodic Fixed-horizon Manufacturing Processes Using Reinforcement Learning.” International Journal of Control, Automation and Systems 18 (6): 1593–1604. doi:https://doi.org/10.1007/s12555-019-0120-7.

 Web of Science ®Google Scholar

Ma, Yan, Wenbo Zhu, Michael G. Benton, and José Romagnoli. 2019. “Continuous Control of A Polymerization System with Deep Reinforcement Learning.” Journal of Process Control 75: 40–47. doi:https://doi.org/10.1016/j.jprocont.2018.11.004.

 Web of Science ®Google Scholar

Lu, Xinglong, Bahare Kiumarsi, Tianyou Chai, and Frank L. Lewis. 2016. “Data-Driven Optimal Control of Operational Indices for A Class of Industrial Processes.” IET Control Theory & Applications 10 (12): 1348–1356. doi:https://doi.org/10.1049/iet-cta.2015.0798.

 Web of Science ®Google Scholar

Guo, Fei, Xiaowei Zhou, Jiahuan Liu, Yun Zhang, Dequn Li, and Huamin Zhou. 2019. “A Reinforcement Learning Decision Model for Online Process Parameters Optimization From Offline Data in Injection Molding.” Applied Soft Computing 85: 105828. doi:https://doi.org/10.1016/j.asoc.2019.105828.

 Web of Science ®Google Scholar

Yoo, Haeun, Boeun Kim, Jong Woo Kim, and Jay H. Lee. 2021. “Reinforcement Learning Based Optimal Control of Batch Processes Using Monte-Carlo Deep Deterministic Policy Gradient with Phase Segmentation.” Computers & Chemical Engineering 144: 107–133. doi:https://doi.org/10.1016/j.compchemeng.2020.107133

 Web of Science ®Google Scholar

Yu, Jianbo, and Peng Guo. 2020. “Run-to-Run Control of Chemical Mechanical Polishing Process Based on Deep Reinforcement Learning.” IEEE Transactions on Semiconductor Manufacturing 33 (3): 454–465. doi:https://doi.org/10.1109/TSM.2020.3002896.

 Web of Science ®Google Scholar

Jiang, Yi, Jialu Fan, Tianyou Chai, Jinna Li, and Frank L. Lewis. 2018. “Data-Driven Flotation Industrial Process Operational Optimal Control Based on Reinforcement Learning.” IEEE Transactions on Industrial Informatics 14 (5): 1974–1989. doi:https://doi.org/10.1109/TII.2017.2761852.

 Web of Science ®Google Scholar

Jiang, Yi, Jialu Fan, Tianyou Chai, and Frank L. Lewis. 2019. “Dual-Rate Operational Optimal Control for Flotation Industrial Process With Unknown Operational Model.” IEEE Transactions on Industrial Electronics 66 (6): 4587–4599. doi:https://doi.org/10.1109/TIE.2018.2856198.

 Web of Science ®Google Scholar

Guo, Li, Huan Wang, and Jun Zhang. 2019, August 10–12. “Data-Driven Grinding Control Using Reinforcement Learning.” 2019 IEEE 21st International Conference on High Performance Computing and Communications, Zhangjiajie, China. IEEE.

 Google Scholar

Mazgualdi, Choumicha El, Tawfik Masrour, Ibtissam El Hassani, and Abdelmoula Khdoudi. 2021. “A Deep Reinforcement Learning (DRL) Decision Model for Heating Process Parameters Identification in Automotive Glass Manufacturing.” In Artificial Intelligence and Industrial Applications, Vol. 1193, 77–87. Cham: Springer International Publishing.

 Google Scholar

Tewari, Ashutosh, Kuang-Hung Liu, and Dimitri Papageorgiou. 2020. “Information-Theoretic Sensor Planning for Large-Scale Production Surveillance Via Deep Reinforcement Learning.” Computers & Chemical Engineering 141: 106–988. doi:https://doi.org/10.1016/j.compchemeng.2020.106988.

 Web of Science ®Google Scholar

Hubbs, Christian D., Can Li, Nikolaos V. Sahinidis, Ignacio E. Grossmann, and John M. Wassick. 2020. “A Deep Reinforcement Learning Approach for Chemical Production Scheduling.” Computers & Chemical Engineering 141: 106982. doi.https://doi.org/10.1016/j.compchemeng.2020.106982.

 Web of Science ®Google Scholar

Dong, Tingting, Fei Xue, Chuangbai Xiao, and Juntao Li. 2020. “Task Scheduling Based on Deep Reinforcement Learning in a Cloud Manufacturing Environment.” Concurrency and Computation: Practice and Experience 32 (11): e5654. doi:https://doi.org/10.1002/cpe.5654.

 Web of Science ®Google Scholar

Zhu, Huayu, Mengrong Li, Yong Tang, and Yanfei Sun. 2020. “A Deep-Reinforcement-Learning-Based Optimization Approach for Real-Time Scheduling in Cloud Manufacturing.” IEEE Access 8: 9987–9997. doi:https://doi.org/10.1109/ACCESS.2020.2964955.

 Web of Science ®Google Scholar

Zhou, Longfei, Lin Zhang, and Berthold K. P. Horn. 2020. “Deep Reinforcement Learning-Based Dynamic Scheduling in Smart Manufacturing.” Procedia CIRP 93: 383–388. doi:https://doi.org/10.1016/j.procir.2020.05.163.

 Google Scholar

Hu, Liang, Zhenyu Liu, Weifei Hu, Yueyang Wang, Jianrong Tan, and Fei Wu. 2020. “Petri-Net-Based Dynamic Scheduling of Flexible Manufacturing System Via Deep Reinforcement Learning with Graph Convolutional Network.” Journal of Manufacturing Systems 55: 1–14. doi:https://doi.org/10.1016/j.jmsy.2020.02.004.

 Web of Science ®Google Scholar

Wu, Chen-Xin, Min-Hui Liao, Mumtaz Karatas, Sheng-Yong Chen, and Yu-Jun Zheng. 2020. “Real-Time Neural Network Scheduling of Emergency Medical Mask Production During COVID-19.” Applied Soft Computing 97: 106–790. doi:https://doi.org/10.1016/j.asoc.2020.106790.

 Web of Science ®Google Scholar

Liu, Chien-Liang, Chuan-Chin Chang, and Chun-Jan Tseng. 2020. “Actor-Critic Deep Reinforcement Learning for Solving Job Shop Scheduling Problems.” IEEE Access8: 71752–71762. doi:https://doi.org/10.1109/ACCESS.2020.2987820.

 Web of Science ®Google Scholar

Park, Junyoung, Jaehyeong Chun, Sang Hun Kim, Youngkook Kim, and Jinkyoo Park. 2021. “Learning to Schedule Job-Shop Problems: Representation and Policy Learning Using Graph Neural Network and Reinforcement Learning.” International Journal of Production Research 59 (11): 3360–3377. doi:https://doi.org/10.1080/00207543.2020.1870013.

 Web of Science ®Google Scholar

Baer, Schirin, Danielle Turner, Punit Mohanty, Vladimir Samsonov, Romuald Bakakeu, and Tobias Meisen. 2020. “Multi Agent Deep Q-Network Approach for Online Job Shop Scheduling in Flexible Manufacturing.” 2020 International Conference on Manufacturing System and Multiple Machines, Tokyo, Japan, pp. 1–9.

 Google Scholar

Baer, Schirin, Jupiter Bakakeu, Richard Meyes, and Tobias Meisen. 2019, September. “Multi-Agent Reinforcement Learning for Job Shop Scheduling in Flexible Manufacturing Systems.” 2019 Second International Conference on Artificial Intelligence for Industries (AI4I). Laguna Hills, CA: IEEE, pp. 22–25.

 Google Scholar

Zhou, Tong, Dunbing Tang, Haihua Zhu, and Liping Wang. 2021. “Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory.” IEEE Access 9: 752–766. doi:https://doi.org/10.1109/ACCESS.2020.3046784.

 Web of Science ®Google Scholar

Han, Bao-An, and Jian-Jun Yang. 2020. “Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN.” IEEE Access 8: 186474–186495. doi:https://doi.org/10.1109/ACCESS.2020.3029868.

 Web of Science ®Google Scholar

Lin, Chun-Cheng, Der-Jiunn Deng, Yen-Ling Chih, and Hsin-Ting Chiu. 2019. “Smart Manufacturing Scheduling With Edge Computing Using Multiclass Deep Q Network.” IEEE Transactions on Industrial Informatics 15 (7): 4276–4284. doi:https://doi.org/10.1109/TII.2019.2908210.

 Web of Science ®Google Scholar

Rummukainen, Hannu, and Jukka K. Nurminen. 2019. “Practical Reinforcement Learning -Experiences in Lot Scheduling Application.” IFAC-PapersOnLine 52 (13): 1415–1420. doi:https://doi.org/10.1016/j.ifacol.2019.11.397.

 Google Scholar

Lee, Seunghoon, Yongju Cho, and Young Hoon Lee. 2020. “Injection Mold Production Sustainable Scheduling Using Deep Reinforcement Learning.” Sustainability 12 (20): 8718. doi:https://doi.org/10.3390/su12208718.

 Web of Science ®Google Scholar

Park, In-Beom, Jaeseok Huh, Joongkyun Kim, and Jonghun Park. 2020. “A Reinforcement Learning Approach to Robust Scheduling of Semiconductor Manufacturing Facilities.” IEEE Transactions on Automation Science and Engineering 17 (3): 1420–1431. doi:https://doi.org/10.1109/TASE.2019.2956762.

 Web of Science ®Google Scholar

Chen, Baotong, Jiafu Wan, Lan Yanting, Muhammad Imran, Di Li, and Nadra Guizani. 2019. “Improving Cognitive Ability of Edge Intelligent IIoT Through Machine Learning.” IEEE Network 33 (5): 61–67. doi:https://doi.org/10.1109/MNET.001.1800505.

 Web of Science ®Google Scholar

Leng, Jinling, Chun Jin, Alexander Vogl, and Huiyu Liu. 2020. “Deep Reinforcement Learning for A Color-Batching Resequencing Problem.” Journal of Manufacturing Systems 56: 175–187. doi:https://doi.org/10.1016/j.jmsy.2020.06.001.

 Web of Science ®Google Scholar

Shi, Daming, Wenhui Fan, Yingying Xiao, Tingyu Lin, and Chi Xing. 2020. “Intelligent Scheduling of Discrete Automated Production Line Via Deep Reinforcement Learning.” International Journal of Production Research 58 (11): 3362–3380. doi:https://doi.org/10.1080/00207543.2020.1717008.

 Web of Science ®Google Scholar

Palombarini, Jorge A., and Ernesto C. Martinez. 2018. “Automatic Generation of Rescheduling Knowledge in Socio-Technical Manufacturing Systems using Deep Reinforcement Learning.” In 2018 IEEE Biennial Congress of Argentina (ARGENCON), San Miguel de Tucumán, Argentina, June 6–8.

 Google Scholar

Palombarini, Jorge A., and Ernesto C. Martínez. 2019. “Closed-loop Rescheduling Using Deep Reinforcement Learning.” IFAC-PapersOnLine 52 (1): 231–236. doi:https://doi.org/10.1016/j.ifacol.2019.06.067.

 Google Scholar

Xie, Shufang, Tao Zhang, and Oliver Rose. 2019. “Online Single Machine Scheduling Based on Simulation and Reinforcement Learning.” In 18. ASIM Fachtagung Simulation in Production und Logistik, Chemnitz, September 19–20.

 Google Scholar

Marc-André, Dittrich, and Silas Fohlmeister. 2020. “Cooperative Multi-Agent System for Production Control Using Reinforcement Learning.” CIRP Annals 69 (1): 389–392. doi:https://doi.org/10.1016/j.cirp.2020.04.005.

 Web of Science ®Google Scholar

Luo, Shu. 2020. “Dynamic Scheduling for Flexible Job Shop with New Job Insertions by Deep Reinforcement Learning.” Applied Soft Computing 91: 106208. doi:https://doi.org/10.1016/j.asoc.2020.106208.

 Web of Science ®Google Scholar

Heger, Jens, and Thomas Voß. 2020. “Dynamically Changing Sequencing Rules Wirth Reinforcement Learning in a Job Shop System with Stochastic Influences.” Proceedings of the 2020 Winter Simulation Conference 1608–1618. Orlando, FL: IEEE.

 Google Scholar

Zheng, Shuai, Chetan Gupta, and Susumu Serita. 2020. “Manufacturing Dispatching Using Reinforcement and Transfer Learning.” Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases 655–671. doi:https://doi.org/10.1007/978-3-030-46133-1_39.

 Google Scholar

Kumar, Ashish, Roussos Dimitrakopoulos, and Marco Maulen. 2020. “Adaptive Self-Learning Mechanisms for Updating Short-Term Production Decisions in An Industrial Mining Complex.” Journal of Intelligent Manufacturing 31 (7): 1795–1811. doi:https://doi.org/10.1007/s10845-020-01562-5.

 Web of Science ®Google Scholar

Waschneck, Bernd, Andre Reichstaller, Lenz Belzner, Thomas Altenmuller, Thomas Bauernhansl, Alexander Knapp, and Andreas Kyk. 2018a. “Deep Reinforcement Learning for Semiconductor Production Scheduling.” In 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs, NY, USA, April 30 – May 3.

 Google Scholar

Kuhnle, Andreas, Jan-Philipp Kaiser, Felix Theiß, Nicole Stricker, and Gisela Lanza. 2020. “Designing An Adaptive Production Control System Using Reinforcement Learning.” Journal of Intelligent Manufacturing 32: 855–876. doi:https://doi.org/10.1007/s10845-020-01612-y.

 Web of Science ®Google Scholar

Waschneck, Bernd, André Reichstaller, Lenz Belzner, Thomas Altenmüller, Thomas Bauernhansl, Alexander Knapp, and Andreas Kyek. 2018b. “Optimization of Global Production Scheduling with Deep Reinforcement Learning.” Procedia CIRP 72: 1264–1269. doi:https://doi.org/10.1016/j.procir.2018.03.212.

 Google Scholar

Stricker, Nicole, Andreas Kuhnle, Roland Sturm, and Simon Friess. 2018. “Reinforcement Learning for Adaptive Order Dispatching in The Semiconductor Industry.” CIRP Annals 67 (1): 511–514. doi:https://doi.org/10.1016/j.cirp.2018.04.041.

 Web of Science ®Google Scholar

Altenmüller, Thomas, Tillmann Stüker, Bernd Waschneck, Andreas Kuhnle, and Gisela Lanza. 2020. “Reinforcement Learning for An Intelligent and Autonomous Production Control of Complex Job-Shops Under Time Constraints.” Production Engineering 14 (3): 319–328. doi:https://doi.org/10.1007/s11740-020-00967-8.

 Google Scholar

Tomé, Silva, and Américo Azevedo. 2019. “Production Flow Control Through the Use of Reinforcement Learning.” Procedia Manufacturing 38: 194–202. doi:https://doi.org/10.1016/j.promfg.2020.01.026.

 Google Scholar

Feldkamp, Niclas, Soeren Bergmann, and Steffen Strassburger. 2020. “Simulation-Based Deep Reinforcement Learning for Modular Production Systems.” Proceedings of the 2020 Winter Simulation Conference, 1596–1607. IEEE.

 Google Scholar

Hu, Hao, Xiaoliang Jia, Qixuan He, Shifeng Fu, and Kuo Liu. 2020. “Deep Reinforcement Learning Based AGVs Real-Time Scheduling with Mixed Rule for Flexible Shop Floor in Industry 4.0.” Computers & Industrial Engineering 149: 106749. doi.https://doi.org/10.1016/j.cie.2020.106749.

 Web of Science ®Google Scholar

Malus, Andreja, Dominik Kozjek, and Rok Vrabič. 2020. “Real-Time Order Dispatching for A Fleet of Autonomous Mobile Robots Using Multi-Agent Reinforcement Learning.” CIRP Annals 69 (1): 397–400. doi:https://doi.org/10.1016/j.cirp.2020.04.001.

 Web of Science ®Google Scholar

Hildebrand, Max, Rasmus S. Andersen, and Simon Bøgh. 2020. “Deep Reinforcement Learning for Robot Batching Optimization and Flow Control.” Procedia Manufacturing 51: 1462–1468. doi:https://doi.org/10.1016/j.promfg.2020.10.203.

 Google Scholar

Liang, Huagang, Xiaoqian Wen, Yongkui Liu, Haifeng Zhang, Lin Zhang, and Lihui Wang. 2021. “Logistics-Involved QoS-Aware Service Composition in Cloud Manufacturing With Deep Reinforcement Learning.” Robotics and Computer-Integrated Manufacturing 67: 101991. doi:https://doi.org/10.1016/j.rcim.2020.101991.

 Web of Science ®Google Scholar

Xia, Kaishu, Christopher Sacco, Max Kirkpatrick, Clint Saidy, Lam Nguyen, Anil Kircaliali, and Ramy Harik. 2020. “A Digital Twin to Train Deep Reinforcement Learning Agent for Smart Manufacturing Plants: Environment, Interfaces and Intelligence.” Journal of Manufacturing Systems 58: 210–230. doi:https://doi.org/10.1016/j.jmsy.2020.06.012.

 Web of Science ®Google Scholar

Kim, Ju-Bong, Ho-Bin Choi, Gyu-Young Hwang, Kwihoon Kim, Yong-Geun Hong, and Youn-Hee Han. 2020. “Sortation Control Using Multi-Agent Deep Reinforcement Learning in N-Grid Sortation System.” Sensors 20 (12): 3401. doi.https://doi.org/10.3390/s20123401.

 PubMed Web of Science ®Google Scholar

Watanabe, Keijiro, and Shuhei Inada. 2020. “Search Algorithm of the Assembly Sequence of Products by Using Past Learning Results.” International Journal of Production Economics 226: 107–615. doi:https://doi.org/10.1016/j.ijpe.2020.107615.

 Web of Science ®Google Scholar

Yu, Tian, Jing Huang, and Qing Chang. 2020. “Mastering the Working Sequence in Human–Robot Collaborative Assembly Based on Reinforcement Learning.” IEEE Access 8: 163868–163877. doi:https://doi.org/10.1109/ACCESS.2020.3021904.

 Web of Science ®Google Scholar

Zhao, Minghui, Xian Guo, Xuebo Zhang, Yongchun Fang, and Yongsheng Ou. 2019. “ASPW-DRL: Assembly Sequence Planning for Workpieces Via a Deep Reinforcement Learning Approach.” Assembly Automation 40 (1): 65–75. doi:https://doi.org/10.1108/AA-11-2018-0211.

 Web of Science ®Google Scholar

Li, Fengming, Qi Jiang, Sisi Zhang, Meng Wei, and Rui Song. 2019. “Robot Skill Acquisition in Assembly Process Using Deep Reinforcement Learning.” Neurocomputing 345: 92–102. doi:https://doi.org/10.1016/j.neucom.2019.01.087.

 Web of Science ®Google Scholar

Luo, Jianlan, Eugen Solowjow, Chengtao Wen, Juan Aparicio Ojea, Alice M. Agogino, Aviv Tamar, and Pieter Abbeel. 2019. “Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assembly.” In 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, May 20–24.

 Google Scholar

Inoue, Tadanobu, Giovanni De Magistris, Asim Munawar, Tsuyoshi Yokoya, and Ryuki Tachibana. 2017. “Deep Reinforcement Learning for High Precision Assembly Tasks.” In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, September 24–28.

 Google Scholar

Li, Fengming, Qi Jiang, Wei Quan, Shibo Cai, Rui Song, and Yibin Li. 2019. “Manipulation Skill Acquisition for Robotic Assembly Based on Multi-Modal Information Description.” IEEE Access 8: 6282–6294. doi:https://doi.org/10.1109/ACCESS.2019.2934174.

 Web of Science ®Google Scholar

Zhao, Xin, Huan Zhao, Pengfei Chen, and Han Ding. 2020. “Model Accelerated Reinforcement Learning for High Precision Robotic Assembly.” International Journal of Intelligent Robotics and Applications4 (2): 202–216. doi:https://doi.org/10.1007/s41315-020-00138-z.

 Web of Science ®Google Scholar

Luo, Jianlan, Eugen Solowjow, Chengtao Wen, Juan Aparicio Ojea, and Alice M. Agogino. 2018. “Deep Reinforcement Learning for Robotic Assembly of Mixed Deformable and Rigid Objects.” In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 1–5.

 Google Scholar

Xu, Jing, Zhimin Hou, Wei Wang, Bohao Xu, Kuangen Zhang, and Ken Chen. 2019. “Feedback Deep Deterministic Policy Gradient With Fuzzy Reward for Robotic Multiple Peg-in-Hole Assembly Tasks.” IEEE Transactions on Industrial Informatics 15 (3): 1658–1667. doi:https://doi.org/10.1109/TII.2018.2868859.

 Web of Science ®Google Scholar

Hoppe, Sabrina, Zhongyu Lou, Daniel Hennes, and Marc Toussaint. 2019. “Planning Approximate Exploration Trajectories for Model-Free Reinforcement Learning in Contact-Rich Manipulation.” IEEE Robotics and Automation Letters 4 (4): 4042–4047. doi:https://doi.org/10.1109/LRA.2019.2928212.

 Web of Science ®Google Scholar

Wang, Fei, Xingqun Zhou, Jianhui Wang, Xing Zhang, Zhenquan He, and Bo Song. 2020. “Joining Force of Human Muscular Task Planning With Robot Robust and Delicate Manipulation for Programming by Demonstration.” IEEE/ASME Transactions on Mechatronics 25 (5): 2574–2584. doi:https://doi.org/10.1109/TMECH.2020.2997799.

 Web of Science ®Google Scholar

Lämmle, Arik, Thomas König, Mohamed El-Shamouty, and Marco F. Huber. 2020. “Skill-Based Programming of Force-controlled Assembly Tasks Using Deep Reinforcement Learning.” Procedia CIRP93: 1061–1066. doi:https://doi.org/10.1016/j.procir.2020.04.153.

 Google Scholar

Beltran-Hernandez, Cristian C., Damien Petit, Ixchel G. Ramirez-Alpizar, and Kensuke Harada. 2020. “Variable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep-Reinforcement-Learning Approach.” Applied Sciences 10 (19): 6923. doi:https://doi.org/10.3390/app10196923.

 Google Scholar

Kim, Young-Loul, Kuk-Hyun Ahn, and Jae-Bok Song. 2020. “Reinforcement Learning Based on Movement Primitives For Contact Tasks.” Robotics and Computer-Integrated Manufacturing 62: 101863. doi:https://doi.org/10.1016/j.rcim.2019.101863.

 Web of Science ®Google Scholar

Beltran-Hernandez, C. C., D. Petit, I. G. Ramirez-Alpizar, T. Nishi, S. Kikuchi, T. Matsubara, and K. Harada. 2020. “Learning Force Control for Contact-Rich Manipulation Tasks With Rigid Position-Controlled Robots.” IEEE Robotics and Automation Letters 5 (4): 5709–5716. doi:https://doi.org/10.1109/LRA.2020.3010739.

 Web of Science ®Google Scholar

Schoettler, Gerrit, Ashvin Nair, Jianlan Luo, Shikhar Bahl, Juan Aparicio Ojea, Eugen Solowjow, and Sergey Levine. 2020. “Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards.” In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, Nevada, USA, October 25-29. doi:https://doi.org/10.1109/IROS45743.2020.9341714.

 Google Scholar

Tsai, Yu-Ting, Chien-Hui Lee, Tao-Ying Liu, Tien-Jan Chang, Chun-Sheng Wang, S. J. Pawar, Pei-Hsing Huang, and Jin-H. Huang. 2020. “Utilization of A Reinforcement Learning Algorithm for the Accurate Alignment of a Robotic Arm in a Complete Soft Fabric Shoe Tongues Automation Process.” Journal of Manufacturing Systems 56: 501–513. doi:https://doi.org/10.1016/j.jmsy.2020.07.001

 Web of Science ®Google Scholar

Park, JoungMin, SangYoon Lee, JaeWoon Lee, and Jumyung Um. 2020. “GadgetArm–Automatic Grasp Generation and Manipulation of 4-DOF Robot Arm for Arbitrary Objects Through Reinforcement Learning.” Sensors 20 (21): 6183. doi:https://doi.org/10.3390/s20216183.

 PubMed Web of Science ®Google Scholar

Li, Bingran, Hui Zhang, Peiqing Ye, and Jinsong Wang. 2020. “Trajectory Smoothing Method Using Reinforcement Learning for Computer Numerical Control Machine Tools.” Robotics and Computer-Integrated Manufacturing 61: 101847. doi:https://doi.org/10.1016/j.rcim.2019.101847.

 Web of Science ®Google Scholar

Scheiderer, Christian, Timo Thun, and Tobias Meisen. 2019. “Bézier Curve Based Continuous and Smooth Motion Planning for Self-Learning Industrial Robots.” Procedia Manufacturing 38: 423–430. doi:https://doi.org/10.1016/j.promfg.2020.01.054.

 Google Scholar

Miljković, Zoran, Marko Mitić, Mihailo Lazarević, and Bojan Babić. 2013. “Neural Network Reinforcement Learning for Visual Control of Robot Manipulators.” Expert Systems with Applications40 (5): 1721–1736. doi:https://doi.org/10.1016/j.eswa.2012.09.010.

 Web of Science ®Google Scholar

Zhang, Nailong, and Wujun Si. 2020. “Deep Reinforcement Learning for Condition-Based Maintenance Planning of Multi-Component Systems Under Dependent Competing Risks.” Reliability Engineering & System Safety 203: 107–094. doi:https://doi.org/10.1016/j.ress.2020.107094.

 Web of Science ®Google Scholar

Dai, Wenxin, Zhenling Mo, Chong Luo, Jing Jiang, Heng Zhang, and Qiang Miao. 2020. “Fault Diagnosis of Rotating Machinery Based on Deep Reinforcement Learning and Reciprocal of Smoothness Index.” IEEE Sensors Journal 20 (15): 8307–8315. doi:https://doi.org/10.1109/JSEN.2020.2970747.

 Web of Science ®Google Scholar

Ding, Yu, Liang Ma, Jian Ma, Mingliang Suo, Laifa Tao, Yujie Cheng, and Chen Lu. 2019. “Intelligent Fault Diagnosis for Rotating Machinery Using Deep Q-Network Based Health State Classification: A Deep Reinforcement Learning Approach.” Advanced Engineering Informatics 42: 100977. doi:https://doi.org/10.1016/j.aei.2019.100977.

 Web of Science ®Google Scholar

Kuhnle, Andreas, Johannes Jakubik, and Gisela Lanza. 2019. “Reinforcement Learning for Opportunistic Maintenance Optimization.” Production Engineering 13 (1): 33–41. doi:https://doi.org/10.1007/s11740-018-0855-7.

 Google Scholar

Huang, Jing, Qing Chang, and Jorge Arinez. 2020. “Deep Reinforcement Learning Based Preventive Maintenance Policy for Serial Production Lines.” Expert Systems with Applications 160: 113701. doi:https://doi.org/10.1016/j.eswa.2020.113701.

 Web of Science ®Google Scholar

Liu, Yu, Yiming Chen, and Tao Jiang. 2020. “Dynamic Selective Maintenance Optimization for Multi-State Systems Over a Finite Horizon: A Deep Reinforcement Learning Approach.” European Journal of Operational Research 283 (1): 166–181. doi:https://doi.org/10.1016/j.ejor.2019.10.049.

 Web of Science ®Google Scholar

Epureanu, Bogdan I., Xingyu Li, Aydin Nassehi, and Yoram Koren. 2020. “Self-Repair of Smart Manufacturing Systems by Deep Reinforcement Learning.” CIRP Annals 69 (1): 421–424. doi:https://doi.org/10.1016/j.cirp.2020.04.008.

 Web of Science ®Google Scholar

Skordilis, Erotokritos, and Ramin Moghaddass. 2020. “A Deep Reinforcement Learning Approach for Real-Time Sensor-Driven Decision Making and Predictive Analytics.” Computers & Industrial Engineering 147: 106–600. doi:https://doi.org/10.1016/j.cie.2020.106600.

 Web of Science ®Google Scholar

Zhang, Tai, Fan Zhou, Jun Zhao, and Wei Wang. 2020. “Deep Reinforcement Learning for Secondary Energy Scheduling in Steel Industry.” In 2020 2nd International Conference on Industrial Artificial Intelligence (IAI), Shenyang, China, July 24–26.

 Google Scholar

Bakakeu, Jupiter, Dominik Kisskalt, Joerg Franke, Shirin Baer, Hans-Henning Klos, and Joern Peschke. 2020, August 30–September 2. “Multi-Agent Reinforcement Learning for the Energy Optimization of Cyber-Physical Production Systems.” 2020 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), London, ON, Canada. IEEE, pp. 2–8.

 Google Scholar

Yang, Helin, Arokiaswami Alphones, Wen-De Zhong, Chen Chen, and Xianzhong Xie. 2020. “Learning-Based Energy-Efficient Resource Management by Heterogeneous RF/VLC for Ultra-Reliable Low-Latency Industrial IoT Networks.” IEEE Transactions on Industrial Informatics 16 (8): 5565–5576. doi:https://doi.org/10.1109/TII.2019.2933867.

 Web of Science ®Google Scholar

Leng, Jiewu, Guolei Ruan, Yuan Song, Qiang Liu, Yingbin Fu, Kai Ding, and Xin Chen. 2021. “A Loosely-Coupled Deep Reinforcement Learning Approach for Order Acceptance Decision of Mass-Individualized Printed Circuit Board Manufacturing in Industry 4.0.” Journal of Cleaner Production 280: 124405. doi:https://doi.org/10.1016/j.jclepro.2020.124405.

 Web of Science ®Google Scholar

Bakakeu, Jupiter, Schirin Baer, Jochen Bauer, Hans-Henning Klos, Jörn Peschke, Adrian Fehrle, Werner Eberlein, et al. 2018. “An Artificial Intelligence Approach for Online Optimization of Flexible Manufacturing Systems.” Applied Mechanics and Materials 882: 96–108. doi:https://doi.org/10.4028/www.scientific.net/AMM.882.96.

 Google Scholar

Lu, Renzhi, Yi-Chang Li, Yuting Li, Junhui Jiang, and Yuemin Ding. 2020. “Multi-Agent Deep Reinforcement Learning Based Demand Response for Discrete Manufacturing Systems Energy Management.” Applied Energy 276: 115473. doi:https://doi.org/10.1016/j.apenergy.2020.115473.

 Web of Science ®Google Scholar

Huang, Xuefei, Seung Ho Hong, Mengmeng Yu, Yuemin Ding, and Junhui Jiang. 2019. “Demand Response Management for Industrial Facilities: A Deep Reinforcement Learning Approach.” IEEE Access 7: 82194–82205. doi:https://doi.org/10.1109/ACCESS.2019.2924030.

 Web of Science ®Google Scholar

Hu, Wenqing, Zeyi Sun, Yunchao Zhang, and Yu Li. 2019. “Joint Manufacturing and Onsite Microgrid System Control Using Markov Decision Process and Neural Network Integrated Reinforcement Learning.” Procedia Manufacturing 39: 1242–1249. doi:https://doi.org/10.1016/j.promfg.2020.01.345.

 Google Scholar

Samsonov, Vladimir, Chrismarie Enslin, Hans-Georg Köpken, Schirin Baer, and Daniel Lütticke. 2020. “Using Reinforcement Learning for Optimization of a Workpiece Clamping Position in a Machine Tool.” In Proceedings of the 22nd International Conference on Enterprise Information Systems 506–514. doi:https://doi.org/10.5220/0009354105060514.

 Google Scholar

Wu, Wenbo, Zhengdong Huang, Jiani Zeng, and Kuan Fan. 2021. “A Fast Decision-Making Method for Process Planning with Dynamic Machining Resources Via Deep Reinforcement Learning.” Journal of Manufacturing Systems 58: 392–411. doi:https://doi.org/10.1016/j.jmsy.2020.12.015.

 Web of Science ®Google Scholar

Liao, Haiguang, Wentai Zhang, Xuliang Dong, Barnabas Poczos, Kenji Shimada, and Levent Burak Kara. 2020. “A Deep Reinforcement Learning Approach for Global Routing.” Journal of Mechanical Design142 (6): 061701. doi:https://doi.org/10.1115/1.4045044.

 Web of Science ®Google Scholar

Oh, Tae-Ho, Ji-Seok Han, Young-Seok Kim, Dae-Young Yang, Sang-Hoon Lee, and Dong-Il ‘Dan’ Cho. 2020. “Deep RL Based Notch Filter Design Method for Complex Industrial Servo Systems.” International Journal of Control, Automation and Systems 18 (12): 2983–2992. doi:https://doi.org/10.1007/s12555-020-0153-y.

 Web of Science ®Google Scholar

Zhu, Kai, Naihua Ji, and Xiang Dong Li. 2020. “Hybrid Heuristic Algorithm Based On Improved Rules Reinforcement Learning for 2D Strip Packing Problem.” IEEE Access 8: 226784–226796. doi:https://doi.org/10.1109/ACCESS.2020.3045905.

 Web of Science ®Google Scholar

Scheiderer, Christian, Timo Thun, Christian Idzik, Andrés Felipe Posada-Moreno, Alexander Krämer, Johannes Lohmar, Gerhard Hirt, and Tobias Meisen. 2020. “Simulation-as-a-Service for Reinforcement Learning Applications by Example of Heavy Plate Rolling Processes.” Procedia Manufacturing 51: 897–903. doi:https://doi.org/10.1016/j.promfg.2020.10.126.

 Google Scholar

Zhou, Yu, Tong Xing, Yue Song, Yajing Li, Xuefeng Zhu, Guo Li, and Shuiting Ding. 2020. “Digital-Twin-Driven Geometric Optimization of Centrifugal Impeller with Free-Form Blades for Five-Axis Flank Milling.” Journal of Manufacturing Systems 58: 22–35. doi:https://doi.org/10.1016/j.jmsy.2020.06.019.

 Web of Science ®Google Scholar

Wasmer, Kilian, Tri Le-Quang, Bastian Meylan, and Sergey Shevchik. 2019. “In Situ Quality Monitoring in AM Using Acoustic Emission: A Reinforcement Learning Approach.” Journal of Materials Engineering and Performance 28 (2): 666–672. doi:https://doi.org/10.1007/s11665-018-3690-2.

 Web of Science ®Google Scholar

Brito, Thadeu, Jonas Queiroz, Luis Piardi, Lucas A. Fernandes, José Lima, and Paulo Leitão. 2020. “A Machine Learning Approach for Collaborative Robot Smart Manufacturing Inspection for Quality Control Systems.” Procedia Manufacturing 51: 11–18. doi:https://doi.org/10.1016/j.promfg.2020.10.003.

 Google Scholar

Schmidt, Alexander, Florian Schellroth, and Oliver Riedel. 2020. “Control Architecture for Embedding Reinforcement Learning Frameworks on Industrial Control Hardware.” In Proceedings of the 3rd International Conference on Applications of Intelligent Systems, Las Palmas de Gran Canaria Spain, January 7–12. doi:https://doi.org/10.1145/3378184.3378198.

 Google Scholar

Chien, Chen-Fu, Yun-Siang Lin, and Sheng-Kai Lin. 2020. “Deep Reinforcement Learning for Selecting Demand Forecast Models to Empower Industry 3.5 and An Empirical Study for a Semiconductor Component Distributor.” International Journal of Production Research 58 (9): 2784–2804. doi:https://doi.org/10.1080/00207543.2020.1733125.

 Web of Science ®Google Scholar

Wang, Jun Ping, You Kang Shi, Wen Sheng Zhang, Ian Thomas, and Shi Hui Duan. 2019. “Multitask Policy Adversarial Learning for Human-Level Control With Large State Spaces.” IEEE Transactions on Industrial Informatics 15 (4): 2395–2404. doi:https://doi.org/10.1109/TII.2018.2881266.

 Web of Science ®Google Scholar

Liu, Xing, Wei Yu, Fan Liang, David Griffith, and Nada Golmie. 2021. “On Deep Reinforcement Learning Security for Industrial Internet of Things.” Computer Communications 168: 20–32. doi:https://doi.org/10.1016/j.comcom.2020.12.013.

 Web of Science ®Google Scholar