Hybrid Multi-Evolutionary Algorithm to Solve Optimization Problems

ABSTRACT

The article presents a Hybrid Multi-Evolutionary Algorithm designed to solve optimization problems. The Genetic Algorithm and Evolutionary Strategy work together to improve the efficiency of optimization and increase resistance to getting stuck to sub-optimal solutions. Genetic Algorithm and Evolutionary Strategy can periodically exchange the best individuals from each other. The algorithm combines the ability of the Genetic Algorithm to explore the search space and the ability of the Evolutionary Strategy to exploit the search space. It maintains the right balance between the exploration and exploitation of the search space. The results of the experiments suggest that the proposed algorithm is more effective than the Genetic Algorithms and Evolutionary Strategy used separately, and can be an effective tool in solving complex optimization problems.

Introduction

Evolutionary Algorithms (EA) are a group of methods that mimic natural evolution and the theory of evolution, developed for living organisms by Charles Darwin. They simulate, using computer programs, processes of evolution in the world of living organisms – i.e. natural selection, mutation, recombination of genes, especially the rule of the “survival of the fittest.” The common idea of all EA is the same: they work with a population of individuals, in given environments, which can be defined based on the problem solved by the algorithm. Every individual (a potential solution to the problem) in the form of a chromosome is coded by binary, real numbers or composite data structure. A population is processed in a loop, by operators of fitness evaluation, reproduction, and mutation. An iteration of the loop is called generation. Every individual has assigned a value of fitness function (a numeric value, that determines the adaptation of this individual to the given environment). This function describes the ability of each individual to act as a parent for the next generation of the population. The environmental pressure causes, those well-adapted individuals have a better chance of surviving. Based on fitness values, individuals are selected for the parent’s pool and, as a result, they can transfer their genetic material to offspring. Evolutionary Algorithms search for optimum by the process of exploration and exploitation of search space. Exploration is the process of searching for a new region, where an optimum can exist. A search space is probing with the hope of finding other promising solutions that can be refined. Exploitation is the process of searching within the neighborhood of previously visited points, with the hope of improving a solution found till now. One of the most important problems in EA designing is to keep the right balance between exploration and exploitation. Evolutionary Algorithms are usually used in solving complex optimization problems, for which there are no other specialized techniques. They generally return a satisfying solution in an acceptable period of time but do not guarantee to find the global optimum. There are three basic types of EA: Genetic Algorithms (GA), Evolutionary Strategies (ES), Genetic Programming (GP).

Genetic Algorithm is the best-known type of Evolutionary Algorithms. The individuals in the GA, by default, are coded by binary strings (binary representation), but other representation, i.e. real numbers or composite structures of genes, are also possible. During optimization, GA should keep the right balance between exploration and exploitation, to avoid premature convergence to the local optima. The main parameters of the GA (the probability of selection and probability of mutation) control the ability to explore and exploit the search space and maintain the diversity of the population and the selection pressure. The algorithm can get stuck to the local optima if the exploration is too strong, or it can waste time on poor solutions and cannot focus on good solutions in a neighborhood, if the exploitation is too weak. The process of selection is used as a main mechanism of the exploitation, while crossing-over and mutation are methods of exploring the search space. More information on GAs can be found in publications (Goldberg 1989; Michalewicz 1992).

Evolutionary Strategies, in common with Genetic Algorithms, process a population of individuals in a loop, until the termination criterion is met. In contrast to GAs, ESs use primarily mutation and selection as search operators and the principle, that small changes have small effects. During the mutation, by default, a normally distributed random value is added to each gene of an individual. The size of mutation (i.e. the standard deviation of the normal distribution) can be governed by a self-adaptation process. After a predetermined number of fitness function calls or a number of generations, mutation can be adjusted to satisfy 1/5th rule (the ratio of successful mutations to all mutations). If too many mutations are successful, indicating that the search is too local, the mutation-size should increase, to obtain a greater diversity of individuals. If too few mutations are successful, indicating that the mutation-size is too large, the mutation size should decrease, to increase the accuracy of the search and accelerate the convergence of the algorithm. The simplest Evolutionary Strategy $\left(1+1\right)-ES$ operates on a population of two individuals: the parent (current point) and the child (a result of parent’s mutation). If the child’s fitness is equal to or better than the parent’s fitness, the child becomes a parent in the next generation. Otherwise, the new child is created in the next loop. In strategy $(1+\lambda )-\mathrm{E}\mathrm{S}$, λ children can be created and compete with the parent. In strategy $\left(1,\lambda \right)-ES$ the best child becomes the parent in the next generation while the current parent dies. Strategy $\left(\mu +1\right)-ES$ works with a population of $\mu$ individuals. A child is created as a mutation of a randomly selected parent, and together with the other $\mu$ individuals form a temporary population $\left(\mu +1\right)$. From this population, $\mu$ of the best individuals are selected as the next population (a rejection of the worst). Strategy $\left(\mu +\lambda \right)-ES$ uses the population of $\mu$ parents. A $\lambda$ children are created as a result of mutation and recombination of parents. This makes, that strategy is less prone to get stuck to the local optima. More information on Evolutionary Strategies can be found in publications (Bäck, Hoffmeister, and Schwefel 1991; Beyer and Schwefel 2002).

The hybrid intelligent algorithm consists of a combination of methods and techniques of artificial intelligence, i.e. Genetic Algorithms, Fuzzy Logic, Artificial Neural Networks, etc. Such an algorithm is able to utilize the advantages of artificial intelligence methods and techniques while avoiding their disadvantages. They can be used to improve the efficiency, or where simple methods do not produce the satisfying results in the expected time. Hybrid intelligent algorithms, that use artificial intelligence methods, can be used in different optimization problems, for example, in multiobjective optimization (Pytel 2011; Pytel and Nawarycz 2012, 2010), Connected Facility Location Problem (Pytel and Nawarycz 2013), Clustering Problem (Pytel 2016) or Function Optimization Problem (Pytel 2017).

In this paper, a new Hybrid Multi-Evolutionary Algorithm, that combines the Genetic Algorithm and Evolutionary Strategy is proposed, so it is able to utilize the advantages of GA and ES while avoiding their disadvantages. GA is responsible for searching for new areas of optimum and protection against getting stuck to the local optimum. The operation of ES is limited to the local area, found by the GA. ES is responsible for quick convergence to an optimum value. GA and ES can exchange information about the optimization process and transfer the best individuals between them. The efficiency of the search in a proposed algorithm is enhanced in terms of the time needed to reach the global solution, and the number of calls to fitness function.

Many types of hybrid evolutionary algorithm have been proposed (i.e. Grosan and Abraham 2007; Qian et al. 2018). However, the presented approach differs from them in several aspects, as will be clear in the following. The main contributions of the paper can be summarized as follows:

• A new idea of the hybrid multi-evolutionary algorithm is presented, which consists of the Genetic Algorithm and Evolutionary Strategy.

• GA and ES are working in parallel, so the algorithm can use multiple processors and/or cores.

• A new protocol for exchanging information and the best individuals between GA and ES was developed.

• Extensive experiments on test functions show that the proposed algorithm achieves high-quality solutions and very good performance in comparison with other evolutionary methods.

The paper is organized as follows: Section “Problem Formulation” defines the problem to solve, and formalizes it as the Functions Optimization Problem. Section “The Proposed Hybrid Multi-Evolutionary Algorithm” reviews the recent proposals on function optimization and introduces a new hybrid multi-evolutionary algorithm. Section “Computational Experiment” describes in detail the experiments. In the first stage of experiments, different models of the algorithm were tested. As a result of this stage, the most effective model was chosen to stage 2. In the second stage of an experiment, a proposed algorithm was compared to Genetic Algorithms and Evolutionary Strategies. Section “Conclusions”, finally, concludes the paper and suggests future developments.

Problem Formulation

Optimization is the selection of the best element from the set of elements, with regard to some criterion, without violating some constraint. The Functions Optimization Problem (FOP) is a problem in which parameters of the function (variables) need to be adjusted to achieve the best (maximal or minimal) value of an objective function, under given constraints. The function optimization problem (e.g. a function maximizing problem) can be stated as follows:

(1) $\left\{\begin{array}{c}maxf\left(x\right)\\ subjectto:c_i\le 0fori=1,2,\dots ,k\\ x\in S\end{array}\right.$(1)

where: $x=\left[x_1,x_2,x_3,\dots ,x_n\right]\in \mathrm{\Re };n\in N$ – is an n-dimensional vector of decision variables, $f\left(x\right)$ – is the objective function of variables $\mathrm{x}$, ${\mathrm{c}}_{\mathrm{i}}\left(\mathrm{x}\right)$ – are constraints, $\mathrm{S}$ – the search area.

The FOP problem can be used as a benchmark for various optimization methods, including Genetic Algorithms and Evolutionary Strategies. Various methods of solving FOP are discussed in the literature, for example (Jensi and Wiselin 2016; Karaboga and Basturk 2007; Potter and De Jong 1994).

The Proposed Hybrid Multi-Evolutionary Algorithm

Genetic Algorithms can search space of possible solutions, by use operators of cross-over and mutation, to solve sophisticated optimization problems. A binary representation of individuals is usually used in GA, but real numbers or complex data structures can also be used. Its convergence is very much dependent on the initial population. GAs are the most useful to discrete problems. One of the disadvantages of GA is low efficiency in the final search stage. Evolutionary Strategies, by default, use real numbers as a representation of individuals. They use primarily mutation and selection, as evolution operators. Competition between parents and descendants, for the selection as a new parent for the next generation, causes evolution pressure. ESs are prone to get stuck to sub-optimal solutions. The advantage of ESs is the rapid convergence to the optimum value. ESs should be applied to continuous problems. Hybridization of both algorithms can be a promising method for solving many continuous and discrete optimization problems.

The proposed algorithm (GA-ES) combines a GA’s ability to explore the search space and find areas of possible optimality and ability of ES’s to quickly converge to the optima. GA and ES can use the same operators of selection, mutation, individuals’ representation and start with the same initial population. After a predetermined number of generations, the best individuals from both algorithms can be compared. Depending on the result of this comparison, the transposition of individuals between the algorithms may be performed:

• If the best individual in the GA is better than the best individual in the ES, this means finding a new area of optimum. The best individual in the GA replaces the parent, or the worst parent if there is more than one parent in the ES.

• If the best individual in the ES is better than the best individual in the GA, then it means that ES found a new optimal solution. The best individual in the ES replaces the worst individual in the GA population.

The block diagram of a proposed hybrid algorithm is shown in Figure 1.

Figure 1. The hybrid algorithm block diagram.

Computational Experiment

The proposed algorithm can be used in different optimization problems, including solving the Function Optimization Problem. The experiments were divided into two stages. In the first stage, algorithms were compared, in which different types of evolutionary strategies were used, for example $\left(1+1\right)-ES$, $\left(\mu +1\right)-ES$, $\left(\mu ,\lambda \right)-ES$, and $\left(\mu +\lambda \right)-ES$. The aim of this stage was to determine the optimal type of Evolutionary Strategy that best suits solving test problems. In the second stage, the results of the proposed hybrid algorithm were compared with the Evolutionary Strategy and the Genetic Algorithm.

A wide range of functions with different complexities in a diverse environment has been used during research:

• $f_1\left(x_1,x_2\right)$ – an easy function of two variables. The function has many local optima and was used for testing the algorithm’s ability to find the global optimum. The function is given by the formula:

(2) $f_1\left(x_1,x_2\right)=\left(sin\left(x_1\right)+0.6\ast sin\left(20\ast x_1\right)\right)\ast sin\left(x_2\right)$(2)

where: $x_1,x_2\in \left(0,\pi \right)$

The value of maximum 1.6 at point $\left(\pi /2,\pi /2\right)$.

• $f_2\left(x_1,x_2,\dots ,x_{10}\right)$- the function of multiple variables with a low inclination angle, used for testing the ability of the algorithm to determine the exact solution. The function is given by the formula:

  (3) $f_2\left(x_1,x_2,\dots ,x_{10}\right)=\prod _{i=1}^{10}sin\left(x_i\right)$(3)

where: $x_1,x_2,\dots ,x_{10}\in \left(0,\pi \right)$

The value of maximum 1 at point $\left(\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2},\frac{\pi }{2}\right)$.

• $f_3$ – one of the functions proposed in (Kwasnicka 1999). It is a very sophisticated function with many local optima with different values. This function permits to estimate the ability of the algorithm to solve difficult optimization problems. The first generation of individuals was placed in the local optimum (point [5, 5] with value 1.5). The algorithm should find the global optimum (point [50, 50] with value 2.5), avoiding the local optima (with values 1.0 and 2.0). The function is given by the formula:

  (4) $f_3\left(x_1,x_2\right)=\sum _1^7{h}_i\ast e^{-{\mu }_i\ast {\left(x_1-x_{1_i}\right)}^2+{\left(x_2-x_{2_i}\right)}^2}$(4)

where: $h_1=1.5,h_2=1,h_3=1,h_4=1,h_5=2,h_6=2,h_7=2.5$

${\mu }_1={\mu }_2={\mu }_3={\mu }_4={\mu }_5={\mu }_6={\mu }_7=0.01$

$\left(x_{11},x_{12}\right)=\left(5,5\right),\left(x_{21},x_{22}\right)=\left(5,30\right),\left(x_{31},x_{32}\right)=\left(25,25\right),$

$\left(x_{41},x_{42}\right)=\left(30,5\right),\left(x_{51},x_{52}\right)=\left(50,20\right),\left(x_{61},x_{62}\right)=\left(20,50\right),$

$\left(x_{71},x_{72}\right)=\left(50,50\right)$

• The Rastrigin function – is a non-convex, non-linear multimodal function. It was first proposed as a two-dimensional function and has been generalized to an n-dimensional domain (in experiments 2, 5 and 10 dimensions were used). Finding the minimum of this function is very difficult, due to a huge search space and a large number of local minima. The function is usually evaluated on the hypercube $x_i\in \left[-5.12,5.12\right]$, for all $i=1,\dots ,d$ and the minimum $f\left(x^{\ast }\right)=0$ is at point $x^{\ast }=\left(0,\dots ,0\right)$. The function is given by the formula:

(5) $f_{Ra}\left(x\right)=10d+\sum _{i=1}^d\left(x_i^2-10cos\left(2\pi x_i\right)\right)$(5)

• The Styblinski-Tang function – is a continuous, multimodal, non-convex function, generalized to an n-dimensional domain (in experiments 2, 5 and 10 dimensions were used). Finding the minimum of this function is very difficult, due to its huge search space and its shape. The function is usually evaluated on the hypercube $x_i\in \left[-5,5\right]$, for all $i=1,\dots ,d$. The minimum is at point $x^{\ast }=\left(-2.903534,\dots ,-2.903534\right)$, and his value is $f\left(x^{\ast }\right)=-39.16599d$. The function is given by the formula:

(6) $f_{ST}\left(x\right)=\frac{1}{2}+\sum _{i=1}^d(x_i^4-16x_i^2+5x_i)$(6)

• The Rosenbrock function (also referred to as the Valley or Banana function) – is a popular test problem for gradient-based optimization algorithms. The function is generalized to an n-dimensional domain (in experiments 2 dimensions were used). The function is unimodal, and the global minimum lies in a narrow, parabolic valley. However, even though, this valley is easy to find, convergence to the minimum is difficult. The function is usually evaluated on the hypercube $x_i\in \left[-5,10\right]$, for all $i=1,\dots ,d$ and the minimum $f\left(x^{\ast }\right)=0$ is at point $x^{\ast }=\left(1,\dots ,1\right)$. The function is given by the formula:

(7) $f_{Ro}\left(x\right)=\sum _{i=1}^{d-1}\left(100{\left(x_{i+1}-x_i^2\right)}^2+{\left(x_i-1\right)}^2\right)$(7)

• The Shubert function – two-dimensional function with several local minima and many global minima. The function is usually evaluated on the square $x_i\in \left[-10,10\right]$, for all $i=1,2$. The minimum value is $f\left(x\right)=-186.7309$. The function is given by the formula:

(8) $f_{Sh}\left(x\right)=\left(\sum _{i=1}^{5}i\mathrm{c}\mathrm{o}\mathrm{s}\left(\left(i+1\right)x_1+i\right)\right)\left(\sum _{i=1}^{5}i\mathrm{c}\mathrm{o}\mathrm{s}\left(\left(i+1\right)x_2+i\right)\right)$(8)

Some of the chosen test functions are a minimization problem. In this case, they were converted into the maximization problem by negation and adding constant C. The constant C was determined for each function during the initial experiments. The values of the Genetic Algorithm’s parameters were set during the initial experiments. In the experiments, the following values of the Genetic Algorithm’s parameters were used:

• the genes of individuals are represented by real numbers,

• the probability of crossover = 0.8,

• the probability of mutation = 0.15,

• the number of individuals in the population = 25.

Various types of Evolutionary Strategies were used in experiments. For all of them, the parameters were set during the initial experiments. The mutation model with the addition of a random number in accordance with the normal distribution was used. To maintain the rule of 1/5 successes, if for the next 10 generations there was no improvement, the size of the mutation was enlarged by increasing $\sigma$parameter. The number of parents and descendants in various types of ES was set based on recommendations from the literature and presented in Table 1.

Table 1. The number of parents and descendants in different types of ESs.

ES	Number of parents	Number of descendants
$\left(\mu +1\right)-\mathrm{E}\mathrm{S}$	20	1
$\left(\mu ,\lambda \right)-\mathrm{E}\mathrm{S}$	30	200
$\left(\mu +\lambda \right)-\mathrm{E}\mathrm{S}$	20	160

The fitness of the best individuals, found by GA and ES was compared after every 50 generations. Depending on the result of this comparison, the best individuals have been transferred between GA and ES. The algorithm was stopped when the best individual reached the predetermined value of the optimized function. The value of the stop criterion and the value of the constant C used for each function are presented in Table 2.

Table 2. The value of the algorithm stop criterion and the value of the constant C used for each function.

Function	Stop criterion	constant C
$f_1$	1.596	-
$f_2$	0.999	-
$f_3$	2.50048	-
$f_{Ra}2d$	74.999	75
$f_{Ra}5d$	149.999	150
$f_{Ra}10d$	289.5	290
$f_{ST}2d$	656.663	500
$f_{ST}5d$	1641.6	1250
$f_{ST}10d$	2383.0	2500
$f_{Ro}$	1210120.99	1210121
$f_{Sh}$	271.795	187

Each algorithm was executed 10 times on a standard PC computer (CPU: Intel i3, RAM: 8GB, Windows 10 operating system). Table 3 shows the average time and the average number of calls to fitness function needed to reach the predetermined value for test functions in stage 1 of an experiment.

Table 3. The average running time and the number of fitness function calls needed to reach the predetermined value of the optimized function.

Algorithm	Number of fitness calls	σ	Running time [s]	σ
Function ${\mathbf{f}}_{\mathbf{1}}$
$\mathrm{G}\mathrm{A}-\left(1+1\right)\mathrm{E}\mathrm{S}$	4394	2548	0.000029	0.000014
$\mathrm{G}\mathrm{A}-\left(\mu +1\right)\mathrm{E}\mathrm{S}$	57500	31841	0.000022	0.000011
$\mathrm{G}\mathrm{A}-\left(\mu ,\lambda \right)ES$	197700	156065	0.000194	0.000106
$\mathrm{G}\mathrm{A}-\left(\mu +\lambda \right)\mathrm{E}\mathrm{S}$	234696	162480	0.000201	0.000113
Function ${\mathbf{f}}_{\mathbf{2}}$
$\mathrm{G}\mathrm{A}-\left(1+1\right)\mathrm{E}\mathrm{S}$	8450	685	0.000048	0.000016
$\mathrm{G}\mathrm{A}-\left(\mu +1\right)\mathrm{E}\mathrm{S}$	205800	26078	0.000069	0.000007
$\mathrm{G}\mathrm{A}-\left(\mu ,\lambda \right)ES$	927700	275635	0.002464	0.000713
$\mathrm{G}\mathrm{A}-\left(\mu +\lambda \right)\mathrm{E}\mathrm{S}$	755601	210262	0.001792	0.000483
Function ${\mathbf{f}}_{\mathbf{3}}$
$\mathrm{G}\mathrm{A}-\left(1+1\right)\mathrm{E}\mathrm{S}$	7137	2405	0.000072	0.000027
$\mathrm{G}\mathrm{A}-\left(\mu +1\right)\mathrm{E}\mathrm{S}$	840450	548589	0.000158	0.000010
$\mathrm{G}\mathrm{A}-\left(\mu ,\lambda \right)ES$	621400	345817	0.000677	0.000348
$\mathrm{G}\mathrm{A}-\left(\mu +\lambda \right)\mathrm{E}\mathrm{S}$	565194	361504	0.000586	0.000336

Based on results from stage 1 experiments, a combination of GA with (1 + 1)-ES has the highest efficiency. In stage 2, this algorithm will be used to compare results with the Evolutionary Strategy (ES), and the Standard Genetic Algorithm (SGA) described in (Michalewicz 1992) and adapted to optimize the test functions. Each algorithm was executed 10 times on a standard PC computer (CPU: Intel i3, RAM: 8GB, Windows 10 operating system). Table 4 shows the average time and the average number of calls to fitness function needed to reach the predetermined value for the test functions in stage 2 of an experiment. The graph in Figure 2 shows the average running time and the graph in Figure 3 shows the number of calls to fitness function, in the Genetic Algorithm (SGA), the Evolutionary Strategy and the proposed (GA-ES) algorithm.

Table 4. The average time and number of fitness function calls needed to reach the predetermined value of the optimized function.

Function		f1	f2	f3	fRa2d	fRa5d	fRa10d
Number of fitness function calls	SGA	43317	31380	75927	53547	609717	967040
	ES	3459	3136*	16300	750909	*	*
	GA-ES	4427	8320	5990	7092	41730	459019
σ	SGA	35232	3622	49888	28302	298351	306805
	ES	7272	1268*	15662	671132	*	*
	GA-ES	2535	1096	1520	2587	20003	116278
Time [s]	SGA	0.0732	0.0379	0.1506	0.0794	0.7763	1.8962
	ES	0.0309	14.403*	0.0683	0.7091	*	*
	GA-ES	0.0365	0.0614	0.0415	0.0342	0.0997	1.0523
σ	SGA	0.0282	0.0059	0.0778	0.3128	0.3344	0.7065
	ES	0.0074	44.833	0.0399	0.5797	*	*
	GA-ES	0.0069	0.0082	0.0057	0.0064	0.0352	0.2440
Function		fST2d	fST5d	fST10d	fRO	fSH	-
Number of fitness function calls	SGA	60942	193842	857025	651025	42222	-
	ES	46345	*	*	3868	40722	-
	GA-ES	5720	11310	28990	43295	4030	-
σ	SGA	28981	90410	281590	680334	36062	-
	ES	137035	*	*	2522	70610	-
	GA-ES	1397	2297	7606	27897	737	-
Time [s]	SGA	0.0725	0.2098	0.9966	0.3772	0.1182	-
	ES	0.0578	*	*	0.0324	0.1492	-
	GA-ES	0.0398	0.0504	0.0982	0.0641	0.0448	-
σ	SGA	0.0195	0.0807	0.3044	0.3469	0.0623	-
	ES	0.0770	*	*	0.0038	0.1939	-
	GA-ES	0.0056	0.0077	0.0281	0.0184	0.0070	-

*- at least once, the algorithm got stuck to the local optima.

Figure 2. The average running time in the Genetic Algorithm (SGA), the Evolutionary Strategy and the proposed (GA-ES) algorithm.

Figure 3. The number of fitness function calls in the Genetic Algorithm (SGA), the Evolutionary Strategy, and the proposed (GA-ES) algorithm.

Conclusions

The proposed hybrid Genetic Algorithm-Evolutionary Strategy algorithm was able to find an acceptable solution for all tested functions.

In GA-ES algorithm, GA is responsible for searching for new areas, where optimum can exist, and provide protection against getting stuck to the local optima. ES operation can be limited to only the local area. It is responsible for searching the local area found by the GA.

The $\left(1+1\right)ES$ strategy is the most vulnerable to getting stuck to local optima. Strategies $\left(\mu +1\right)ES$, $\left(\mu ,\lambda \right)ES$ and $\left(\mu +\lambda \right)ES$ can increase the algorithm’s resilience to get stuck to the local optima but they require additional computing effort. The combination of GA and $\left(1+1\right)ES$ is the most efficient.

In all tasks, the GA-ES algorithm needed less fitness function calls, than SGA. The standard deviation of the found solutions was lower, which may indicate greater stability of the algorithm. In one task, a running time of the GA-ES algorithm was longer than the SGA, in all other tasks, the running time of GA-ES was shorter compared to SGA.

ES gives the best results when optimizing easy functions such as $f_1$. However, the performance of this algorithm is not satisfying for the $f_2$ function with many local optima or when it has to overcome the valley like in function $f_3$. In this case, it may get stuck to the local optima or requires a large number of calls to the fitness function. ES was unable to solve four tasks from the tests, and in one of the tasks, the algorithm gets stuck to the local optima at least one time. In four tasks, the GA-ES algorithm needed less fitness function calls, than ES. In two tasks, a running time of GA-ES algorithm was longer than ES, in all other tasks, the running time of GA-ES was shorter.

The optimization of the functions has shown that the proposed algorithm has greater convergence and accuracy compared to SGA and the algorithm is more resistant to premature convergence to the local optimum compared to the ES’s.

In the proposed algorithm it is possible to perform parallel calculations by the Genetic Algorithm and the Evolutionary Strategy, e.g. by using multiple processors or processor cores.

The proposed algorithm is an efficient tool for solving function optimization problems. It could be used for solving a very wide range of optimization problems.

In single-objective optimization, for example, Function Optimization Problem, there is only one objective function that should be optimized. This problem is often used to test various optimization methods. In real life, problems with multi-criteria optimization are very common. In this case, there are at least two functions, which have to be optimized at the same time. No single solution exists that simultaneously optimizes each objective. This task is much more difficult and can result in a set of optimal solutions (Pareto optimal solutions). Future work will try to extend the approach to the Multi-objective Optimization Problem.

Statement

This manuscript has not been published elsewhere and that it has not been submitted simultaneously for publication elsewhere.