Recent advances on cooperative control of heterogeneous multi-agent systems subject to constraints: a survey

Abstract

In this paper, the cooperative control of heterogeneous multi-agent systems (HMASs) subject to certain constraints are surveyed. First, HMASs are classified into two categories, namely, weak and strong HMASs, according to different cooperative behaviours. Then, control strategies are discussed for HMASs when facing different sorts of constraints imposed on the agent dynamics and on the communication networks. Subsequently, some latest results on the cooperative control subject to various kinds of constraints are summarized. Finally, conclusions are drawn, and several possible future research directions are pointed out.

Keywords:

• Heterogeneous multi-agent systems
• cooperative control
• anti-disturbance control
• secure control

1. Introduction

In the past few decades, the application of multi-agent systems has multiplied in various areas ranging from the manufacturing industry and electrical engineering to military applications and modern medical industry. These beneficial applications in human community are partly attributed to the extensive and profound academical researches. In practice, agents can be diverse and complicated in the real world. However, in the existing literature, agents in MASs are usually assumed to have homogeneous dynamics which is simple but can rapidly promote the development of this field undeniably. Consequently, these proposed methods encounter great difficulties coping with the heterogeneous MASs which are actually more common than their homogeneous counterparts in the real-world. As a result, we can see from recent researches that more and more scholars have devoted to seeking solutions for better control of the heterogeneous MASs by improving existing algorithms or inventing new ones.

The cooperative control problem is one of the hottest issues of MASs aside from task allocation (M. Li et al., 2021). It can be classified into three categories, i.e. consensus control, formation control and containment control. Consensus control aims to render some states or outputs of all the agents in the MASs to reach consensus. Moreover, it is worth pointing out that in homogeneous MASs, state consensus is most studied, but as for HMASs, output consensus is much more popular for the reason that the possibly different order of each agent brings difficulties to the state consensus. Formation control asks the MASs to form a prescribed shape while moving in order to perform specific tasks or avoid collision. Containment control drives the states or outputs of followers to converge to the convex hull spanned by the states or outputs of leaders (Peng et al., 2020), the major objective of which is for better survival.

When it comes to heterogeneous multi-agent systems, the existing control protocols designed for homogeneous MASs seem inoperative. Challenges arise from a variety of aspects. The main reason is the nature of HMASs that not all agents have exactly the same dynamics, which means a single controller is not always suitable for every agent. Furthermore, as well as their homogeneous counterparts, HMASs are also inevitably surrounded by a variety of limitations in practical engineering but these are more difficult to handle. On the one hand, each agent suffers from disturbances or uncertainties, even severe actuator faults during its working hours. The disturbances may stem from external environmental factors such as wind and electromagnetic interference and the parameter uncertainties can derive from unmodelled dynamics or mechanical wear. On the other hand, the fragile wireless communication network may bring threats to the whole system. For instance, with the development of computer and information technology, the cyber attack draws extensive attention, especially in industrial engineering and military. In addition, sudden changes in situation or adjustment of missions can also cause instability. All these constraints should be put great emphasis on and dealt with properly.

In this paper, we focus mainly on the cooperative control of heterogeneous multi-agent systems with various constraints and aim to present some recent advances in this area as adequately as possible. Constraints on subsystems including parameter uncertainties, disturbances, and faults and on the communication network consisting of network-induced phenomena, switching topologies and cyber attack have been discussed in great detail with some recent advances along with common cooperative control strategies and techniques for HMASs.

The rest of the paper is organized as follows. In Section 2, the types of HMASs and their cooperative behaviours are introduced with some recent advances. Cooperative control of HMASs with constraints on each agent and communication network is surveyed in Sections 3 and 4, respectively. Section 5 gives some latest progress on the cooperative control of HMASs with various constraints. The conclusions and future work are provided in Section 6. The framework of the overview is shown in Figure 1.

Figure 1. Framework of the survey.

2. Cooperation of heterogeneous multi-agent systems

In this section, at first, the HMASs in the existing literature are recalled in detail. Then two main cooperation behaviours are introduced including formation and containment.

2.1. Types of heterogeneous multi-agent systems

Recently many results have claimed that their researches are implemented for heterogeneous multi-agent systems. By going through the related work, we have found that there are two main kinds of HMASs resulting in different avenues to obtain cooperation. In the following, both of these types of HMAS are introduced.

2.1.1. Agents with identical system structures but different parameters

The feature of the first type of HMASs is that the nonlinearities, the system parameters or the uncertainties of each sub-system(agent) are different from each other but the system structures are identical. Giving a practical example for better understanding, this kind of MASs can be seen in the drones show which uses identical UAVs armed with lights to form pictures in the sky. But the drones are possibly faced with different nonlinearities as they are in different ages or positions. It is very common in the literature too (J. Gong et al., 2021; Lim & Lee, 2021; X.-H. Zhang et al., 2018; X. Zhang et al., 2017). Specifically, for linear HMASs, the system matrices or the uncertainties of system matrices are different among agents which is actually the simplest situation in HMASs. But in the real world, nonlinearities are non-negligible. Thus, nonlinear HMASs are also profoundly studied. In nonlinear HMASs, the nonlinear functions can be non-identical but are generally assumed to satisfy the global Lipschitz condition which facilitates derivation yet is too strict to some extent. Therefore, some work proposed weaker conditions so that to include much more nonlinear functions; see in S. Yang et al. (2015). Moreover, some results such as T. Yan et al. (2021) and S. Shi et al. (2019), even removed the Lipschitz or Lipschitz-like conditions. This type of HMASs is easier to deal with considering the identical subsystem structures and to be called weak HMASs(WHMASs) in the following contents.

Another factor that may result in the weak heterogeneity of HMASs is the stochasticity of subsystems. Although some work assumes that the stochasticity is same among each agent B. Li et al. (2020) and Y. Yuan et al. (2018), it is more general that the stochasticity is non-identical, rendering the system heterogenous; see L. Ma, Wang and Lam (2017) and L. Ma et al. (2016). Probabilistic-constrained (Tian et al., 2019b) or chance-constrained (Tian et al., 2019a) methods are preferred to deal with such stochastic components in system dynamics. As is seen in L. Ma, Wang, Han, et al. (2017), a quantity of researches concerning stochastic MASs has found wide applications in many fields recently. Therefore, the stochasticity of HMASs should also be attached great importance to.

2.1.2. Agents with non-identical system structures

It becomes more prevalent that the agents in the systems are completely different due to the more and more complicated tasks. For instance, in the modern battle field, there exists more cooperation among UAVs(Unmanned Ariel Vehicles) in the sky and UGVs(Unmanned Ground Vehicles) on the ground, even more with USVs(Unmanned Surface Vehicles) in the sea. These vehicles have different degrees of freedom and different system characteristics which makes the cooperative control more complicated and hard to accomplish. This kind of heterogeneous MASs seen in the literature is usually composed of first-order and second-order sub-systems (Sun et al., 2021) or with nonlinear in addition (Du et al., 2020; Yadollahi & Atrianfar, 2019). Among all these models, the single or double integral models are most popular because they are derived from Newton's laws of motion which are able to describe the dynamics of a variety of moving objects such as robots and aircrafts. Whereas, the integral models are not suitable in some cases. Therefore, high-order and fractional-order HMASs also deserve more intensive researches. But unfortunately, results are rare in this area. In the following discussion, this kind of HMASs will be called strong HMASs(SHMASs) for short.

2.2. Cooperative behaviours

Multi-agent systems are designed to execute tasks in place of human beings, hence, there are some main cooperative behaviours they are expected to implement such as formation and containment. All the behaviours need the communication and cooperation of the agents. In the next two sections, formation and containment in HMASs are introduced with some recent results.

2.2.1. Formation

Formation is one of the most researched behaviours of MASs as it can be applied to a lot of occasions, such as UAVs and satellites for special purposes. Take the UAV swarm for example, as is well known, the idea that the ‘V’ shape can decrease the flight resistance is inspired by the migration behaviour of birds. Other than that, when meeting obstacles, the linear shape will allow the swarm to pass through safely. There are mainly three ways to achieve this effect. One of the most frequently seen is the leader-follower method.In this structure, there are one or more agents play the roles of leaders in the HMASs and others will serve as followers. Then the formation is realized by controlling the relative position or distance between leader and follower agents. Moreover, the leader agents can also be virtual; see in Han and Luo (2016), Jiang et al. (2018) and W. Cheng, Zhang, et al. (2021). Another two common methods i.e. rigid structure method and behaviour-based method, however, are nearly seen in the researches on HMASs, which are actually widely applied in the formation of homogeneous MASs.

2.2.2. Containment

Containment control, based on the leader-follower strategy, provides the followers with the ability to reach the target point without some sensors by just following the leaders. This cooperative behaviour enables MASs to achieve more functions even when some of the members are not that intelligent. Related results can be seen both for WHMASs (D. Wang et al., 2020) and SHMASs (Zheng & Wang, 2014) and the dynamics are assumed to be non-identical between leaders and followers but identical among leaders or followers. In some work like Zheng and Wang (2014), the outputs of followers are asked to converge to the convex hull spanned by the leaders' outputs while other researches prefer to obtain the states containment (Cong et al., 2018; X. Zhang et al., 2020; S. Zuo et al., 2018). With the development of digital networks, it is worth noted that similar to sampled-data consensus problem (W. Zhang et al., 2018), the containment of MASs with sampled-data also deserves more investigation but it will consequentially encounter much more difficulties. Besides, the sampling period could fluctuate around a nominal value in some cases (L. Zou, Wang & Zhou, 2020). Related work for homogeneous MASs can be seen in W. Zhang et al. (2017) but for HMASs there is few results.

3. Cooperative control of heterogeneous MASs with various constraints on each agent

Agents in HMASs are also constrained to multiple factors. In this section, we recall the cooperative control of HMASs with various constraints on each agent. Three common phenomena in HMASs along with the control protocols are introduced including parameter uncertainties, disturbances, and faults.

3.1. Parameter uncertainties

Parameter uncertainties arise from various aspects. Firstly, generally speaking, the ideal model for HMASs can not reflect the actual dynamics of each agent, resulting in a difference between the model and the true dynamics. Hence, the error modelled by uncertainties must be considered when designing cooperative control protocols. Besides, perturbation in system parameters is also very common in the practical engineering which possibly results from mechanical wear or other unmeasurable changes. Such perturbation is also modelled as uncertainties in the research of HMASs.

There are several approaches to dealing with uncertainties occurred in HMASs. One of the most common methods is to estimate the uncertain parameters in the systems for further design of the controllers. The first step of this method is to set a variable to estimate the uncertain parameter (H.-X. Hu et al., 2021) whereby the estimation problem can be solved using tools such as the LMI technique. The next step is always to design a controller such as the output feedback controller (R. Yang et al., 2020) to achieve cooperative control.

Another famous scheme is the internal model approach which is suitable for nonlinear HMASs. By designing an internal model for a HMAS, a cooperative control problem can be converted into a stabilization problem. Moreover, combined with a distributed controller, uncertainties in HMASs can be well handled (Z. Xu et al., 2016; Y. Zhang & Ji, 2016).

Sliding mode control is also an applicable method to handling such uncertainties. In this control method, the most essential step is to design an appropriate sliding surface, which can include an integral item with a signal function like (Xue & Wu, 2021). With a sliding mode controller, the cooperative control objective can be realized even when existing the parameter uncertainties. For example, Islam et al. (2018) proposed a distributed sliding mode control method that guaranteed the robustness of a WHMAS without knowing the bound of the dynamical uncertainty.

It should be pointed out that if the disturbances are assumed to have the similar forms with the parameter uncertainties, protocols are often designed to deal with these two constraints simultaneously; see in Back and Kim (2017), P. Gong and Lan (2017) and Rezaei and Stefanovic (2021).

3.2. Disturbances

Disturbances are inevitable in a MAS no matter it is homogeneous or heterogeneous. The characteristics of disturbances can be quite different. For example, wind noise and measurement noise can be modelled as low or high frequency disturbances, respectively, and unknown but bounded disturbances can describe the electromagnetic interference or actuator errors. Besides, disturbances can be matched or unmatched (Y. Yuan et al., 2017), or even both. Moreover, the disturbances may be stochastic (M. Wang et al., 2020; L. Zou et al., 2016) in practical engineering.

For HMASs, the internal model approach and sliding mode control mentioned in the last subsection are also suitable for cooperative control problem under the influence of disturbances; see in Lu and Liu (2021), S. Li et al. (2015), C. Wang et al. (2019) and Ye et al. (2018), respectively. The $H_{\mathrm{\infty }}$ control is also a famous method to address the external disturbances for HMASs; see L. Zou et al. (2017) and Sheng et al. (2017). But aside from these approaches, there are other methods that can address this problem, among which the recently emerging observer-based approach designed specifically for anti-disturbance control stands out and draws great attention. In observer-based control scheme, the controller must utilize the estimation of the observer and it is noted that the disturbances are always assumed to be bounded and continuously differentiable. Related work is seen in the literature, for instance, Back and Kim (2017) has utilized a reduced-order disturbance observer to achieve cooperative tracking control and J. Wu et al. (2017) has designed a distributed observer combined with a output feedback control algorithm to obtain robust synchronization of HMASs.

3.3. Faults

In practical applications, actuators in MASs are required to perform control operations. Apparently, it can be expected that the actuators of some agents may fail under some circumstances. Another possible failure is the sensor fault. Without a well-functional sensor, an agent will have no access to its local information. Thus, if the fault is not well treated then the whole system may collapse. Fault-tolerant control theory is proposed and developed to deal with this specific type of problem. Modelling the fault is fundamental in the fault-tolerant control method. Actuator faults are usually presented by the input-output relationship with fault signal. For example, in W. Cheng, Zhang, et al. (2021), the fault model has a function to evaluate the severity of the loss-of-effectiveness fault and a coefficient to denote the bias fault in a double-integral HMAS. These two parameters are enough to describe four kinds of actuator fault situations, namely fault-free, partial loss of effectiveness fault, bias fault and hybrid fault case. Besides, some work also considers the outage fault, which means the origin input is invalid; see X. Li and Shi (2019). Also, there are researches taking only some of these faults into account. Sensor faults are often defined by the perturbed output with fault parameter (Maaruf & Ferik, 2021) and can be classified into bias faults and drift faults (Gao et al., 2021b) but unfortunately work on HMASs with sensor fault is still rarely seen.

Fault-tolerant control mainly includes active control (Gao, Wang, He, & Han, 2021) and passive fault-tolerant control (Gao et al., 2021a). Passive control methods has almost vanished in the literature due to their limitation on fault tolerance capacity compared with the active methods. In some work, by adding auxiliary control gain (Xiao & Dong, 2020) or fault-tolerant function (Deng, 2019) for HMASs, the partial loss of effectiveness actuator fault in nonlinear HMASs can be well addressed. We can say they are active because the control inputs are adaptive to the change of fault coefficients. It is worth noting that there are some cases that the actuator fault is modelled similarly to matched disturbances therefore anti-disturbances control methods for HMASs are applied to resist such faults (C. Liu et al., 2019) and J. Zhao et al. (2019). Besides, fault estimation is needed after fault detection in some cases (J. Hu et al., 2018), and observers are often utilized to design a fault-tolerant controller too; see in C. Chen et al. (2018), Fan et al. (2021) and L.-X. Xu et al. (2021). The above-mentioned active fault-tolerance control methods are on the condition of accurate fault diagnosis, which is not needed in other proposed active fault tolerant control. For instance, the virtual actuator method has been applied to HMASs in Yadegar and Meskin (2021), which is capable of actively dealing with actuator faults without fault detection module. In addition, as we have mentioned before, the sampled-date can also cause difficulties in the fault-tolerant control of MASs, related work for homogeneous MASs is seen in Gao, Wang, He and Yue (2021), however, there is no similar efforts reported in HMASs.

4. Cooperative control of heterogeneous MASs with constraints on communication network

Cooperative control of HMASs is based on the communication among agents or among agents and control centres, but unfortunately, the wireless network is vulnerable to multiple interferences. In this section, three main factors which may make a difference in the control process are recalled with some solutions in recent literature.

4.1. Common network-induced phenomena

Network-induced phenomena include, but are not limited to, missing measurements, fading measurements, signal quantization, time delays, randomly occurring nonlinearities, probabilistic sensor delays, sensor saturations, and so on (J. Hu et al., 2015). In this subsection, some of the network-induced phenomena and solutions for HMASs are introduced including communication time-delay, sensor saturation and signal quantization. The others have not been seen in HMASs related researches so far.

Communication time-delay is inevitable especially for heterogeneous MASs and always accounts for system's performance degradation or sometimes even leads to instability (L. Ma et al., 2019). The delay of information must cause the asynchronism in the cooperative control, for which reason more and more work takes communication time-delay into consideration when designing the controller. In homogeneous MASs, many schemes have been applied to deal with it. However, when it comes to HMASs, the heterogeneity will possibly makes the time-delays among agents or channels quite different which will invalidate those methods. Thus, the communication time-delays in HMASs cooperative control researches are often assumed to be non-uniform. There are three main strategies to deal with the problem for HAMSs, one of which is to utilize the information at previous sample time to compensate for delays actively (Deng et al., 2020; Tan et al., 2020). Another possible choice is adding the time-delay into the control protocol so that the controller is adaptive to the time-varying time-delay (Abdessameud & Tayebi, 2018; Su et al., 2021; J. Wang et al., 2019). Besides, the internal model method combined with output feedback control is also proved to be effective; see in Lu and Liu (2021) and X. Xu et al. (2017).

The sensor saturation, which stems from the limited ability to measure signals, is generally described as nonlinearity in the outputs of agents. It is also called output saturation in some researches because the outputs of agents in MASs are always measured by sensors in practical engineering. Unfortunately, as far as we know, there is only one work (H. Zhao et al., 2021) on cooperative control for HMASs with sensor saturation. The authors have proposed a model-free adaptive control approach to obtain consensus in a class of WHMASs. It is optimistically believed that in the future there will be more results which can address the sensor saturation problem in HMASs.

With the development of digital technology, more and more MASs are armed with digital sensors and other digital devices. Although the high accuracy of digital devices is beneficial, the quantization problem arising from the nature of them also brings difficulties in the cooperative control of MASs, which has aroused great interest recently (L. Zou, Wang, Hu, et al., 2020). For HMASs, there are rare results focusing on this particular problem except Choi and Yoo (2021) where the authors utilize a neural network to design a quantized-states-based local adaptive control for the tracking problem.

Other network-induced phenomena can be seen in a great deal of results concerning homogeneous multi-agent systems (Ding, Wang, Ho, et al., 2016; W. Xu et al., 2017) or sensor networks (J. Hu et al., 2017, 2020; Shen et al., 2016) that are instructive for future research on HMASs.

4.2. Switching topologies

Topologies in MASs are very likely to be time-varying or say switching due to the possible build of new links or communication failures. They are usually described as piecewise constant, changing among all possible known choices. A switching signal is always proposed to describe the moment when topologies change. But more importantly, it can be utilized to design adaptive control protocols to deal with switching topologies. Moreover, it is worth pointing out that the Markov switching topology which is more practical in the real world deserves more investigations as well. In Markov switching process, the topology of a MAS at each instance can be selected randomly from all the possible topologies, and the probability of each possible topology being selected is only related to the state at the last time step. Ji et al. (2020) has studied the consensus control of HMASs with Markov switching topology with continuous Markov process analysis. Similar results can also be seen in Meng et al. (2017) but apart from these two pieces of research, there is little related work in the literature.

There are two main solutions in the existing literature. Firstly, event-triggered strategy is well applied; see in J. Zhang et al. (2020) and H. Zhang et al. (2020). Specifically, the edge-event trigger mechanism is proposed and applied suitable for the cooperation problem in HMASs. The triggering instants depend on the switching moment of topology and the triggering functions are decided by the edge of each agent. It is worth mentioning that the Zeno behaviour that the triggering occurs an infinite number of times in a finite amount of time should be ruled out, often by adding exponential function to the triggering function. An edge-based adaptive control protocol has been developed in Lv et al. (2017) and a common Lyapunov function for every possible graph is designed. Besides, a node-duplication-based technique is proposed in Guangqi et al. (2019) to address the SHMASs consensus problem under switching topologies. In other work without event-triggering mechanism, authors just add the switching signal to the controller design for the purpose of adapting the switching topologies; see in Hua et al. (2018) and J.-L. Shi et al. (2019).

However, the existing results concerning cooperative control of HMASs under switching topologies are almost all based on the known possible topologies with fixed agent members, which actually can not cover some conditions in the real situation. For example, the random loss of some members will cause the topology totally unpredictable. Such complicated occasions still remain to be discussed.

4.3. Secure control of heterogeneous MASs when under cyber attacks

Communication network is susceptible to malicious attacks which will paralyze the whole system if not well coped with. In the literature regarding MASs, there are three main kinds of attacks being widely studied. The first and most common one is the Denial-of-Service(DoS) attack where the transmission of information is blocked by the attacker. The second one is the deception attack who destroys systems by injecting false data into the transmitted information. Another well-known malicious attack is the replay attack which has once caused great loss in industry around the world. The attacker will store the previous data and replay them in the next time step to confuse systems.

Secure control is developed including resilient control for MASs under threat of cyber attacks. For homogeneous multi-agent systems, fruitful results are accessible in secure control under DoS attack (H. Yang et al., 2019; Y. Yang et al., 2020), deception attack (Cui et al., 2018; L. Liu et al., 2020) and (Ding, Wang, Han, et al., 2016) and replay attack (Franzè et al., 2020; J. Liu et al., 2020). Moreover, the hybrid attacks composed of more than one kind of attack are considered and addressed for homogeneous MASs in X.-M.Li et al. (2021). And just like other topics, secure control for HMASs still has a long way to go. In the existing literature so far, only cooperative control under the simplest attack, DoS attack has been studied for HMASs.

To describe a DoS attack behaviour needs two essential parameters, i.e. the number of attacks and the attack duration in a finite time interval. Besides, because of the limited energy of adversaries, the DoS attack can only last for a period of time. In researches of HMASs, the DoS attack is modelled in different ways. For instance, in D. Zhang and Feng (2019), the attack is modelled as a general time-varying delay without any stochasticity, while in D. Zhang et al. (2018) and Z. Xu et al. (2020), the DoS attacks are described as a Bernoulli model and a Markovian jump model, respectively. It is noted that some work such as D. Zhang et al. (2018) assumes the attack happens aperiodically which is actually more practical in engineering.

To address the secure control of HMASs, event-trigger mechanism is obviously an imaginable solution. Even-triggering can not only reduce communication burden (Ding, Wang, & Han, 2019a; Ge et al., 2018; Q. Li, Shen et al., 2020) but also can prevent the MASs from the influence of DoS attack if the triggering conditions are well designed; see Yin et al. (2020). Distributed resilient observers are designed to resist the DoS attack for HMASs in Deng and Wen (2020) and Q. Li, Xia et al. (2020). Based on the designed observers which can estimate the unknown dynamics, feedback controller or $H_{\mathrm{\infty }}$ controller is proposed to realize the cooperative control of HMASs under DoS attack. $H_{\mathrm{\infty }}$ control and other similar robust control methods were originally proposed in order to handle perturbations of system parameters and unmodelled dynamics, but after the unremitting endeavour of a mass of scholars, these control methods now are able to cope with a great variety of system uncertainties. Among them stands out the $H_{\mathrm{\infty }}$ control. Plenty of control problems with regard to robustness can be transformed into standard $H_{\mathrm{\infty }}$ control problems to be solved. In addition, the feedback control method is able to resist the DoS attack as well without any observer. For example, in D. Zhang et al. (2018), the authors add the attack rate and duration into the control protocol to resist periodical DoS attack for HMASs adaptively.

5. Latest progress

Very recently, the cooperative control problem of heterogenous multi-agent systems with various constraints have drawn considerable attention and some elegant results have been reported. In this section, we highlight some of the newest work with respect to this topic.

1. In Katsoukis and Rovithakis (2021), a more general and practical nonlinear SHMASs model has been proposed consisting of a leader and a lot more followers. The followers are with different and perhaps high dynamic orders, moreover the nonlinearities are also unknown. A distributed control protocol has been designed to address the output consensus control problem, proved to have reduced the complexity of analysis and calculation. It is of great significance because agents need less information and less expensive equipped computers to achieve output consensus.

2. Finite-time and fixed-time control based on state feedback are applied to linear WHMASs in Hao et al. (2020). These two proposed control methods will render the system obtain output consensus in a finite time or a fixed time, the latter of which is independent of the initial states of agents. The output regulation technique and Lyapunov stability theory are also utilized to obtain the results.

3. In Mahmoud and Karaki (2020), a dissipative approach has solved the output synchronization of discrete-time WMASs. Sufficient conditions have been derived to guarantee dissipativity for each agent and obtain the controller parameters, respectively, and the proposed control scheme has been proved to be of strong robustness facing norm-bound uncertainties. Moreover, an event-triggering mechanism has been designed so as to deal with the admissible uncertainties and time-varying delay.

4. For a typical affine nonlinear SHMAS, H.-J. Ma et al. (2020) has addressed the optimal dynamic formation problem by a decision unit and a control unit. The control unit is designed via a displacement-gradient method to obtain the local optimal solution based on the generated desired trajectory by the decision unit. It is worth mentioning that an event-triggering mechanism is also applied to solve the optimal dynamic formation problem when the communication is intermittent.

5. Uncertainties in HMASs can be static or dynamic, both of which are considered in Tang and Wang (2020). The optimal output consensus for a type of WHMASs has been converted into a distributed partial stabilization problem by designing optimal signal generators. Subsequently a novel adaptive control protocol with a dynamic compensator has been proposed which can handle the difficulties brought by two types of uncertainties.

6. In Sun et al. (2020), except for a SHMAS comprising both first-order and second-order dynamic agents, a type of HMASs composed of both continuous-time and discrete-time agents has been studied. For the latter type of HMASs, which can be seen in MASs consisting of analog and digital subsystems in practical engineering, the authors have designed a distributed subgradient algorithm so that it will drive the agents to obtain optimal positions.

7. Difficulties of cooperative control brought by non-uniform and time-varying information delay have been overcome in J. Wang et al. (2019). It has been noted that a large enough damping coefficient can deal with the aforementioned type of uncertainties so that an adaptive scheme has been proposed, where the damping coefficient is able to adapt to the change of the time delay without knowing its exact value. The robust rendezvous problem for a class of second-order WHMASs has been well addressed with two types of node-based adaptive control protocols, respectively.

6. Conclusions and future work

In this paper, the cooperative control problem of heterogeneous multi-agent systems has been reviewed with some recent advances. Two main types of constraints in HMASs along with some methods have been surveyed. Based on the literature review, some related topics for future research work are listed as follows.

It is noticed that most of the studied HMASs are simple weak HMASs which are actually similar to homogeneous MASs and less practical. Therefore, more general and complicated HMASs models should be constructed so that proposed control protocols can be better applied to real engineering. With the increasing number of agents in a whole system, the interaction network becomes more and more complex which has stirred great interest recently (Q. Li et al., 2018; Shen, Wang, Wang, & Li, 2019). Hence, the cooperative control of heterogeneous multi-agent systems with complex networks is also an important research direction in the future. Moreover, discrete-time heterogeneous multi-agent systems are rarely considered for its complexity but related work can be seen for homogeneous MASs (W. Chen, Wang, et al., 2021; L. Wang et al., 2018) and sensor networks (Ge et al., 2017; M. Wang , 2019) in the existing literature.

Furthermore, plenty of network-induced phenomena (e.g. probabilistic sensor delays, randomly occurring nonlinearities, etc.) have not been studied so far for HMASs which are actually important in practical engineering (Shen, Wang, Wang, Luo, et al., 2019). Furthermore, some advanced communication protocols designed to avoid data collisions (L. Zou et al., 2019) may also cause such phenomena. Consequently, cooperative control still remains open and challenging with such constraints.

Another future research direction is the secure control problem for HMASs on which the relevant research has been far from adequate. It can be seen from our survey that only DoS attack has been considered so far. Other types of attacks such as replay attack may also happen in the control process of HMASs. Hence, it is an urgent topic to be discussed in the future.

The actuator saturation is one of the most common nonlinearities, which has been extensively investigated for homogeneous MASs; see in Gao et al. (2020) and Gao, Wang, He, and Han (2021). In HMASs, it is expected that the saturation characteristic of each agent is non-identical and it can bring multiple difficulties when designing control protocols. Thus, the cooperative control of HMASs under heterogeneous actuator saturations should be intensively studied.

Neural networks (X.-M. Zhang et al., 2017) and deep-reinforcement learning (Zeng et al., 2021) have been extensively studied in the past few decades and there are widespread applications in various fields including the cooperative control of MASs (Ding, Wang, & Han, 2019b). Neural networks can be utilized to approximate the nonlinear kinematics function involving various complex cases (Ding, Han, et al., 2019) and reinforcement learning can help agents make better decisions. It is imaginable that in the near future these artificial intelligence techniques be applied to the control of HMASs.

Data availability statement

The authors confirm that the no data are used in this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported in part by the Natural Science Foundation of Jiangsu Province [grant number BK20130766], and in part by the Ministry of Industry and Information Technology [grant number JCkY2021602B035].