2.1.1 Complex and unpredictable systems

MASs are complex systems and also can be unpredictable systems. Complex systems are composed of two or more interconnected components that create an object network that interacts with each other and exhibits dynamic and aggregate behavior. Also, the action of an element can affect the activities of different elements in the network. It is essential to emphasize that in this paradigm ‘the whole is greater than the sum of its parts’.

Unpredictable systems are complex systems with a high degree of instability and unpredictability in the decision-making process and the results of the agents’ actions. It is important to point out that these aspects need to be addressed dynamically. According to Lempert (Reference Lempert2002), agent-based models are often useful under conditions of considerable uncertainty, where a reliable prediction of the future is not possible.

2.1.2 Self-organizing and open systems

MAS is self-organizing and open systems. Self-organization is a process where the organization of a system is not guided or managed by an external entity but by its entities that make up the system. In self-organized MAS, the agents can structure themselves spatially and temporally in an environment, allowing agents to adapt to their environment through an autonomous reorganization dynamically. With the capacity for self-organization, it is possible to develop MASs with complex dynamics and adaptable to environmental disturbances, without complete knowledge of future conditions.

MASs are considered open systems because the agents are able to enter and leave society, and even with the insertion/withdrawal of some of its members, the system maintains its regular operation. The migration of agents to other MAS is intended to help in problem-solving. Because they are open systems, MASs have high adaptability to the environment, so they get to establish the relationships and communication between agents dynamically.

2.1.3 Emergent behaviors

MASs present emergent behaviors. According to Axelrod (Reference Axelrod1998), these behaviors can be described as large-scale effects caused by local agent interactions. Such results are noted as stable and non-obvious macroscopic patterns emerging from local interaction rules between individual agents.

It is worth pointing out that the emergent behaviors are not explicitly programmed in the agents because they are socially constructed behaviors. In some models, the emergent properties can be formally deduced, but they can also be unpredictable and unexpected since even the consequences of simple local interactions can present considerable difficulties in being anticipated (Ruas et al., Reference Ruas, das, Marietto, de Moraes Batista, dos, França, Heideker, Noronha, da Silva, Alkhateeb, Maghayreh and Doush2011).

2.1.4 Reactive, cognitive, and hybrid agents

In MASs, the agents are usually classified in the following groups: reactive, cognitive, and hybrid. A reactive agent acts reactively to its environment changes. Therefore, its functional model is based on stimulus response. In general, these agents do not have any memory, their future actions are not planned, and direct communication with other agents does not happen. The indirect communication happens by observing the changes occurred in the environment. The reactive agents are implemented with low computational complexity, based on biological or ethological organization models, such as ant and termite societies (Dorigo et al., Reference Dorigo, Birattari and Stützle2006).

Cognitive agents have a specified computational complexity and an explicit representation of the environment and community members. They have memory and can think about their actions in the past and plan their future. These agents communicate with each other directly. In this case, their perception systems (that allows to examine the environment) and the communication (that allows the message exchanges between agents) are different. The human groups are the metaphor used for structuring the agents’ community. In these groups, teams of specialists solve problems in a coordinated way, and the complexity exceeds the individual capacities of each of its members.

Hybrid agents combine the functionalities of reactive and cognitive agents. The objective of this combination is to overcome the limitations presented in the reactive and cognitive architectures. The limitation of cognitive architecture refers to its difficulty in performing fast and efficient reactions to an unforeseen situation. On the other hand, reactive architectures have difficulties in defining alternatives for the agent behaviors and change their actions plans when the condition of the environment diverges from their pre-established objectives.

2.1.5 Autonomous agents

Agents are considered autonomous when they operate without the direct intervention of other external systems (humans or computers), presenting some control over their actions and internal states.

In Rezende (Reference Rezende2005), an agent is considered autonomous when it can analyze a specific situation to be solved, to create alternatives, and to make choices that best suit the specific goals. There are cases in which the agent is not able to recognize the environment. However, to find a solution to the problem, the agent chooses a previous experience.

2.1.6 Coordination: cooperation, collaboration, and competition

In order to make the community act coherently, Nick Jennings (Reference Jennings1995) defines coordination as a process by which an agent rationalizes its local actions and anticipates other agents’ actions. MASs assume coordination among their agents because: (i) no individual has sufficient competence, resources, or information to solve the problem in an isolated manner; (ii) there are interdependencies between agents’ actions; (iii) the system presents global constrains, which must be respected; and (iv) information, resources, and expertise must be distributed.

The coordination process must be able to deal with systems formed by agents with antagonistic or common objectives. In the case of antagonistic objectives, coordination involves issues related to competition. The collaboration is established when the agent has the autonomy to perform a task and can perform it alone. However, for example, if this agent wants to optimize the execution time of this task, it can accept or request collaboration with other agents. Cooperation occurs when an agent is not able to perform a particular task. Therefore, it is needed to ask for the help of other agents.

In this search for a solution in a distributed intelligent system, the coordination process must manage the agents’ behavior since they have local and incomplete information on the problem. This management aims to avoid conflicts, redundant efforts, deadlocks, among others. Therefore, the more the incidence of such situations, the less coordinated the system is.

2.1.7 Communication

In a MAS, the agents often need to exchange information in a coordinated way, such as knowledge and action plans. The communication between agents can occur in the following ways (Ruas et al., Reference Ruas, das, Marietto, de Moraes Batista, dos, França, Heideker, Noronha, da Silva, Alkhateeb, Maghayreh and Doush2011; Yan et al., Reference Yan, Jouandeau and Cherif2013):

• Direct (Explicit): in this type of communication, the agents exchange information directly between them. Therefore, the MAS must provide mechanisms for the agents to identify themselves and know the ‘addresses’ of the other agents in the system. The message exchange mechanism has been adopted in direct communication, applying specific languages to the agents’ communication;

• Indirect (Implicit): this type of communication occurs in the environment where the agents are inserted. For example, using signals flags or shared structures, such as blackboards;

• Without communication: this interaction occurs when an Agent A can directly change the KB of an Agent B by inserting in Agent B’s KB beliefs, goals, among others.

Blackboards. A blackboard is a type of shared memory where agents can exchange information. MAS that uses this strategy as a communication method consists of the following components:

• An independent set of agents;

• A shared data structure called blackboard, through which agents communicate with one another;

• A control system is responsible for defining the order in which agents operate on the blackboard.

Agents write their information items on the blackboard. In this case, there is no direct communication between agents, and the communication and coordination processes are mediated by the state of the solution stored in the blackboard.

Message exchange. In the message exchange, instead of the blackboard systems, the messages are exchanged directly between agents. These messages can be used to establish communication and coordination mechanisms through predefined protocols and to acquire new facts and beliefs. With this type of explicit communication, each agent must know more about the other agents involved in the system (Fallah-Seghrouchini et al., Reference Fallah-Seghrouchini, Haddad and Mazouzi2001).

Speech Acts is one of the linguistic approaches that have been presented adequately to model the syntactic and semantics of a natural language. This theory was first developed by John Austin and later improved by John Searle (Reference Searle1969). Austin noted that certain expressions were like physical actions that changed the state of the world. For example, when a jury declares a defendant guilty, an action is taken. In this case, the situation and social status of the defendant are changed (Huhns & Stephens, Reference Huhns and Stephens1999).

Speech Acts Theory has served as inspiration for the creation of Agent Communication Languages (ACLs). An ACL is a language used specifically for the communication between agents. This language offers a theoretical-technical structure where an agent can identify the responses that another agent emits about its message. In some cases, it can predict the possible receiver’s reactions. Thus, to achieve their goals, agents can plan their speech acts to affect the recipients’ beliefs, goals, and other mental states. Knowledge query and manipulation language (KQML) (Chalupsky et al., Reference Chalupsky, Finin, Fritzson, McKay, Shapiro and Wiederhold1992) and foundation for intelligent physical agents (FIPA) ACL (ACL, 2002) languages are examples of ACLs.

2.2.1 Homogeneous, heterogeneous, and hybrids

Homogeneity refers to the degree of similarity that exists between the robots of the same MRS. The homogeneous robots are built based on the same architecture, which is reproduced in each robot. For this reason, in this type of robot, the robotic agents have high similarity in the design of their internal modules, software, and hardware components and, consequently, in their operation.

In homogeneous systems, each robot performs the same actions with similar results. Thus, if any of these robots fail, the others can compensate for it. The idea is to reduce the susceptibility of fault system and increase the reliability and robustness of the solution. However, the homogeneity of the robots reduces the possibility of change in the functioning of the MRS. Consequently, these systems become less adaptable.

Murphy (Reference Murphy2000a) describes an example of a homogeneous MRS presented in a competition called ‘Pick Up the Trash’, held in 1994 and organized by the Association for the Advancement of Artificial Intelligence (AAAI). In this competition, the robots Io, Ganymede, and Callisto, developed by Georgia Tech, obtained the first place. They quickly and efficiently recognize trash, such as aluminum cans, and throw them in a specific deposit. For this, reactive behaviors were used to collect trash, and cooperative behaviors provide for cooperation between robots. The robots were also equipped with an infrared sensor in the gripper hands used to alert the object in range for grasping. The gripper captures and holds the soda cans, and a color camera was responsible for identifying the cans, wastebaskets, and other robots (Balch et al., Reference Balch, Boone, Collins, Forbes, Mackenzie and Santamaria1995).

On the other hand, heterogeneous robots are constructed from different architectures, differing by their internal modules, control mechanisms, and methods of interpreting information (Murphy, Reference Murphy2000a; Iocchi et al., Reference Iocchi, Nardi and Salerno2001). The heterogeneous MRSs allow a greater adaptation to a new situation and, in some cases, in dynamic environments. If there are robots in the environment with different capabilities interacting for a common purpose, then they cover a greater number of problems. However, there is a greater challenge and difficulty in implementing such a system since it is necessary to integrate each robot’s functions so that a coordinated interaction is possible (Iocchi et al., Reference Iocchi, Nardi and Salerno2001). The motion planning and stabilization approach are proposed in Spurny et al. (Reference Spurny, Baca and Saska2016). It provides an effective technique that solves heterogeneous unmanned ground vehicles (UGV) and micro aerial vehicles (MAVs) formation to target region problem. This technique uses the model predictive control method based on the virtual leader-follower approach as the formation control strategy.

The marsupial robots are an example of a heterogeneous MRS. The main objective of these robots is to develop micro-rovers to be used in areas of difficult access, such as in rescue missions. A marsupial team consists of a large robot that carries one or smaller robots to the task site, much like a kangaroo mother carries a joey in her pouch (Murphy, Reference Murphy2000a). A marsupial and shape-shifting robots for urban search and rescue (USAR) are proposed in Murphy (Reference Murphy2000b) and called Silver Bullet and Bujold robots, respectively. The USAR is focused on locating and extracting people trapped in collapsed or damaged structures. The Bujold robots navigate through voids. The robots are asked to change their configuration to be able to explore different types of environments. Therefore, the authors focused on solving the USAR by considering Bujold robots and a Silver Bullet’s Power Wheels chassis as members of the cooperative and heterogeneous team of robots. The heterogeneous MRS described in Roldán et al. (Reference Roldán, Garcia-Aunon, Garzón, de León, del Cerro and Barrientos2016) includes both ground and aerial vehicles were used to monitor the environmental variable of greenhouses, such as temperature, humidity, luminosity, among others.

Silver Bullet robot is responsible for carrying the Bujold robot, much like a kangaroo carrying its young. In order to accomplish this task, the Silver Bullet has wheels, an onboard computer, radio Ethernet link, GPS, six sonar sensors, inclinometers, two cameras, and a thermal sensor for detecting survivors. On the other hand, the Bujold is a micro-rover physically connected to Silver Bullet. It has a camera, a microphone for hearing survivors, and two headlights. This robot also changes three configurations: sitting up and facing forward, sitting up and facing backward, and laying flat (Murphy, Reference Murphy2000b). Silver Bullet and Bujold robots are developed to work together. For example, the Silver Bullet controls the Bujold, the Bujold uses the Silver Bullet’s computer to run its dedicated software, or the Silver Bullet’s radio Ethernet link is responsible for controlling the Bujold through teleoperation.

The hybrids MRSs evolve groups of homogeneous robots, and at least one heterogeneous robot. In Simmons et al. (Reference Simmons, Apfelbaum, Burgard, Fox, Moors, Thrun and Younes2000), a team of two Pioneer AT robots from Real World Interface Inc. (RWI) and an Urbie robot from IS Robotics is used for exploration and mapping of an unknown indoor environment. These robots are equipped with SICK laser scanners that have a 180° field of view, and the communication is realized via Breezecom radio links. The proposed MRS has two homogeneous robots and one heterogeneous robot, and therefore it is considered as a hybrid MRS.

2.2.2 Coordination

MAS area uses several mechanisms for the management of coordination processes. In this section, some of them are presented, with examples of MRSs that also use the same mechanisms, such as synchronization, organizational structures, and planning.

Synchronization The synchronization of actions in a MAS aims to coordinate the parallel tasks in time, chaining the actions carried out by its agents. Synchronization processes typically use distributed systems techniques.

In non-distributed systems, running on a single central processing unit, semaphores and monitors are generally used to solve synchronization issues. However, synchronization algorithms in distributed systems consider that each machine has its time mechanisms, which moves the software clock forward. The great challenge of these algorithms is to synchronize the clocks (software) of all machines for a similar time pattern. In Lamport (Reference Lamport1978), Lamport showed that the synchronization between different clocks is not needed to be absolute, and an algorithm to synchronize actions in a distributed system is presented.

As an example of MRS that uses synchronization techniques to coordinate the actions of its robots, in Bouteraa et al. (Reference Bouteraa, Mansour, Ghommam and Poisson2011), a synchronized control of manipulator robots is proposed. The strategy is to synchronize each robot in position and velocity concerning the commonly desired trajectory. The control combines trajectory tracking and coordination algorithms. In order to follow a common trajectory, the authors applied the control to synchronize three identical manipulator robots interconnected under a cooperative scheme.

Organizational structure An organizational structure is the formalization of the interaction types between human beings, offering a framework of constraints and expectations on the agent behaviors. This framework guides the interactions between agents through expected behaviors, roles to be assumed by agents, authority relations, among others. In the literature, it is possible to find some types of organizational structures, for example, hierarchical organization, centralized organization, and community of specialists.

In a hierarchical organization, the agents are separated into groups, and a supervisor coordinates each group. The organization is composed of interrelated subsystems, where the power of decision and control is assigned to the supervisor of each group, and a supervised agent belonging to a group can be a supervisor in another group. This type of organization structure allows the distribution of coordination flows between different levels of supervision.

The mapping, navigation, and planning software is organized hierarchically in the MRS described in Butzke et al. (Reference Butzke, Daniilidis and Kushleyev2012). In this work, the main objective was to attend the Multi Autonomous Ground-robotic International Challenge (MAGIC). In the MAGIC 2010 competition, the challenge was to develop a MRS that could explore and map a large dynamic urban environment. Robots should locate, classify, and respond to threats with minimal human intervention. Also, the solution was to use a hierarchical architecture decomposition of perceptual, planning, and control algorithms.

The hierarchical software architecture integrates high-level human operator commands with autonomous and semiautonomous operations. The main objective is to have an efficient high-level interaction from the human operators while simultaneously, the robots operate autonomously. In the sensing and mapping tasks, a bottom-up hierarchical architecture is proposed. However, in the planning and strategic control, a top-down hierarchical architecture is introduced. It allows human operators to efficiently issue task commands to get propagated down a chain of planners and controllers. For example, human operators can order the robots to move to some locations and execute clear-cut actions (Butzke et al., Reference Butzke, Daniilidis and Kushleyev2012).

In a centralized organization, an agent has authority over the other agents in the MAS. Therefore, a subordinate agent should accept the commands from the centralized agent. For instance, in Barrett et al. (Reference Barrett, Rabideau, Estlin and Chien2001), a population of three rovers is remotely commanded by central planning in a leader/follower approach. A lander generates plans that are executed by the rovers. In order to facilitate projection revision, the rovers use sensors to observe local conditions and send updates back to the lander.

A community of specialists is organized based on the following items: (i) it is flat because all agents are on the same level, that is, there are no hierarchical differences between them; (ii) each agent is specialist in a particular domain; (iii) the agents interact with each other, aiming a mutual adjustment of their actions so that they can collectively reach the solution of the problem. Swarmanoid (Dorigo & et al., Reference Dorigo2013) is an example of a specialized agent community. In this work, three different robot types have been used, and each robot has a specialty to solve a particular task. The small foot-bots are specialized in transporting objects or other robots. Also, they move on even and uneven terrains. The hand-bots are capable of climbing some vertical surfaces and manipulating small objects, for example, a book. Finally, the eye-bots are flying robots specialized in exploring the environment and searching for the target. In this work, the objective was to find the shelf and retrieve a book.

Swarmanoid robots are assumed to start from a single deployment area. During the exploration, until finding the target location, the eye-bots build a wireless network that spans the environment by sequentially connecting to the ceiling. This network is used to guide the new eye-bot that joins in the exploration task. The foot-bots form another wireless network on the ground that follows the eye-bot network topology. These networks can pass range and bearing messages between each other. When the eye-bot finds a shelf containing objects, the hand-bot is asked to move at the shelf location to retrieve a book. This request is sent to the deployment area through the network of eye-bots and foot-bots. In this area, the foot-bots assemble to hand-bots and start collectively transporting them to the shelf. Now, these robots are called foot-hand-bot. When they arrive at a shelf location, the eye-bot tells them to climb and to what height. After that, the foot-bots disassemble from the hand-bot. Latter starts to climb the structure, grasps the book, and descends from the shelf. Then, the foot-hand-bot reassembles and moves back to the deployment. Finally, the foot-hand-bot disassembles and waits for further swarmanoid tasks (Dorigo & et al., Reference Dorigo2013).

Planning The coordination work can be aided by an explicit division of tasks, making the agents’ behavior to be directed toward the overall objectives of the system. It can be done through planning theory. In general, multi-agent planning is either centralized or decentralized. In centralized planning, an agent generates a plan for the whole society. This plan is split into subplans that are distributed among the agents in society. In the decentralized planning, each agent generates its partial plane, and during the operation of the MAS, the agents exchange information among themselves to identify and resolve conflicts that arise.

Hierarchical task network (HTN) is proposed to coordinate a team of agents in dynamic domains, such as in the RoboCup Soccer Simulation League 3D. The proposed HTN planner is decentralized because each agent has its planner to coordinate their actions. The robotic soccer domain is a dynamic and non-deterministic environment. In order to achieve coordinated team behavior, the HTN planners are used in each of the agents for high-level coordination and description of team strategies. The main idea in this work is to use the HTN to perform complex or primitive tasks, and also all planning should be done in a distributed fashion. However, for complex tasks, the planner expands them into subtasks until the tasks are primitive (Obst & Boedecker, Reference Obst, Boedecker, Bredenfeld, Jacoff, Noda and Takahashi2006).

A centralized motion planning framework within dynamic robot networks is presented in Clark et al. (Reference Clark, Rock and Latombe2003). It enables multiple mobile robots to maneuver and achieve goals safely in dynamic environments, for example, in the planetary rover and free-floating space. However, robots have limited ranges of sensing and also communication. In order to overcome these weaknesses, the approach is based on dynamic robot networks. This approach allows forming dynamically whenever communication and sensing capabilities permit, sharing the world models and robot targets within each network, and also constructing coordinated trajectories for all robots in each network using a fast-centralized motion planner. The framework is validated in planning the robots within a bounded workspace. The robot was able to re-plan and react in real time to changes in the environment with other robots, stationary, and moving obstacles.

The problem of concurrent assignment and planning trajectories for MRS was considered in Turpin et al. (Reference Turpin, Michael and Kumar2014). A centralized solution to the problem of assigning goals and planning trajectories was applied to a team of eight KMel Robotics NanoQuad quadrotors, which successfully arrive at their goal locations. However, to reduce the maximum distance traveled between a pair of robots, a decentralized algorithm was also proposed. It required local reassignment and re-planning within the communication range.

2.2.3 Communication

As in MAS, communication between robots is necessary for effective coordination in MRSs. Each robot must perceive the environment, act as needed, and also analyze what happens to other robots through communication processes. In this process, robots receive and send information for intelligent decision-making. The main objective of communication is to share information and also to coordinate activities among robots. A team of three homogeneous planar robots was deployed in Hsieh et al. (Reference Hsieh, Cowley, Kumar and Taylor2008) to obtain a radio connectivity map in an urban environment. The radio signal strength data were important for operations, such as surveillance, reconnaissance, among others.

As seen in Section 2.1, the communication is explicit when the messages, via unicast or broadcast, are exchanged directly between the robots. In implicit communication, the robots get information about other robots through the environment or communicate with each other trough blackboard systems. In this case, it is needed to embed different kinds of sensors in the robot (Yan et al., Reference Yan, Jouandeau and Cherif2013).

In Robotics, there are several communication types available, and the robotic agent decides which is the most efficient. For example, communication by wiring cables, infrared, microwave, and wireless. It is worth pointing out that, according to Reis et al. (Reference Reis, Lima, Garcia, Behnke, Veloso, Visser and Xiong2014), wireless communications are one of the technical problems that must be considered by a group of robots. For example, some robots take more time to transmit information, or the network is consistently overloaded. In this case, the cooperative behaviors tasks are ineffective. Ducatelle et al. (Reference Ducatelle, Caro, Pinciroli, Mondada and Gambardella2011) tested in simulation and on real robots a communication-based navigation algorithm for robotic swarms through the wireless network. It used a wireless communication device that provides only line-of-sight communication so that communication links can be related to navigable paths. Therefore, to make the robots to follow paths detected through communication, an Infrared Range-and-Bearing communication system was used to link received messages to relative position information about their sender.

In order to establish reliable wireless communication with a team of client agents who are performing an independent task, Gil et al. (Reference Gil, Kumar, Katabi and Rus2015) deployed an auxiliary team of robot routers. The idea is to find the best positions of the routers that support variable demanded rates to the client’s robots. A novel method is proposed to capture the spatial variation of wireless signals and also to provide the infrastructure to support the requested quality of communication. In the paper, the method is explained in detail, and the results show its efficiency.

A decentralized flocking model for a large group of autonomous flying robots was proposed in Vásárhelyi et al. (Reference Vásárhelyi, Virágh, Somorjai, Nepusz, Eiben and Vicsek2018). It worked in a noisy environment with short-range communication devices, and in the presence of substantial communication delay and with possible local communication outages. It is important to highlight that the communication outages and delays are common in developing decentralized control algorithms for swarms of flying robots.

2.2.4 Centralized and decentralized

One of the great challenges of the Robotics field is related to the decision made by the robot and also in the next action to be performed after a specific task is finished (Pirjanian, Reference Pirjanian1997). In MRS, these decisions can be made in a centralized and decentralized way.

In a centralized system, there is a central control agent, sometimes called a leader, who is in charge of coordinating and organizing the work of other robotic agents. In this case, the leader is involved in the whole system operation, while the other members in the MRS act according to the leader’s guidelines. Also, the leader has all environmental information, including information about the robots.

According to Yan et al. (Reference Yan, Jouandeau and Cherif2013), the advantage of centralized architectures is that the globally optimal plans can be produced because the leader has a global view of the environment. However, the robots are not able to adapt in dynamic environments, once if the leader fails and another robot is not available, the entire MRS is disabled. Due to this, it is not suited MRS operating in unknown and changing environments, for instance, Stanek (Reference Stanek2012).

A centralized MRS architecture to be used in industrial automation is designed in Dahale et al. (Reference Dahale, Chaudhari and Rane2014). It consists of three pick and place robots, three mobile robots, a microcontroller as a controller for coordinating the robots, and a network of infrared sensors to detect objects. The central controller uses the network of sensors to get all the information about the environment, for example, if the robot finished or not a certain task. This information is obtained through a centralized approach, and also optimal plans can be produced from global information.

In this MRS, the work area is divided into zones, and each robot is responsible for executing a task in its zone. After the robot performs the task, it is ready for the next task to be executed or remains idle. Also, the robot assists the other robot only if an available task is detected in its zone (Dahale et al., Reference Dahale, Chaudhari and Rane2014).

On the other hand, a decentralized system is composed of autonomous agents in their decision-making processes, without any leader to control the system. Robots make their own decisions, but in a coordinated way, considering the other robots in the MRS. A decentralized MRS is more scalable, autonomous, independent, and able to achieve its best performance in the execution of tasks. For example, a self-organized swarm of 30 drones without central control exhibiting flocking with collective collision and object avoidance was proposed in Vásárhelyi et al. (Reference Vásárhelyi, Virágh, Somorjai, Nepusz, Eiben and Vicsek2018).

Matsumoto et al. (Reference Matsumoto, Asama, Ishida, Ozaki and Endo1990) presented an autonomous and decentralized robot system called ACTor-based Robots and Equipments Synthetic System (ACTRESS). It is composed of multiple robotic elements, such as mobile robots, manipulator robots, intelligent sensors, intelligent actuators, special equipment, among others. In order to achieve a task, the robots communicate with each other by exchanging messages. In the cooperation problem, the robots negotiate, offer, inquiry, or announce the information.

A decentralized air traffic control solution using drones was proposed in Balázs and Vásárhelyi (Reference Balázs and Vásárhelyi2018). The multiple-path conflicts and flow bottlenecks scenarios were considered in the simulation and real experiments. In these scenarios, drones were placed in a crosswalk situation and had the center of the square-formed common destination. It is important to point out that the simulation framework provides solutions for worst-case scenarios regarding noise and delay to make sure the uncertainty in the reality gap will not decrease safety. A 2D traffic experiment at 20 m altitude was done in the real experiments. They were performed with a tailor-made quadcopter fleet, based on a PixHawk low-level autopilot, running a custom modified version of the ArduCopter code. The authors used 30-drone fleet that took place on a windless, sunny day around Paty, Hungary, moving in a 150 × 150 m arena with 6 m/s according to the more difficult crosswalk target scenario.

3.1.1 Environment

The environment in the V-REP platform is discretized into a square grid with a size 10 × 10. Therefore, there is a specific coordinate for each cell in the scenario. With these data, each robot is able to identify its Cartesian coordinates (x, y) and the Euler angles. These angles specify the robot orientation about the inertial system, describing the orientation of a rigid body and representing the rotations about the axes of a coordinate system (Bonev et al., Reference Bonev, Zlatanov and Gosselin2002).

V-REP platform provides a CAD-3D environment with several features available for the development of a MRS. This environment, shown in Figure 2, was used to implement and simulate the tracking and surrounding the stationary target. In the surround task, the robots move around the environment until the sensors detect an object, which can be a target or an obstacle.

Figure 2 Scenario developed in the V-REP platform with n robots, a target, and walls

The object is considered as a target if the blue color is recognized. Otherwise, it is an obstacle that can be a wall or a robot and should be avoided. It is important to point out that the sensors monitor the environment during the entire execution of the simulation.

The scenario illustrated in Figure 2 is composed of a square grid with size 10 × 10 with n robotic agents r_i(i = 1, …, n), walls, and a target T(x, y). Initially, the target’s position is not known a priori. Also, the robots and the target T are randomly initialized in the scenario.

After the initialization process, if the target T is not detected, the robots start to track. The first robot that detects it must inform its (x, y) coordinates to other robots by message exchange. Finally, the surround task is performed. The important information about the robotic agents and the tracking and surrounding tasks are described in the next sections.

3.1.2 The robot

The robot called BubbleRob (2017), shown in Figure 3, is available in V-REP platform, and it was used in this work. It has a vision sensor (Sensor, Reference Sensor2017b), an ultrasonic sensor (Sensor Reference Sensor2017a), two wheels of diameter 10 and 20 cm shaft, and finally a ball caster in (a), (b), (c), and (d), respectively. The caster is a great third contact point for the robots. It avoids the possibility of having a part of the robot scrape the ground.

Figure 3 The robot created on the V-REP platform

The vision sensor in Figure 3(a) has a range of 56 cm and is able to identify shapes and colors. In turn, the ultrasonic sensor in (b) detects an object in the environment closer than 50 cm without observing details of shape and color, for instance. Therefore, the detection of any object occurs first by the vision sensor, and afterward by the ultrasonic sensor. In this scenario, the vision sensor is responsible for identifying the color of any object in front of it.

The surround task can be performed if the vision sensor identifies the blue color, and the ultrasonic sensor detects the target T. However, if any other color is identified, the ultrasonic sensor should detect the obstacle that must be avoided by the robot.

In the next subsections, the behaviors of tracking, surrounding, and detection of the target T are presented. These behaviors are carried out in parallel with n robots. In order to make the MRS operation more efficient and fault tolerance, the tracking and surrounding tasks are realized in a distributed way.

Tracking, target detection, and obstacle avoidance In the tracking task, the robots in the proposed MRS are programmed to move in a straight line, with a constant speed until the sensors detect the target or an obstacle. Almost the entire environment is covered through this blind search method, which is used until a robot finds the target.

It is worth mentioning that other robots and the walls are detected by the sensors and should be considered obstacles. When a robot detects an obstacle, it rotates approximately 120°. This movement consists of changing the speed of the wheels in the opposite direction and counterclockwise during a time interval, generating an arc trajectory. After that, the blind search continues until the target T is found (Botelho et al., Reference Botelho, Marietto, Mendes, Ferreira and Silva2017).

Figure 4 illustrates the scenario in which the robot should avoid the wall (obstacle). In (a), the robot moves toward the wall. The detection occurs in (b), and a rotation around 120° on its axis occurs in (c) and (d). Also, the distance from the robot to the wall is indicated in (a)–(c).

Figure 4 Tracking and detection (wall) can be observed in (a) and (b), respectively. Wall avoidance happens in (c) and (d)

Target detection After identifying the blue object (target) using the vision sensor, the robot must maintain the straight line until the ultrasonic sensor also detects the target T. The environment in the V-REP is discretized into a grid of rectangular cells, and the coordinates of each cell are made available by the platform. By such representation, in the target detection task, the first robot that detects the target T informs its coordinates to other robots. Also, to maintain the same distance between each robot, the messages exchanged between the robots occur in the surround task, and this communication is described in Section 3.1.4.

The description of the required angles for the robot r_i(i = 1, …, n) to be directed to the target T can be found in Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017).

Surrounding the target The surround task is realized in the form of a regular n-sided polygon figure, with each robot occupying a vertex of that polygon, and the center of the target T is the center of the polygon. The following properties can justify the choice of the regular polygon: (i) its sides and angles are equal and (ii) it can be inscribed in a circumference.

The regular polygon properties allow the surround formation using well-known mathematical relations with asymmetrical form and efficient functionality. The polygon radius is determined by the line segment that joins the center of the target T to the robot’s center of mass. More information about the mathematical model can be found in Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017).

Surround speed control In order to obtain the ideal linear speed of the circular motion of any of the n robots during the execution of the surround task, the ideal proportion of the linear speed of their right wheel in relation to the left wheel should be considered, and it was described in Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017).

After describing the proposed model related to the tracking, target detection, and surround tasks, the next task, described in the next section, is to validate, by simulation, the scenario shown in Figure 2.

3.1.3 Scenario on the V-REP platform

MRS implementation of the tracking and surround task of a stationary target, proposed in Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017), was performed on the V-REP platform, version 3.1.3 Educational 32-bit. This platform was installed on an ASUS notebook, with an Intel Inside Core Processor i5, with 6GB of RAM and Windows 8. Movies available in Mendes et al. (Reference Mendes, Botelho and Marietto2018b) and Mendes et al. (Reference Mendes, Botelho and Marietto2018a) demonstrate the MRS simulation results with three and four robots, respectively.

The scenario implemented in the V-REP platform, shown in Figure 5, is composed of three homogeneous robots (r ₁, r ₂, and r ₃), a stationary target T, and walls. Additionally, robots are considered as obstacles. In this scenario, a MRS with three robots was modeled because this amount of robots is enough to explain the proposed idea without overloading the environment. However, it is possible to add more robots in the scenario illustrated in Figure 2. For this, it is necessary to add pieces of code responsible for identifying the new robot added in the scenario.

Figure 5 Simulation results with three robotic agents (Mendes et al. Reference Mendes, Botelho and Marietto2018b)

3.1.4 Simulation results

The initial position of r ₁, r ₂, r ₃, and T are randomly initialized in the scenario, as shown in Figure 5(a). In (b), r ₁, r ₂, and r ₃ always move in a straight line, until the sensors detect the target T or an obstacle. If an obstacle is found, the robot starts the obstacle avoidance process, as shown in Figure 4. After that, the new rectilinear path directed to another location. At this moment of the simulation, the robots act reactively, because their actions are taken as reactions to the perceptions of the environment.

The robot r ₁ detects the target T in Figure 5(b). After that, the coordinates (x, y) of the target T are sent through a broadcast communication to other robots (r ₂ and r ₃) of the MRS. This communication allows the target T to be found faster by all robots. In this case, the first robot that finds the target T must send its position to the other robots in the environment. In this task, the communication is explicit because the message is exchanged directly between the robots.

The robot r ₃ is directed to the target T, as illustrated in Figure 5(c). In this movement, this robot performs the procedure described in Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017) until it reaches the target T. Meanwhile, r ₂ is still performing the procedure for directing to the T in (c)–(d) of Figure 5. Thus, it is possible to note that, based on the message sent, the robots move in a coordinated way until they find T.

After T is detected by r ₁, r ₂, and r ₃ in Figure 5(e), the surround task, and also the adjustment begins to be performed. The adjustment, described in Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017), aims to guarantee the distance between two robots. The radius R adopted in the surround task was 0,7 m, defined by the difference between the speeds of the right wheels and the left wheels.

The surround task with r ₁, r ₂, and r ₃ starts to become more stable in (f) and (g) of Figure 5. In this scenario, the robots must remain equidistant at 120° from each other. Therefore, according to Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017), the calculation of the ideal distance that each center of the robot should have, with the center of its neighbor, was approximately 1,1 m. In this step, the algorithm explained in Botelho et al. (Reference Botelho, Marietto, Mendes, Ferreira and Silva2017) was used with the message exchange defined in Figure 6, so that the distances between r ₁, r ₂, and r ₃ are symmetrical.

Figure 6 In order to maintain the equidistance in the surround task, the robotic agents send the information of their respective positions

In Figure 6, the robots send their coordinates to ensure the equidistance between them in the surround task. However, Figure 7 shows that only r ₁, which was the first robot to identify T, sends the T coordinates to ensure that all robots move toward the T. Finally, Figure 5(h) shows that r ₁, r ₂, and r ₃ are surround of T. It is possible to notice that the autonomy and the emergent behavior performed through coordinated and synchronized communication allowed the surround task to be fulfilled efficiently.

Figure 7 The r ₁ finds the target and sends its coordinates to the other robots that move toward the target and start the surround task