1. INTRODUCTION
The instructor of a full mission marine simulator has to set exercises at a suitable level of difficulty for the students, while retaining a high level of behavioural realism. As the students become more adept, this may necessitate increasing the number of vessels and introducing the effects of current and wind. In addition, the instructor must respond to various communications and monitor the students. It is crucial to maintain the realism of the simulation exercise for the key student tasks. If several ships’ bridges are simulated this can become a daunting task.
The Multi-Agent Realm for Investigating Navigators’ Educational Simulators (MARINES) has been designed as an experimental bridge simulator instructor station. The primary aim of the research is to investigate the potential of agent-based systems to assist the instructor of a simulator whilst maintaining the behavioural realism of the simulation. A further objective is to create more realistic motion for the instructor controlled target ships.
1.1. Challenges facing the marine simulator instructor and simulator manufacturer
Chen researched the use of marine simulators in Taiwan ROC; he found that many instructors were intimidated by simulators that were difficult to use and unrealistic (Chen, Reference Chen1992). Guicharrousse has criticised computer simulations that “will indefinitely produce the same results in answer to the same manoeuvres. It is very different from the real world where a manoeuvre will never recur in the same way” (Guicharrousse, Reference Guicharrousse1990). He suggested that students became good at recognising what to do during exercises but that the response might be more difficult to determine in the real world. The Seafarer’s Training Certification and Watchkeeping Code (STCW) states that a training simulator must “have sufficient behavioural realism to allow a candidate to exhibit the skills appropriate to the training objectives” (STCW, 1995). Therefore sophisticated graphics can be less important than realistic behaviour. For example, all the simulated ships should manoeuvre realistically and be affected by the set and drift of the current.
It has been demonstrated that the use of intelligent technology for navigational tasks can be beneficial for pilots and ships’ officers (Grabowski & Sanborn, Reference Grabowski and Sanborn2003). These expert systems provide manoeuvring advice to navigators in specific locations that have high traffic densities (for example, in the St Lawrence Seaway). However, developing a system that reduces the operator’s workload is challenging.
1.2. Sea stabilised motion under the set and drift of a current
The effects of land and sea stabilised displays for electronic navigational aids have been debated in a number of papers. Smeaton et al. discuss the effect on target histories and vectors on an Electronic Chart Display and Information System (ECDIS) (Smeaton et al., Reference Smeaton, Dineley and Tucker1994). Furthermore, after the collision between the ‘Norwegian Dream’ and the Panamanian-flag containership ‘Ever Decent’ in August 1999, one of the recommendations of the enquiry was that “All bridge watchkeepers should be reminded that speed input for an anti-collision plot should always be speed through water, not speed over ground” (AMO, 2000).
However, the effect on a simulator of having a sea stabilised own ship and land stabilised target ships does not seem to have been widely considered. Consider two ten knot ships meeting head on, encountering a three knot current. When the (sea stabilised) own ship (under student control) is travelling with the current, the (land stabilised) target ship appears to be travelling at thirteen knots. When travelling against the current the target will appear to be travelling at seven knots. If both ships were sea stabilised the approach speed would be ten knots in both cases. If a simulated ARPA is being used then the target’s information will be displayed on the own ship bridge and this information may conflict with the visual cues. To a lesser extent the distance and speed errors inherent in ARPA systems should also be simulated (Lenart, Reference Lenart2005).
Consider the effect when a cross current applies as shown in Figure 1. In the example on the left, the sea stabilised own ship is set northwards by the current and is involved in a close quarters situation. A simulated ARPA (Figure 2) should detect that the ship was crossing and display a crossing vector, although from its visual aspect the ship would appear to be passing clear. In the real world, both ships would be affected by the current and be set northwards, passing clear of each other, as shown in the right hand diagram. In the real world an ARPA would display the target ship on a reciprocal course. If the ships are moving relatively slowly compared to the rate of the current, the use of land stabilised target ships will result in manoeuvres that fail to achieve the desired passing distances and confuse rather than instruct the student.
Figure 1. The effect of a cross current in a nearly head-on situation on a simulator.
Figure 2. The radar plot for Figure 1 from the own ship bridge.
1.3. Ship collision avoidance
Several groups have produced automatic collision avoidance systems for ships at sea, e.g. Grabowski (Reference Grabowski1990), Blackwell et al. (Reference Blackwell, Rangachari and Stockel1991)and Smeaton and Coenen (Reference Smeaton and Coenen1990). The MARINES project has investigated the application of this technology to enhance a marine training simulator. On a simulator, it is often left to the instructor to act as the navigator of a target ship in order to avoid a collision. However, in some situations an agent will be able to provide a good solution and ease the instructor’s burden. One or two simulators (e.g. PC-Maritime’s “Officer of the Watch” simulation) have included collision avoidance capability. However, all the target ships are normally controlled by a single, central, collision avoidance engine. Yang et al. have also used a collision avoidance system as a tutor for students (Yang et al., Reference Yang, Phan, Kuo and Lin2001).
2. AGENT ARCHITECTURES
The advantage of using agents in marine simulation has recently been acknowledged by the Royal Netherlands Navy (van Doesburg et al., Reference van Doesburg, Heuvelink and van de Broek2005). Minski (Reference Minski1986)stated that computer agents “produce an effect” and may exhibit some intelligence; many simple agents working together may produce results that are greater than the sum of the individual agent’s capabilities and lead to the formation of a “society of agents” that permit cooperation (Henesey, Reference Henesey2004), communication (Weyns et al., Reference Weyns, Steegmans and Holvoet2004) and coordination (Ricci et al., Reference Ricci, Omicini and Denti2002). Some aspects of this “collective intelligence” are explored in the exercises discussed in section 4 of this paper. MARINES agents are medium to coarse grained agents (Maitre and Laasri, Reference Maitre, Laasri, Demanzeau and Muller1990) with deliberative and reactive tiers, being more complex than neurons in a neural network but simpler than many agents with the ability to perform reflection and complex planning (Sloman and Scheutz, Reference Sloman and Scheutz2002).
3. THE MARINES ARCHITECTURE
The MARINES architecture follows an object-oriented approach, with agents modelled upon real-world behaviour of ship’s officers in the simulation application domain (Roberts & Dessouky, Reference Roberts and Dessouky1998). This use of agent-based simulation can make abstraction simpler (Lee et al. Reference Lee, Lim and Chi2004; Ricci et al. Reference Ricci, Omicini and Denti2002) and produce software that is relatively easy to maintain. Since human navigators may display slightly different characteristics, object-oriented programming techniques, such as inheritance and polymorphism, can be employed to improve realism and software reuse. MARINES is a message passing system, which has advantages for security of the information and also helps to make agents independent from their environment and hence from each other. In addition to a descriptive model of the simulation, functional modelling of the environment has to occur within the agent (Craig, Reference Craig1991). As the environment changes the agent’s environmental model will also need to change dynamically.
MARINES is intended more for research into how agent-based technology may assist simulation than as an aid to agent experimentation. Therefore, the control and feedback is not intended to provide the precise repeatability sought by some agent researchers. Marine navigation simulation is highly interactive – the student operators and the instructor are key components in the simulation, which has to respond in a realistic and timely fashion. In some ways, the lack of precise repeatability here can be seen as a strength; the simulation produces rich environment which reflects real world unpredictability. It is this richness that is considered desirable in some simulation domains. A student should be able to run the same exercise several times and still be unable to guarantee that a particular manoeuvre for a particular ship will result in safe navigation. Rather, the student will need to remain alert and respond to the developing situation.
3.1. The instructor station
The central environment is the process that provides a sub-set of the functionality found in an instructor station on a simulator. A plan view of the scenario is provided, permitting the relative position and motion of the ships to be viewed. Ten computer generated target ships are provided, one super tanker model, one fishing vessel model and eight tanker models. The historical tracks of the ships can be displayed, showing how the ships respond to changing conditions, such as changes in the current and the dangers presented by other ships.
Some controls that are common to many simulator instructor stations are provided: controls to pan and zoom the plan view; start stop, pause and exit the simulation; enter the set and drift of a current; select the computer generated target ships; and a control to display the historical tracks of the ships. Additional controls have been added to assist with experimentation: to set the simulation rate between real time and five times real time permitting experiments to be performed rapidly; to permit the effect of current set and drift to be enabled or disabled for any ship, thus allowing comparisons to be made between sea stabilised and land stabilised motion. A simple 3D view is provided that permits the view from the bridge of a specific ship to be displayed. This process is not an autonomous agent; it simply provides a view from an agent’s perspective.
A software interface is provided for agents to connect to the environment; this permits requests for obtaining information and issuing orders. Examples include, obtaining own ship information, such as speed, heading, position, etc.; information about visible targets, such as range, bearing and aspect; manoeuvring instructions, such as setting the telegraph and the desired auto pilot heading. In this prototype version up to thirty processes can be connected to the environment at any one time.
3.2. MARINES agents
Two general types of agent have been created. One agent type performs collision avoidance between the ships (IRPCS, 1989). The second agent type performs track-keeping, following the desired courses set by the instructor. Each process in MARINES is both a client and a server – messages are sent to the servers by the clients.
In automatic mode, each collision avoidance agent applies rules based upon the IRPCS (1989)in order to avoid approaching vessels. As each agent is an independent entity, the actual rules used can be varied for each instance of an agent. An inference engine uses a forward chaining mechanism to parse a set of prioritised production rules and resulting actions, for example an alteration of course to starboard for a head on encounter. The number of degrees to alter is then determined by a planning component using mathematical functions, or using a ‘best guess’ from the avoidance reactor if the approaching ship is nearby.
Each collision avoidance agent, shown in Figure 3, performs collision avoidance in much the same way as a human navigator. The agent sends the environment a request for visible target ships, a message is returned to the agent for each ship within visible range. The information is stored for a number of iterations. When sufficient information is available, the historical positions are extrapolated to infer whether a dangerous situation is developing. This is based upon the Closest Point of Approach (CPA) and time of Closest Point of Approach (TCPA) of approaching targets. A formula for the danger coefficient (Smeaton & Coenen, Reference Smeaton and Coenen1990) is used to determine which target presents the greatest danger. If such danger is detected then the rules are parsed to determine what, if any, action should be taken to avoid the collision.
Figure 3. The collision avoidance agent.
For each agent controlled ship, a track-keeping agent, shown in Figure 4 co-operates with the relevant collision avoidance agent shown in Figure 3. The waypoints for the desired track are entered by the instructor/experimenter and subsequently these agents adjust the course of the ship to try to keep the ship on the desired track. The set and drift of the current is calculated by the agent based upon a comparison between the dead reckoning position and the actual course made good. Each track-keeping agent also has beliefs about the manoeuvring characteristics of the ship it is controlling, so that it can anticipate when to begin an alteration of course.
Figure 4. The track-keeping agent.
3.3. Adapting to incomplete information
In a worst case scenario the collision avoidance agents in MARINES must make a reactive manoeuvre to avoid a ship that is detected late. This is a simple avoidance manoeuvre if a target ship infringes the domain (IRPCS, 1989) around the agent’s vessel.
A more normal scenario is that a ship may be detected, the information stored, and then a subsequent scan may fail to detect the target. The agent will then keep the previous information until the ship is detected again and then perform the calculations for the time period between several detections of the target. It has been found that storing six historical positions for each target is sufficient to provide a reasonably stable basis for inferring the course and speed of the target.
4. EXPERIMENTAL RESULTS
4.1. Ship model tuning and autopilot adjustment
A set of initial experiments were performed to ensure that the behaviour of the ship models was consistent with their generic type. Limited tuning of the ships’ mathematical models was performed to improve the accuracy of the motion. The ship models were based upon ship trial information from McCallum (Reference McCallum1980)and Pourzanjani (1990), although a simplified mathematical model was employed providing only three degrees of freedom; surge, sway and yaw. This is less accurate than is usually employed for the ship models of the students’ own ship, but more realistic than the simple model that is often used for target ships. Once the ship manoeuvring characteristics had been tuned, the autopilots were adjusted under instructor control. The tuning was done by altering the helm multiplier, counter helm and dead band settings, in the same way as tuning an autopilot aboard a ship.
4.2. Track-keeping agent belief
Once the autopilots were operating well, the beliefs of the track-keeping agents had to be adjusted to deal with the manoeuvring characteristics of the specific ship they were controlling. Included in this was the adjustment of the agents’ knowledge of the manoeuvring characteristics of the vessel. Following this, the track-keeping of the models was tested to compare the behaviour of the different models when controlled by a track-keeping agent. Figure 5 shows how tuning the model and adjusting the track-keeping agents’ beliefs produced acceptable track-keeping capabilities for all three model types that were considered. The expert users who assessed the system agreed that models representing generic ship types are suitable for the simulated target ships and they considered the manoeuvring and track keeping of all the models to be reasonably realistic.
Figure 5. The track-keeping performance of the ships and agents after initial tuning.
4.3. Track-keeping in a current
Track-keeping tests were then performed to compare the track-keeping when a current was introduced as shown in Figure 6. In this scenario both the track-keeping agents are trying to make good an easterly course at ten and a half knots. Ship 2 acts in a similar manner to the target ship on many simulators and is unaffected by current. Ship 3 is affected by a three knot northerly current and the agent corrects the course steered accordingly. Both ships started from the same position but ship 3 was pushed northwards by the current before the agent determined the correct course to steer and is gradually returning to the desired track. Of course, ship 3 actually makes good a slightly slower speed as it is counteracting the current. The 3D view clearly demonstrates the difference of visual aspect that the navigation student will detect.
Figure 6. Track-keeping in a current (Collision avoidance switched off).
4.4. Collision avoidance
Collision avoidance scenarios were set up to determine the collision avoidance agents’ capability in single ship collision situations. This was performed with both track-keeping and collision avoidance agents navigating each target ship, in order to return the ship to course after the manoeuvre. Scenarios were developed for each of the situations determined from the steering and sailing rules. This resulted in approximately fifteen major scenarios for a number of head on, overtaking and crossing situations. These were repeated with different types of ships travelling at different speeds with and without current. Goodwin (Reference Goodwin1975)discusses the sectors to be applied to the collision avoidance problem in more detail. The results of some of these tests are outlined in this section. The ability of the ships to return to track at the correct time was also assessed. The return should be soon enough to remain within the boundaries of the desired track and to avoid unnecessary deviations. However, it should not create another close quarters situation.
Initially, a head on situation was considered, under rule 14 (IRPCS, 1989). This experiment is shown in Figure 7. A number of other near head on situations were also assessed. In an overtaking situation rule 13 applies and the overtaking vessel is required to keep out of the way of the vessel being overtaken until she is finally past and clear. A simple example of this situation is shown in Figure 8. Following this, crossing situations (rule 15 IRPCS, 1989), were considered and an example is shown in Figure 9 with courses at 90 degrees to each other.
Figure 7. A head on situation without current.
Figure 8. An overtaking situation without current.
Figure 9. A crossing situation without current.
4.5. Ambiguous responsibility
One of the more difficult collision avoidance situations for a mariner is detecting which is the give way vessel when approached by a power driven vessel crossing from close to two points abaft the starboard beam. In Figure 10, ship 2 is the give way vessel (Rule 15, IRPCS, 1989). In this case the navigator aboard ship 2 observing ship 0 approaching can make an accurate assessment of whether ship 0 is an overtaking or a crossing ship. However, the navigator aboard the approaching ship 0 may be unable to make an accurate judgement of the aspect of ship 2 and has to assume that ship 0 is an overtaking vessel and should give way. In Figure 10, both agents assumed they had to give way and acted accordingly. However, if ship 0 had actually been the overtaking vessel and had failed to make a manoeuvre then the agent on ship 2 would be in an unenviable situation and should ‘stand on’ following rule 17. In the expert user evaluation (Section 5), the agents in MARINES were considered to take action when they were already too close to the other vessel, although one master mariner did suggest that this is, in fact, a common problem at sea.
Figure 10. Ambiguous responsibility between vessels leads to unnecessary manoeuvring.
Similarly, the agents were found to leave emergency manoeuvring too late when the agent on the give way ship was intentionally prevented from making a manoeuvre. It is relatively easy to increase the distance at which emergency manoeuvring takes place but this can occasionally fool the agent into unnecessary manoeuvres and may be considered as contravention of the collision regulations. In a real situation the navigator of the stand on ship would probably attempt to contact the give way ship by giving a warning using the ship’s whistle and/or an ALDIS signal light before making a manoeuvre himself. This is not currently modelled in MARINES as additional messages would need to be passed between agents on different ships and this would once again raise the complexity. With the current advances in technology it would seem more appropriate now to consider AIS communications (Fisher, Reference Fisher2003) as this would largely remove the ambiguity.
4.6. Introducing a strong current
Scenarios that had initially been performed with no current were re-run with a set and drift from various directions. When a current is introduced into a scenario the risk of collision may change as the agents apply corrections to their courses to maintain the desired track. Therefore, the first scenario shown in Figure 7 that was initially head on involving risk of collision, becomes a crossing scenario when a strong current setting to the north is introduced as shown in Figure 11. It can be seen that ship 3 becomes the give way vessel under rule 15 and the agent now makes a more pronounced alteration of course, while the agent on ship 2 stands on. When a strong current setting to the west is introduced into the initial scenario (Figure 9), the ships pass at a safe distance without manoeuvring, as shown in Figure 12. This demonstrates how the agents respond in a natural way to changes in the environment and thereby improve the realism of the simulation.
Figure 11. In a current, the head on situation from Figure 7 becomes a crossing situation.
Figure 12. The same scenario as Figure 9 with a strong current setting to the west.
4.7. Special Rules
A different rule base was introduced for one agent to show that an agent architecture is flexible and permits common misconceptions to be modelled quite simply. In this case, the agent on the fishing boat was on a reciprocal course, fine on the starboard bow of the super tanker and made a small alteration of course to port, in contravention of the collision regulations, at the same time as the super tanker turned to starboard. This is a typical error encountered at sea and resulted in a close quarters situation, as shown in Figure 13.
Figure 13. An agent with common misconception of the rules.
4.8. More than two ships
Although collision situations involving more than two ships have not been catered for, multiple two ship situations can be dealt with and avoidance actions may alter the future simulation. Exercises containing first four ships and then ten ships were created. An exercise for ten ships is shown in Figure 14; it can be seen that the collision avoidance manoeuvres may prevent several vessels from following their desired tracks. In this exercise ship 2 was forced to stay on the original track beyond the point at which it should have altered course in order to avoid ship 4. Approaching an alteration of course, the agents on ship 3 had to make a turn to starboard to avoid ship 6. Having missed the alteration point ship 3 made its way back to track and was once again overtaking ship 6. However, ship 6 was approaching another alteration and had to maintain the old track until ship 8 had passed. This demonstrates the way in which the agents enhance the simulation experience. It should be noted that this is a prototype and the agents will not solve every manoeuvre in an appropriate manner. For example, the collision avoidance agents are not yet sophisticated enough to consider an alteration of speed in place of an alteration of course.
Figure 14. An exercise involving ten ships.
5. EXPERT USER EVALUATIONS
The results of the experiments have been analysed with the assistance of seven subject experts. Their combined experience included simulator instruction, simulator design, simulator exercise design, marine navigation (four were master mariners) and marine lecturing. These expert users examined both the plots of results and performed experiments themselves. In performing collision avoidance they were given a questionnaire to assess the realism of the simulation when compared to the real world. As discussed in the experiments section some collision avoidance manoeuvres were considered to be less than optimum. For example, it was considered that under certain crossing situations the give way vessel’s manoeuvre was applied too late. However, the findings were generally positive with the straightforward avoidance manoeuvres being handled effectively and the others demonstrating actions commonly seen in the real world (although better manoeuvres might be possible). All the evaluators agreed that this was more realistic than conventional simulators. One instructor preferred simpler training exercises and under these circumstances the agents may be switched off.
6. CONCLUSIONS
The MARINES research has demonstrated that an agent-directed architecture can be used to build a simulator with reasonably realistic target ship motion, including track-keeping and some collision avoidance manoeuvres. This has the potential to benefit both students and instructors. The complex behaviour of the simulation derives, not from complex heuristics approximating to the desired results, but rather from modelling the individual characteristics of simpler processes and letting them interact. In the agent architecture for this application domain, the interaction of simplified real world factors produces a complex simulation. For a number of relatively simple experiments, this has been shown to create a simulation that responds naturally to environmental changes.
The agents are based upon the human navigators found on the bridge of a vessel, rather than on the ships themselves. The modelling is therefore based upon an analysis of the real world. The interaction of relatively simple autonomous agents helps to produce a reasonably realistic, complex simulation. A heuristic model of the observed behaviour of the complete system would be more tightly coupled and arguably more difficult to achieve using conventional monolithic programming techniques.
The agents in MARINES follow normal practice for officers on a bridge watch; the collision avoidance agent takes command of the ship and the track-keeping agent acts as an assistant, sending the collision avoidance agent advisory messages. This hierarchy provides a simple solution to conflict resolution between the agents. If the agents were allowed to be completely independent then when the collision avoidance agent alters course to miss an approaching vessel, the track-keeping agent would immediately apply set to bring the ship back onto track.
The agents receive similar information to a real officer of the watch. For example, the agents store the bearing and range of any approaching target over a period of time and extrapolate the results to determine a plan. Therefore, the reasoning is based on imperfect information, for example, the agent may be unaware of a recent manoeuvre by another ship. The accuracy of the inferred results is affected by natural constraints, the ships yaw slightly in the seaway, preventing a completely accurate assessment of the situation. If one of the ships alters course this calculated information becomes quite inaccurate in a similar way to a vector on a real ARPA. Therefore, when running the same exercise several times there will be subtle differences, as in the real world. If a current is introduced then the agents will adjust accordingly, applying corrections to maintain track. The adjustments may produce different collision avoidance responsibilities and the agents will alter their plans and manoeuvre accordingly.
As in the real world, the accuracy of the track-keeping and collision avoidance depends upon: the manoeuvring characteristics of the ship; the adjustment of the auto-pilot controls; the agent navigator’s perception of the ship’s manoeuvring characteristics; the tightness of the turn; and the presence of other traffic in the area. The resulting track of the ship does not follow nice mathematical curves, nor is the rate of turn of the ship constant. The ship may overshoot or undershoot the desired track and return gradually onto the heading and track. If the set and/or drift of a current are altered at run time, this dynamically affects all the vessels in the simulation and, therefore, their collision avoidance manoeuvres. The 3D view displays a gradual change of aspect as an approaching target ship under agent control responds to the changing current.
It is considered that this prototype demonstrates the potential of an agent-based system for controlling target vessels in a simulation. Even a relatively inexperienced instructor would be able to alter an exercise at run time to match the ability of the students, for example by adjusting the current or altering the desired track of a target ship, without worrying about conflicts with other target vessels. Additionally, because of the loosely coupled nature of agent architectures and the relative simplicity of individual agents, agents may readily be developed that exhibit traits found in real life, thereby increasing the realism of the simulation.
7. FUTURE WORK
While MARINES has successfully prototyped the idea of agent architectures for marine simulation it would be beneficial to construct a complete simulator. This would require a more sophisticated inference engine capable of tackling multiple ship avoidance situations; Smeaton, Tucker and others (Smeaton et al. Reference Smeaton and Coenen1990, Reference Smeaton, Dineley and Tucker1994) have investigated the multiple ship collision avoidance problem for radar sets fitted to small vessels in collaboration with Kelvin Hughes. Additionally, the agents might need to be aware of the depth of water, although for most training exercises carefully determined target routes should suffice, provided the agents were programmed to slow the ship down at a pre-determined point on the track. For multi-ship scenarios, investigation of multi-ship collision avoidance using Meta-Heuristic algorithms (Osman and Kelly, Reference Osman and Kelly1996) appears promising; meta-heuristic algorithms are able to produce a “best so-far” solution at any time. A cost function would have to be developed to evaluate the danger of different avoidance solutions.
It should be fairly easy to create ship models and describe rules to simulate Fast and Manoeuvrable Craft (FMC) (Undrum, Reference Undrum2005) and High-Speed Craft (HSC) (Pike, Reference Pike2005) in order to ascertain their suitability.
A further area of interest is in Automatic Identification Systems (AIS) technology (AMSA, 2004) (Fisher, Reference Fisher2003). AIS reports ground stabilised information rather than sea stabilised. This will often be shown on an ECDIS (Smeaton et al., Reference Smeaton, Dineley and Tucker1994) and will need to be mixed with information from relative plots of vessels from radar/ARPA. As discussed by Moeller (Reference Moeller2002), navigators will need to be trained in the use of this technology. It is believed that an agent architecture would simplify the simulation of ships with AIS, pleasure craft with ‘Class B’ AIS and others without AIS.
