Seaways Free Article: Bridging the gap
New tools revolutionising simulator-based situational awareness training
Alan Bury MNI, Liverpool John Moores University
Abdul Khalique, Liverpool John Moores University
Kamil Kaminski, Liverpool John Moores University
Over the past 18 months, Liverpool John Moores University has been utilising its £2.5 million ship simulation suite to trial various techniques for assessing situational awareness in cadets and junior officers. Of the methods tested, the Situational Awareness Global Assessment Technique (SAGAT) emerged as the most effective tool.
What is SAGAT?
Developed by Mica Endsley in the late 1980s, SAGAT has become a gold standard for measuring situational awareness in complex, dynamic, safety-critical environments such as aircraft cockpits, nuclear power station control rooms and hospital operating theatres. Given its success in these fields, SAGAT shows great potential for achieving similar significance in the maritime sector.
At its core, SAGAT is designed to evaluate situational awareness by temporarily halting a simulation and questioning trainees about their perception, comprehension, and projection of the current situation. This approach provides valuable insights into trainees’ understanding of the operational environment and their ability to anticipate future developments. The technique is grounded in Endsley’s three-level model of situational awareness, which encompasses:
- Perception of relevant environmental elements (level one);
- Comprehension of their significance (level two);
- Projection of their future status (level three).
The first level, perception, involves the basic detection of important information in the environment. In a maritime context, this might include identifying the position of other vessels, recognising changes in weather conditions, or noting alterations in navigational aids.
The second level, comprehension, goes beyond mere perception to understand the significance of this information in relation to one’s goals. For a deck officer, this could involve interpreting the implications of a changing weather pattern or understanding the intentions of nearby vessels based on their movements.
The third level, projection, represents the highest level of situational awareness, where individuals can anticipate future actions or states of the environment. In maritime navigation, this might manifest as predicting potential collision risks or forecasting how current conditions might affect the vessel’s planned route.
SAGAT’s strength lies in its ability to objectively measure these three levels of situational awareness in a systematic and quantifiable manner. By freezing the simulation at predetermined points and asking specific questions related to each level of awareness, SAGAT provides a comprehensive picture of an individual’s situational awareness at critical moments during a scenario.
Application in bridge simulators
Modern bridge simulators are sophisticated systems that provide a controlled environment where a wide array of navigational scenarios can be replicated with remarkable detail, closely mimicking real-world ship operations. The integration of SAGAT into these simulators allows trainers to assess the situational awareness of trainees during complex navigational tasks with unprecedented accuracy and depth.
The SAGAT assessment process begins with careful scenario design. Trainers develop realistic maritime situations incorporating various challenges relevant to the training objectives. These scenarios might include navigating through busy shipping lanes, handling adverse weather conditions, responding to equipment failures or managing emergency situations. The complexity of these scenarios is tailored to match the trainees’ experience level and the specific learning objectives of the training course.
As trainees engage with the simulator, navigating through the designed scenario, the simulation is paused at predetermined intervals. These pause points are strategically placed at moments that are particularly critical or challenging, allowing for assessment of situational awareness when it matters the most. During these pauses, trainees are presented with a series of questions related to their current situational awareness, covering aspects such as the vessel’s status, environmental conditions, surrounding traffic, potential hazards, and projected future states.
The questions posed during SAGAT freezes are carefully crafted to probe each level of situational awareness. For example, a perception-level question might ask about the current position of nearby vessels or the direction of the wind. A comprehension-level question could inquire about the implications of current traffic patterns for the vessel’s planned route. A projection-level question might ask a trainee to predict potential conflicts or hazards in the next 10 to 15 minutes based on the current situation.
Trainees provide answers to these questions, usually through a computerised interface, after which the simulation resumes. This process allows for a comprehensive evaluation of their awareness at critical points throughout the scenario. It is crucial to ensure that SAGAT is seamlessly integrated into the simulation process in order to minimise disruption to the flow of the scenario while still providing valuable assessment data.
Trainers then analyse the responses, comparing them to the actual state of the simulated environment at the time of each freeze. This analysis provides a robust assessment of the trainee’s situational awareness capabilities, highlighting areas of strength and identifying potential gaps in awareness. The data collected through SAGAT can be used to track a trainee’s progress over time, compare performance across different scenarios, and identify common areas of difficulty across multiple trainees.
Benefits for deck officer training
The implementation of SAGAT in simulator-based training has many benefits, significantly enhancing the effectiveness of training courses and, by extension, contributing to improved safety and efficiency in real-world maritime operations.
Foremost among these benefits is that it offers an objective and standardised measure of situational awareness. Unlike subjective assessments, which can be influenced by observer bias or inconsistency, SAGAT provides a quantifiable measure of awareness that can be consistently applied across different training sessions and trainees. This objectivity is crucial in an industry where standardisation and consistency are paramount for safety and efficiency. It allows for the tracking of an individual’s progress over time, meaningful comparisons between trainees, and evaluation of the effectiveness of different training methodologies.
SAGAT’s comprehensive approach, addressing all three levels of situational awareness, offers a holistic evaluation of a trainee’s awareness capabilities. This multi-level assessment provides a nuanced understanding of a trainee’s strengths and weaknesses that might not be apparent through observation alone. A trainee might, for instance, excel at perceiving relevant information in the environment, but struggle with projecting future states. Or they might consistently overlook certain types of information or misinterpret specific cues in the environment. The detailed insights available through SAGAT allows for targeted training interventions focusing on these specific aspects of situational awareness.
The real-time insights provided by the freeze technique allow for assessment of situational awareness as it evolves throughout different phases of navigation tasks. This aspect of SAGAT is particularly valuable in maritime contexts, where situations can rapidly change and evolve. By assessing awareness at multiple points throughout a scenario, SAGAT provides a more complete picture of a trainee’s ability to maintain awareness over time and adapt to changing circumstances.
The data collected through SAGAT assessments serves a dual purpose. Not only does it inform individual trainee development, but it also provides valuable insights for refining training courses as a whole. Instructors can identify trends in situational awareness gaps across trainees, allowing for curriculum adjustments that address common areas of weakness. This data-driven approach to curriculum development ensures that training programmes remain relevant and effective, adapting to the evolving needs of trainees and the maritime industry.
The future of seafarer training
As the maritime industry continues to evolve, SAGAT’s versatility extends to its use in comparing different bridge designs and validating the impact of new technologies on awareness levels. SAGAT provides a reliable method for evaluating how automated systems and the introduction of technologies like Augmented Reality affect situational awareness. This capability is crucial for ensuring that new technologies enhance rather than detract from the deck officer’s ability to perform their role, and that the training officers receive adequately prepares them to work with these new tools.
SAGAT also contributes to the development of metacognitive skills. By regularly reflecting on their situational awareness through SAGAT questions, trainees become more conscious of their own cognitive processes and decision-making strategies. This increased self-awareness can lead to improved self-monitoring and self-correction in real-world situations, enhancing overall performance and safety. Additionally, SAGAT can be used to assess team situational awareness in bridge team scenarios, providing insights into how information is shared and understood among team members. This is particularly relevant in modern maritime operations, where effective teamwork and communication are essential for safe and efficient navigation.
Finally, the objective data provided by SAGAT can be valuable for regulatory compliance and quality assurance in maritime training programmes. It offers concrete evidence of training effectiveness and trainee competence, which can be crucial for meeting industry standards and regulatory requirements.
Challenges and considerations
While SAGAT offers numerous benefits, its use in bridge simulators is not without challenges. Addressing these challenges is crucial for maximising its effectiveness and ensuring its successful integration into a wider range of maritime training programmes.
The scenarios used in the simulator must be designed to reflect realistic and relevant challenges. This requires striking a delicate balance between complexity and learning objectives. Scenarios must be sufficiently challenging to provide meaningful assessment opportunities, yet not so overwhelmingly complex that they confuse or discourage trainees. Moreover, a wide range of scenarios is needed to cover all the situations a deck officer might encounter, from routine operations to rare but critical emergency situations.
Questions must be clear, relevant, and aligned with training objectives, covering all three levels of situational awareness without being leading or overly complex. Crafting questions that accurately probe each level of awareness requires deep understanding of both the SAGAT methodology and the specific context of each particular scenario. Furthermore, questions need to be phrased in a way that minimises the potential for misinterpretation, especially given the time pressure under which trainees must respond during a freeze.
The timing and frequency of pauses for SAGAT questions must be carefully considered to avoid disrupting the flow of the scenario while still capturing critical moments. If interruptions are too frequent, this can break the immersion and realism of the simulation, potentially affecting the trainee’s performance and the validity of the assessment. On the other hand, if freezes are too far apart, trainers might miss crucial points in the scenario where situational awareness is particularly critical. Finding the right balance requires careful planning and often necessitates pilot testing to refine the timing.
The potential for artificiality introduced through pausing the simulation must be acknowledged. It is widely accepted that one of the most significant strengths of a simulator is its immersiveness and SAGAT is intentionally breaking this. The act of freezing the scenario may also influence trainee responses, potentially altering their natural thought processes or decision-making strategies. This factor should be considered when interpreting results, and efforts should be made to minimise its impact, such as ensuring freezes occur at natural transition points in the scenario.
Proper familiarisation with the SAGAT process is essential to ensure trainees understand the purpose and mechanics of the assessment. Without adequate preparation, they may find the freezes disorienting or struggle to respond to questions effectively, potentially skewing the assessment results. Training programmes need to incorporate practice sessions to ensure trainees are comfortable with the process before it is used for formal assessment.
Analysis and interpretation
Developing effective methods for analysing and interpreting SAGAT data in the maritime context is crucial for deriving meaningful insights that can drive improvements in both individual performance and overall training programmes. The volume and complexity of data generated by SAGAT assessments can be substantial, especially when used across multiple scenarios and trainees. Trainers and programme developers will need robust analytical tools and frameworks to make sense of this data and translate it into actionable insights.
While SAGAT is a powerful tool on its own, its effectiveness can be enhanced when used in conjunction with other training tools to create a comprehensive learning experience. This integration requires careful planning to ensure that different assessment and training methods complement rather than conflict with each other.
There are also technical challenges to consider. When automated to its fullest extent, the implementation of SAGAT requires sophisticated simulator systems capable of seamlessly integrating freeze points and question interfaces without compromising the overall simulation experience. This may require upgrades to existing simulator hardware and software, which could be costly and time-consuming.
Finally, there is the challenge of ensuring that improvements in SAGAT performance translate to enhanced situational awareness in real-world maritime operations. While SAGAT provides valuable insights within the simulator environment, validating its predictive power for actual on-the-job performance will require long-term studies and the careful correlation of training outcomes with real-world performance metrics.