Experiences of development and first operational methodological and technical demonstrator application
SSM perspective
Background
The ability of the control room operators is paramount to secure safe and stable operations of nuclear power plants. Due to changes in the industry, new operators might not stay as long in each position, nor experience the same amount of operating incidents, as operators did twenty years ago. The last ten years concerns on operators’ future ability to meet challenges have been raised, e.g., the operators’ integrated system knowledge, the technical basis for procedures, the reasons for operational practices and power plant fundamentals.
Integrated System Validation (ISV), based on NUREG 0711, is often used to ascertain that a proposed design meets applicable demands, and to optimize usability. In ISV a proposed design is often evaluated against knowledge of general ergonomic principles but also against the specific prerequisites of the proposed users, in this case the operators.
The Swedish Radiation Safety Authority commissioned this study to inform itself on the applicability and value of utilizing a competency-based assessment approach in the ISV-process, and when training main control room operators in full-scale simulators.
Traditional methods for assessing an operator’s ability typically focus on a collection of specific components of competence, or on specific tasks. A competency-based method could complement traditional methods by mapping competence components to clusters, or competence domains, resulting in a more purposeful training, and in the operators’ ability being more resilient.
Results
Thirteen shift teams performed tasks in a simulator, and were assessed with the tool CADDIE. The tool was adapted to the situation based on thorough task analyses through SAT-methodology complemented with WANO’s Operator Fundamentals. Instructors and operators thought that the tool was valuable and efficient in supporting their ordinary process and has relevance to ISV processes and training processes.
Even though no empirical conclusions concerning ISV applicability could be drawn, due to lack of appropriate projects during the study, the authors believe the results to have clear relevance for ISV as well as the design and assessment of training programs.
Relevance
This study aimed at testing the applicability and value of a competency-based tool for assessing the performance of control room operators during training in full-scale simulators. The results imply the method and tool to be a useful complement in assessing operator performance, both in ISV for proposed design and in the training of control room operators. In utilizing the method studied, and in providing more knowledge on how to train operators to give them a more resilient ability to operate the plant safely.
The study relates to the continued development of regulations and guidelines concerning the competencies of the control room operators. With a solid SAT analysis as a foundation, the competency-based assessment approach opens possibilities for new ways to identify and express assessment criteria as it provides a complementary framework upon earlier assessment options.
In the specific aspect of assessing and evaluating the operators’ ability and level of competence, the CADDIE-tool can be seen as a valuable complement to traditional methods. In part based on that it may be adapted to specific needs, and in part of it is reported to be efficient and easy to use, and perhaps most important because it captures more nuances in the responses from the operators than traditional methods do.
SSM welcomes the development in the ISV area and believe it can contribute to the further development and strengthening of the traditional validation methodology. The study also illustrates the value of conducting research and co-operation between various stakeholders and as part of working with continuous improvements within the framework of safety.
Need for further research
A suggestion for continued research linked to this theme would be studies where the methodology is used in appropriate projects as part of the ISV process. Such research could lead to ecologically more valid scenarios.
Another suggestion for continuing research linked to this theme is further tests of the CADDIE tool as a complementary method in the training of control room operators in full scope simulators and in workplace coaching (APC). The purpose of such studies could be to analyse whether there are differences in how the CADDIE tool works in practice and in a broader context, i.e., a form of validation. A positive result could lead to studies of possible specific contextual conditions that could facilitate a more regular use of the methodology. Also, a study of the effect of applying this complementary method in the training of control room operators in full scope simulators and workplace coaching (APC), could be interesting.