Joint Audit and Evaluation of the Risk-Informed Compliance Verification Processes, Directorate of Nuclear Cycle and Facilities Regulation

Executive Summary

The objective of this joint audit and evaluation engagement was to review the risk-informed compliance verification processes used across the directorates of the Canadian Nuclear Safety Commission (CNSC) in order to:

• provide reasonable assurance that the processes were systematic, well-documented, objective, and incorporated consistency where applicable
• assess the performance of the processes and the effectiveness of their delivery in order to determine the extent to which expected outcomes were being achieved and whether the measurement indicators were sound

This engagement was conducted using a phased approach, by directorate. This first phase covers the Directorate of Nuclear Cycle and Facilities Regulation (DNCFR). As a result, the observations and recommendations in this report are tailored to DNCFR.

Why this is important

The CNSC’s compliance verification activities are in place to ensure that licensees are operating safely, securely, and in compliance with the requirements set out in the Nuclear Safety and Control Act, its associated regulations, licences and certificates. These activities are guided, in part, by DNCFR’s baseline compliance plan, which outlines the compliance verification activities to be completed over a 10-year period. This is validated annually as part of the annual compliance planning process.

It is imperative that the development of the baseline compliance plan, annual compliance plan and other related activities use a risk-informed approach so that key risks are identified, monitored and adequately addressed to ensure that licensees are demonstrating a high level of compliance with the CNSC’s regulatory framework.

Key findings

The engagement found that DNCFR has a comprehensive governance framework in place to ensure adequate oversight of risk-informed planning. Tools and guidance are provided to DNCFR personnel to support risk-informed inspection planning and the conduct of activities, and inspection plans are periodically reviewed to ensure relevancy and accuracy of risks. There is effective collaboration between DNCFR personnel and other operations staff throughout the inspection process. Additionally, the performance of DNCFR’s compliance verification activities is consistently monitored against planned results and reported to management periodically by means of an adequate process to manage the performance data related to activities.

Existing process documentation can be improved by adding more guidance on inspection cancellations and deferrals and by consolidating the documentation, as the information is highly dispersed among a number of documents. The most recent facility risk-ranking exercise documented by DNCFR management lacks a clear rationale to support the assigned rankings; this was previously identified as an area for improvement in a 2022 Office of the Auditor General (OAG) audit. There is an opportunity to explore risk-informed planning techniques deployed in the Directorate of Power Reactor Regulation and other benchmarked organizations that focus more on higher-risk inspections, including the use of more unannounced inspections. Information management can be enhanced by improving consistency in the documentation and by tracking and monitoring inspection-related information, such as lessons learned and licensee document submissions. Performance metrics used for monitoring planned compliance verification activities are generally progress-based and are insufficient to measure effectiveness. There are opportunities to expand the coverage of the indicators and the quality of data collection. It appears that the CNSC’s compliance verification program influences the safety culture of licensees, but licensees have identified opportunities for improvement that are worth exploring further, including in relation to the frequency of inspections and the clarity of regulations.

This engagement includes 5 recommendations aimed at addressing the above-noted areas for improvement (see section 8). Management agrees with the recommendations, and its responses indicate its commitment to taking action.

1. Background

The Canadian Nuclear Safety Commission (CNSC) has a mandate under the Nuclear Safety and Control Act to regulate the development, production and use of nuclear energy for all nuclear facilities and nuclear-related activities in Canada. To support this mandate, the Regulatory Operations Branch (ROB) and the Technical Support Branch (TSB) conduct compliance verification activities, such as inspections, to ensure that CNSC licensees exhibit a high level of compliance with the CNSC regulatory framework.

An inspection refers to a particular compliance verification activity conducted by various directorates in ROB and TSB, whereby information is gathered, analyzed and recorded for the purpose of evaluating whether a licensed activity is in compliance with regulatory requirements. Inspections are either planned or reactive. Reactive inspections can be triggered by desktop reviews, technical assessments or unplanned events, such as the occurrence of rare or unplanned regulated activities. Unannounced inspections are those for which the licensee is not notified in advance, and they may be performed at any time. Reactive and unannounced inspections are conducted based on pre-established criteria, while ensuring that the entities subject to the reactive inspection are aware of the inspection process, criteria and outcomes. For planned inspections, compliance plans are developed annually by each of the ROB directorates in cooperation with subject-matter experts, most of whom are provided by TSB.

The planning process ensures that compliance verification activities are planned in a systematic and risk-informed manner, known as risk-informed compliance verification (RICV). RICV considers key elements, such as regulatory requirements, licensee performance history, organizational considerations and the CNSC’s strategic outcomes.

Compliance plans outline the scope, scheduling, resourcing and time frame(s) for the verification activities to be undertaken for the next compliance cycle for a particular licensee. Compliance activities are guided by directorate-specific baseline plans, which are validated annually. Deviations from the baseline plans may be triggered by a variety of factors, including, but not limited to, inspections that were not completed during the previous year as planned, the recent performance history of the licensee, or significant upcoming changes to the licence program.

In 2016, the OAG examined whether the CNSC had adequately managed its site inspections of Canadian nuclear power plants. One of the 5 audit recommendations was to develop and implement a well-documented planning process for site inspections of nuclear power plants that can demonstrate that the process is systematic and risk-informed. The President of the CNSC subsequently directed all non-nuclear power plant directorates that conduct licensee inspections to also address the OAG audit report recommendations as they related to their inspection programs. As a result, the CNSC refreshed its risk-informed approach to inspection planning between 2016 and 2020. The purpose of this joint audit and evaluation is to determine whether the risk-informed compliance verification processes used across the CNSC directorates are systematic, well-documented, objective, effective and performing as intended.

2. Authority

This joint audit and evaluation is part of the CNSC’s approved Risk-Based Audit and Evaluation Plan for 2023–24 to 2024–25.

3. Objective, scope and approach

The audit objective was to provide reasonable assurance that the risk-informed compliance verification processes used across the CNSC directorates were systematic, well-documented, objective, and incorporated consistency where applicable.

The evaluation objective was to assess the performance of the compliance verification processes and the effectiveness of their delivery in order to determine the extent to which expected outcomes were being achieved and whether the measurement indicators were sound.

The engagement scope covered compliance verification activities in the Directorate of Nuclear Cycle and Facilities Regulation (DNCFR), including processes in place for developing risk-informed inspection plans, conducting risk-informed inspections as planned, and measuring the achievement of intended outcomes. The engagement scope covered the period from fiscal year 2021–22 to 2023–24, excluding the height of the pandemic when procedures were modified.

To complete the engagement, the following methods were used:

• Interviews with relevant stakeholders
• Documentation review
• Onsite observation of an inspection
• Audit testing of completed compliance verification inspections
• Benchmarking against other nuclear regulators and federal government departments
• Survey of DNCFR licensees

4. Statement of conformance

This engagement conforms with the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing and with Canadian Evaluation Society Standards.

5. Acknowledgements

The engagement team would like to acknowledge and thank management and staff for their support throughout the conduct of this audit and evaluation.

6. Findings and Observations

6.1 Governance and Procedural Documentation

A governance framework is a set of rules and practices by which an organization ensures accountability and transparency with all of its stakeholders. Given the various internal and external stakeholders involved in the risk-informed compliance verification processes, having a clearly defined and integrated governance structure is key to ensuring program outcomes are achieved. The engagement expected to find a governance framework in place to ensure that roles and responsibilities are clearly defined and understood, with adequate oversight of risk-informed inspection planning.

Key findings

The engagement found that a comprehensive governance framework is in place to ensure adequate oversight of risk-informed planning.

Documented guidance is in place that makes it possible to distinguish between inspection types and determine when each type is warranted, and it covers key activities of the compliance verification process.

However, there is an opportunity to enhance documented guidance on inspection cancellations and deferrals. There is also an opportunity to consolidate the documentation as guidelines are dispersed among a wide variety of documents, which creates a risk that staff will be unaware of all the guidance available when conceptualizing key processes.

Additionally, a number of procedural documents were last updated in 2018 and 2019, and one document reviewed appeared to be obsolete. It is important to revisit the documentation to ensure that the content is still relevant and accurate.

6.1.1. Responsibility and Accountability

A key component of an effective governance framework is well-defined and documented accountabilities. Roles and responsibilities are defined and documented in more than 10 DNCFR procedural documents. Some documents include responsibility assignment matrices to define team roles across 4 categories: responsible, accountable, consulted and informed. These are in place for various activities throughout the compliance verification process, such as:

• assigning a risk ranking to a facility or activity
• developing the 10-year compliance plan
• developing the annual compliance plan
• conducting compliance activities
• selecting and applying graduated enforcement action
• reporting on licensee compliance

Procedural documents are available for all above-noted activities and incorporate oversight mechanisms, such as management review and approval of the assigned risk rankings, approval of the 10-year compliance plan by DNCFR and the subject-matter expert (SME) directorates, and consultation with the Facility Assessment and Compliance (FAC) team. FAC teams consist of project officers, SMEs and site-specific experts, and they are consulted at regular intervals in the inspection process when developing the facility-specific compliance plan.

There is formal definition of roles and responsibilities between inspectors and technical specialists as per the two-key system. The two-key system ensures that both project officers, as site specialists, and SMEs, as technical specialists, are engaged heavily in making compliance verification decisions, thus giving a variety of perspectives and eyes on files.

Interviews with DNCFR project officers demonstrated that roles and responsibilities are clearly understood by staff. It was also clear that staff understood the key risk-informed compliance verification processes, as their descriptions corroborated what was outlined in the procedural documentation.

6.1.2. Documented Guidance – Completeness

The engagement team assessed whether procedural documentation was comprehensive. Based on the documentation review, it is evident that the various types of inspections are well-defined and documented, and that they include detailed considerations to help determine when a specific compliance activity is warranted. Descriptions, including related triggers, are defined for various compliance-related activities, such as desktop reviews, baseline inspections, reactive inspections, and facility-specific and unannounced inspections.

The assessment determined that there was little to no procedural guidance on inspection cancellations or deferrals, including no examples of when such actions are required, or on the approval process to facilitate the change(s). One procedural document noted that both facility-specific and unscheduled compliance activities “may result in the cancellation or deferral of scheduled facility specific compliance activities that are deemed to be of less safety significance”, yet no additional context was provided. Despite the limited documented guidance on this topic, management reported that there was adequate documentation and approvals on file when cancellations or deferrals occurred.

According to the 2022–23 Q4 Corporate Performance Report, inspection deferrals and cancellations averaged approximately 14% of DNCFR’s total planned inspections prior to the COVID-19 pandemic in fiscal year 2019–20. This increased to approximately 23% during the height of the pandemic and dropped back to 14% in fiscal year 2022–23. The lack of procedural guidance on inspection deferrals or cancellations poses a risk that delays may occur without adequate consideration of relevant risk(s) and/or appropriate oversight.

Although deferrals and cancellations represent a small portion of the total planned inspections, there is an opportunity to develop stronger procedural guidance on this process. This may allow for greater oversight and consistency, effective knowledge management, and efficient resource allocation practices.

6.1.3. Documented Guidance – Ease of Use and Efficiency

Though there is a wealth of guidance published for DNCFR staff, the engagement found that the guidelines for constructing the DNCFR baseline plan are dispersed among a variety of documents throughout the CNSC’s internal document portal, the Navigator. For example, there are separate process documents on risk ranking for facilities, assigning risk rankings for the purpose of determining the number of compliance activities, and developing baseline plans. This creates a risk that staff may be unaware of all the documented guidance available or of how the processes are linked to one another. A lack of consolidated documentation can make it difficult to conceptualize the process and can create challenges when training new staff. This will negatively impact overall efficiency for those seeking information from the guidance. Since there are more than 10 procedural documents on compliance verification activities, it would be helpful to amalgamate and connect them to ensure that the guidance is complete, easily accessible and easy to follow.

It is also noted that many of the procedural documents were last updated in 2018 and 2019, and one document reviewed appeared to be obsolete. It would be beneficial to review the procedural documentation to ensure that the content is relevant and up to date and that it adheres to existing Navigator requirements for maintaining documentation. Management stated in interviews that procedural documentation is reviewed every 5 years; however, there is no evidence of this practice documented.

6.1.4. Conclusion

The risk-informed inspection planning process is supported by a comprehensive governance framework with defined responsibilities, accountabilities and oversight mechanisms. There is a myriad of documented guidance to describe the different types of inspections, when each type is warranted, and key activities throughout the inspection process. There is a risk that DNCFR staff may be unaware of all the documented guidance available or of how the processes are linked to one another owing to the high volume of dispersed documentation.

Recommendation

1. DNCFR should update the process documentation in order to:

1. include more documented guidance and context for inspection cancellations and deferrals
2. consolidate its compliance verification procedural documents
3. ensure that content is current and relevant

6.2 Risk Management

Risk management refers to the identification and evaluation of risks to organizational objectives. The engagement expected to find a systematic process used to identify, evaluate and prioritize risks to support the planning and conduct of compliance verification activities at the CNSC.

Key findings

The engagement found that the most recent facility risk-ranking exercise documented by DNCFR management lacks a clear rationale to support the assigned rankings. This was previously identified as an area for improvement in a 2022 OAG audit. In September 2024, a revision to the process was approved by DNCFR management. The Major Projects and Strategic Support Division (MPSSD) is currently re-examining the original risk ranking conducted as part of the 2019 baseline compliance plan using the revised process. This activity is scheduled to be completed by March 31, 2025.

The benchmarking exercise conducted with other international nuclear regulators and Canadian federal government departments highlights opportunities for improvement in DNCFR with respect to risk identification practices and the inclusion of more unannounced inspections.

6.2.1. Identifying and Ranking Risks

Risk-ranking processes should be documented and accompanied by a clear rationale. The existing process to identify and rank DNCFR licensed facilities is documented in a work instruction with examples of risk considerations. However, the most recent risk-ranking exercise conducted by DNCFR management lacked a supporting rationale to justify the facility risk rankings. Facilities were ranked as high, medium and low. The rationale provided for these rankings was broad, and it was unclear whether the rationale applied to some or all of the facilities and activities listed in the risk-ranking document. In addition, the risk-ranking exercise did not consider financial risks, such as licensee cash flow, which may result in less stringent practices that no longer fully comply with licence conditions and/or that may impact safety. A licensee with a long-standing overdue account with the CNSC can be an indicator as to where a financial risk may exist. It is important to ensure that a clear, documented rationale is available to support the risk rankings and the resulting inspection plan. This promotes transparency and helps to ensure that the risk-informed inspection planning process is guided by well-supported evidence, is applied consistently, and places resources where they are most needed.

The recommendation to consistently document the rationale was made in the 2022 OAG performance audit entitled Management of Low and Intermediate Level Radioactive Waste. The CNSC committed to a management action plan (MAP) to fully and consistently document the rationale for its risk ranking. The MAP was to identify process gaps by March 2023 and complete document revisions by March 2024. Management confirmed that work is underway to update the approach. According to management, the current baseline process is in the midst of being updated to be more granular. Instead of a generic high/medium/low facility risk profile, risk rankings will look at safety control area (SCA) risks specific to each facility.

The MPSSD was established within DNCFR during the engagement and has been tasked with overseeing the revision to this process. In carrying out this revision, there is an opportunity to leverage some of the best practices deployed by other Canadian federal government departments and by international nuclear regulators for identifying and ranking risks. In September 2024, a revision to the process was approved by DNCFR management. MPSSD is currently re-examining the original risk ranking conducted as part of the 2019 baseline compliance plan using the revised process. This activity is scheduled to be completed by March 31, 2025.

The engagement team conducted a benchmarking exercise with other federal departments and international regulators. Through benchmarking, it was found that other federal government departments deploy distinctive models to establish systematic risk-ranking processes in their organizations. Each risk model considers factors that are specific to each of the regulated parties, rather than applying a generic approach (refer to figure 1 for details).

Figure 1 - Risk identification practices of other Canadian federal government departments

• One department has a science-based risk model that drives its inspection plan, focusing on 3 key elements: inherent risk, regulatory control requirements, and the compliance history of the establishment.
• One department has a risk model that assesses the risks or “consequences” for all of the regulated organizations it evaluates, using a tailored approach for each organization. Risk factors include product or service provided, related hazard, and proximity of the potential hazard to people, water, habitat and so on.
• One department generally applies a numeric scale (usually from 1 to 5) to represent the degree of perceived risk within the targeted entity. Some programs use advanced machine-learning algorithms to generate their rankings, while others rely on expert opinion and judgment.
• One department deploys risk profiling to inform inspection prioritization using data collected from previous inspections, licensing applications and international partners. Various factors are taken into consideration when risk profiling a site, with the profiles differing based on factors such as location and compliance history.

Similarly, all international nuclear regulators use a risk-informed inspection process, whereby the level of regulatory attention applied to the licensee is based on multiple factors, such as the level of hazard and risk posed by the facility or activity, and the licensee’s history of safety and security. All of these assessments are underpinned by qualitative and quantitative measures, such as the number and significance of regulatory issues and the number and significance of incidents, gathered through regulatory activities. Additional risk identification practices deployed by international regulators are highlighted in figure 2.

Four inspections in the Nuclear Waste Management Facilities sub-program were deferred from one fiscal year to the next, falling within the 10-year baseline plan. However, this one-year delay may indicate that the 4 inspections were not related to high-risk areas. Therefore, it is worth examining how inspections are prioritized to ensure that resources are assigned to those areas with highest risk.

Figure 2 - Risk identification practices of other international nuclear regulators

• One regulator moved away from reviewing everything on a periodic basis in 2018 to promote the efficient use of resources, especially for low-risk areas. The regulator now uses a risk- and intelligence-based review, consisting of risk-informed sampling to ensure that areas of greater risk and areas of least control are targeted, rather than reviewing everything periodically.
• Two regulators use a sampling approach rather than 100% inspection. This sampling approach is informed by risk and with the underlying premise that the licensee is responsible for controlling hazards and risks associated with its undertakings.
• One regulator uses 3 levels of regulatory attention: routine, enhanced and significantly enhanced.
• One regulator applies the principle of proportionality when determining its actions, so that the scope, conditions and extent of its regulatory actions are commensurate with the human and environmental protection implications involved.
• One regulator stated that regulatory oversight is informed by the complexity and potential harm posed by the licensed activity or facility. This means that those posing lower risk will be subject to less oversight and scrutiny.

There is an opportunity for management to assess the risk-ranking practices identified in the benchmarking exercise to determine whether any of the methods could be tailored for CNSC use, which may support the completion of this action and improve the process.

6.2.2. Reactive Inspections

Reactive inspections are clearly defined in CNSC process documentation, setting out the responsibilities of the lead inspector/project officer and providing examples of relevant triggers. Through interviews with DNCFR personnel, it was noted that the process for adding reactive inspections to the plan consists of a multi-step approach. This approach includes consultation with the monitoring officer to assess the feasibility of adding a new inspection to the existing plan, documenting the rationale for adding the reactive inspection, and obtaining director approval. While these steps are not formally documented, it is evident that the process was well understood by staff.

Similar to the CNSC, other international nuclear regulators and Canadian federal government departments conduct reactive inspections. The benchmarking exercise found that some regularly conduct reactive inspections depending on the risk of the entity and complaints received, while others allocate a percentage of their annual inspection efforts specifically to reactive inspections. One federal government department noted that, when a reactive inspection is triggered by a complaint, it often conducts an unannounced onsite compliance inspection, as it is most likely to get a true assessment of operations.

Approaches to resource allocation varied as well in benchmarking observations. One federal government department indicated that 15% of its plan is allocated to reactive inspections, while one of the international regulators indicated a 5% allocation. Anticipating reactive inspections in advance may reduce the risk of high workloads among staff, while ensuring that sufficient capacity is available for unexpected events as needed. This practice was recently implemented at the CNSC by the Directorate of Power Reactor Regulation (DPRR). According to DPRR personnel, the number of planned baseline inspections was reduced to allow more room and flexibility for reactive work.

6.2.3. Unannounced Inspections

DNCFR has documented guidance in place that defines announced and unannounced inspections, including details on advantages and disadvantages and potential triggers. On the basis of discussions with DNCFR personnel, it was noted that the conduct of unannounced inspections is uncommon, and only 2 were conducted within DNCFR in fiscal year 2023–24. This is due to the possibility that appropriate staff may be unavailable at the licensee site during the unannounced visit to provide relevant information or answer questions as is needed during a regularly scheduled inspection. However, the perception among some DNCFR personnel is that unannounced inspections may allow inspectors to view licensees more organically, without any pre-inspection preparation or coaching. This practice is used in DPRR when inspectors identify areas that may require further attention.

The benchmarking exercise revealed that all benchmarking participants perform unannounced inspections. Two of the 4 departments that conduct unannounced inspections generally do so in response to a trigger, such as a complaint or an issue observed during an inspection. Refer to figure 3 for additional details.

Figure 3 - Unannounced inspections by other Canadian federal government departments and international nuclear regulators

• Two international nuclear regulators perform unannounced inspections in response to a trigger (e.g., whistleblower information).
• One international nuclear regulator conducts unannounced inspections only on nuclear power plants.
• One international nuclear regulator conducts unannounced activities on a routine basis. To improve the efficiency of inspection trips and to reduce agency costs, they may contact the licensee in advance to verify that the inspection can be performed effectively and efficiently at the intended time.
• One Canadian federal department conducts unannounced inspections on an ad-hoc basis if an inspector is in the vicinity of a potential issue. In addition, random inspections that are unannounced are practised by some programs in this department for quality control of the risk-based program.
• One Canadian federal department often conducts unannounced inspections in response to complaints.
• One Canadian federal department noted that unannounced inspections can be done in response to a concern or complaint. The preference is to avoid them in order to ensure that the regulated party has the staff and paperwork available at the time of the inspection.
• One Canadian federal department conducts some unannounced inspections on an infrequent basis.

Benchmarking participants and DPRR personnel noted that unannounced inspections can provide a more complete picture of a situation at a facility; allow the regulator to maintain its authority status by keeping the licensee aware that the regulator can come in at any time; prevent the licensee from managing its work plan according to what was announced; and increase the likelihood of getting a true assessment by foregoing advance notice to the regulated licensee.

Furthermore, unannounced inspections were identified as an opportunity for improvement by the Integrated Regulatory Review Service (IRRS). In 2019, the IRRS conducted a mission in Canada in which they conducted a peer review of Canada’s regulatory framework for nuclear and radiation safety against International Atomic Energy Agency (IAEA) safety standards. The IRRS mission found that no unannounced inspections had been conducted by DNCFR for uranium fuel fabrication, refining and conversion facilities since 2017. This resulted in a suggestion for the CNSC to consider performing unannounced inspections for these facilities. The CNSC further codified how unannounced inspections would be considered as part of the Nuclear Fuel Cycle and Research Reactors Program.

In 2022, CNSC staff conducted a self-assessment (SA)^{Footnote 1} of the corporate inspection process to identify opportunities for improvement and standardize inspection practices across all directorates. In the SA report, it was noted that “further details on the use of unplanned/unannounced and field inspections in the current CNSC inspection process will allow for additional flexibility in the use of these types of inspections”. As a result, the corporate inspection process document was revised to include additional information on the use of unplanned and unannounced inspections. Directorates are now tasked with aligning their directorate procedures with the revised corporate documentation by the end of March 2025. Although the revisions do not include guidance on the frequency of unannounced inspections, directorates have the flexibility to incorporate more specific guidance in their documentation as needed.

The feedback from DNCFR staff, benchmarked organizations, the IRRS and the inspection SA provide an opportunity for DNCFR to consider the value of conducting more unannounced inspections.

6.2.4. Conclusion

The key area for improvement with respect to risk management pertains to the current risk-ranking exercise, which lacks a clear rationale to support the rankings. This may lead to a lack of transparency and corporate memory of compliance verification planning decisions. The lack of a rationale may also hinder knowledge transfer efforts and inhibit optimal resource allocation. The benchmarking exercise may provide information on how to approach this issue.

There is an opportunity to conduct unannounced inspections as a proactive measure to mitigate risks that may otherwise go unnoticed.

Recommendation

2. DNCFR should conduct an analysis of the current risk-informed compliance verification planning process to identify how it can leverage best practices used in other organizations and within the CNSC, including, but not limited to:

1. implementing the use of a risk- and intelligence-based planning approach to focus more on higher risk inspections, which promotes increased resource and outcome efficiency
2. increasing the use of unannounced inspections to strengthen and complement existing compliance verification practices

6.3 Planning and Conduct of Inspections

The engagement expected to find mechanisms in place, including periodic reviews of the inspection plan and documented guidance to support staff, in order to ensure that the planning and conduct of inspections are consistent with the associated level of risk.

Key findings

The engagement found that staff are well versed in DNCFR work instructions and processes supporting risk-informed compliance verification planning and conduct. Tools and guidance are provided to DNCFR personnel to support risk-informed inspection planning and the conduct of activities. Staff reported that these tools are adequate to meet their needs.

Through audit testing, it was confirmed that DNCFR completed inspections that were commensurate with the risks identified in its risk-ranking exercise and were carried out in accordance with established procedures.

Inspection plans are periodically reviewed to ensure relevancy and accuracy of risks. Review processes are also in place for modifying inspection plans, and these processes are understood and consistently used across the directorate. However, there is a lack of clarity pertaining to the midpoint review of the DNCFR 10-year baseline plan. The revised process that was approved by DNCFR management in September 2024 clarifies the purpose of the midpoint review as a means to update risk criteria and also re-validate the 10-year baseline plan.

6.3.1. Planning Risk-Informed Inspections

The documentation review demonstrated that process documentation was in place to support DNCFR risk-informed inspection planning. The documentation included guidance on key activities, such as establishing the preliminary baseline plan, which is used to develop the facility-specific baseline plan, and validating the plan with the FAC team. The work instructions and processes mandated that licensee performance is to be considered during baseline and annual planning of compliance verification activities. Additional details are illustrated in Appendix B.

There were some aspects of the process that lacked detailed guidance and standardization. There was no requirement to justify the inclusion of certain compliance verification criteria over others when constructing the compliance matrix. As well, there was limited guidance for project officers when determining what information to consider in preparing for upcoming inspections. There was a step in the inspection process that instructed inspectors to review lessons learned from previous inspections. However, the engagement team found that there was no standard format across DNCFR for communicating and reviewing lessons learned. As a result, inspectors use their own judgment on what to review. Additionally, through the review of a sample of completed inspections, it was found that only one inspection reviewed had the lessons learned tab populated in the Case Management System (CMS).^{Footnote 2} There was no indication that the omission of this step led to any risk.

A discussion with management revealed that MPSSD plans to implement a lessons learned database. Implementation of this tool may support inspectors when planning future inspections. Lessons learned was also identified as an opportunity for improvement in the 2022 SA, which found that findings from lessons learned are not consistently communicated to inspectors. As a result, there is an action underway to ensure that lessons learned from inspections are tracked and shared with appropriate audiences following a graded approach.^{Footnote 3} As of July 2024, inspection community events, such as the quarterly inspector information sessions and the Inspector Forum, have been used to share lessons learned, and the corporate inspection process document was revised to include guidance for sharing lessons learned. In addition, a working group was established to carry out the actions from the SA, and one of the focus areas is to identify any new lessons learned noted by directorates that can improve corporate guidance.

A lack of detailed guidance and a lack of standardization of best practices could pose a significant risk in terms of knowledge transfer and quality assurance. Should project officers or inspectors with institutional knowledge change roles or leave the CNSC, there is a risk that inadequate compliance verification planning decisions could result from a lack of information and effective knowledge transfer.

In examining this issue through the benchmarking exercise, the results demonstrated that all international regulators include risk as a key component of their planning. Similar to the CNSC, the regulators prepare a long-term plan and corresponding annual plan. The long-term plan sets out the entire suite of activities to be performed over a multi-year period, with the annual plan setting out in detail the activities to be performed each year.

6.3.2. Conducting Risk-Informed Inspections

The engagement team used non-statistical sampling to select a sample of DNCFR-completed inspections from fiscal year 2023–2024. The sample consisted of a range of inspections covering both high and medium risk facilities, varied SCA coverage, and different inspection types (e.g., baseline, facility-specific, reactive). The purpose of the testing was to determine whether DNCFR inspections were consistent with the key risks identified in the baseline risk-ranking exercise, and whether the inspections were conducted in accordance with DNCFR procedures. Many of the completed inspections from the testing sample aligned with the risk profile established through baseline planning.

The audit testing also demonstrated that inspections were conducted in accordance with DNCFR inspection procedures and the documented workflow steps in the CMS.

Key documentation, such as kick-off meeting agendas, was consistently created and saved in CMS. The rationale for reactive inspections was consistently documented and maintained by the monitoring officer. In addition, action items stemming from inspection reports were consistently entered and tracked separately in the Regulatory Information Bank (RIB). The engagement found inspection documentation to be, in most cases, accessible, reliable, accurate and complete.

However, a few of the inspections had not been properly closed in CMS and were still listed as open. This may lead to inconsistent or inaccurate performance reporting, which considers the number of business days taken to submit the inspection report to the licensee.

6.3.3. Supporting Templates and Guidance

Templates to support the conduct of inspections are also established within DNCFR. Templates include such documents as licensee notification letters, inspection agendas, compliance matrices and inspection reports. The templates are all saved in a centralized location accessible to DNCFR staff. The consensus among the project officers and inspectors interviewed was that these templates are helpful and useful.

The engagement team attended one inspection at a DNCFR facility in February 2024. The purpose of the visit was to observe the inspection process in real time, while taking note of any audit- or evaluation-related considerations, such as the use of tools and templates by staff during inspections. During the inspection observed by engagement team members, the inspectors used the compliance matrix template onsite to guide their inspection, as well as the preliminary facts and findings report template to debrief the licensee at the end of the inspection. Throughout the inspection process, the engagement team did not observe any discrepancies from the documented inspection processes or procedures. The tools and templates were used as intended.

6.3.4. Review and Approval Processes

Inspection plans undergo a review every fall as part of the annual compliance planning process, which allows project officers to validate the baseline plan and make necessary adjustments. The annual review comprises key steps, such as consulting key stakeholders to identify major activities for the upcoming fiscal year and to assess alignment with the risk profile. Baseline plans are updated every 10 years to ensure the continued accuracy of facility-specific risks. At the time of the engagement, the current 10-year plan was being revisited by DNCFR for its mid-cycle review to assess its existing approach. In discussing with management, it was noted that the mid-cycle review is not formalized. It was also unclear what the mid-cycle review consisted of, the approach taken, and its progress to date. There is an opportunity to formalize the mid-cycle review, to ensure that a clear objective is defined, while adding value and minimizing any duplicate efforts with the annual planning exercise. Baseline risk assessments are the foundation of compliance verification plans at DNCFR. A formalized periodic validation of the baseline risk assessments would help ensure that the baseline plan keeps up with licensee and industry changes.

According to DNCFR personnel, if there is a desire to deviate from the annual plan mid-year, either by deferring, canceling or adding a reactive inspection, the project officer contacts the monitoring officer in writing to request the change. The project officer provides relevant information on feasibility and a rationale for the change, and the relevant director is tasked with approving the change. If accepted, the change is then recorded by the monitoring officer in the tracking sheet with the rationale included. Interviews with DNCFR staff demonstrate that the process to change the annual plan is understood across the directorate. Documentation in the form of 2 examples was provided by DNCFR directors to corroborate this process description. In both instances, a memo was sent by the director to the director general requesting approval of the change. It is unclear whether this process is formalized. Formalization of review processes for modifying inspection plans could help ensure consistency in approach.

6.3.5. Conclusion

The engagement found that DNCFR staff are well-informed of the documented guidance in place to support risk-informed compliance verification planning and conduct. Templates and guidance are provided to DNCFR staff to support risk-informed inspection activities, and evidence demonstrates that they are used to complete inspections. Additionally, DNCFR‑completed inspections are commensurate with the risks identified in the DNCFR risk-ranking exercise and were completed in accordance with DNCFR procedures.

Although process documentation is in place, decisions on risk-ranking and prioritization of inspections are primarily based on discretion. Without an accompanying documented rationale and procedures for effective knowledge transfer, it is difficult to assess decisions retrospectively and learn from experience. Furthermore, not having this information may negatively impact public trust or lead to increased scrutiny from key stakeholders such as the Commission, external auditors or the general public.

Recommendation

DNCFR should ensure that lessons learned from inspections are consistently documented in the appropriate information management system and that a process is developed to regularly analyze knowledge and share it with other stakeholders as appropriate.

6.4 Operations, Information and Knowledge Management

Operations management is the administration of business practices to build efficiency within an organization.^{Footnote 4} Information management^{Footnote 5} and knowledge^{Footnote 6} management serve to identify, collect, store and disseminate information to harness collective knowledge within an organization. The engagement expected to see a formal approach to information and knowledge management to ensure the effectiveness of compliance inspection activity and business continuity. This includes the consistent use of information management (IM) tools that deliver accessible, reliable and accurate information, and effective collaboration within the directorate.

Key findings

While corporate IM systems are used fairly consistently, there are opportunities to improve the ease of use, consistency in application and availability of information, and to better integrate current IM tools to reduce duplicative efforts.

Formally tracking important licensee document submission requirements may further strengthen compliance activities and collaboration among staff by ensuring that stakeholders are receiving key information in a timely manner.

There is an opportunity to leverage IM practices deployed in other organizations where the use of IM tools is centralized and integrated and enhances overall efficiency.

There is effective collaboration between DNCFR personnel and other operations staff throughout the inspection process.

6.4.1. Information Management Tools

DNCFR personnel use a variety of digital tools for the purposes of information management. Figure 4 identifies the main tools used across the CNSC and provides a description of general use within DNCFR. Some of these tools, such as the CMS and the RIB, are more rigidly designed and provide an audit trail. Other tools, such as the Microsoft suite, are flexible and fit for purpose.

Figure 4 - IT tools for DNCFR compliance verification activities

IT tool	Use by DNCFR staff
Case Management System	Tracks inspection workflow
Regulatory Information Bank	Tracks actions stemming from inspection reports, such as notices of non-compliance
E-Access	Used as a filing repository to securely organize and manage work documents
Microsoft Excel	Used internally by inspectors or divisions to keep track of details in their own files
Microsoft Word	Used to write key documents, such as reports
Microsoft Power BI	Used by the monitoring officer to track progress against the inspection plan and as an interactive dashboard for reporting

Interviews with DNCFR personnel highlighted a consensus among staff that the CMS is cumbersome and not user-friendly, which is consistent with audit testing results in this area. As a result, staff reported reverting to other tools that are simpler to use, such as Excel spreadsheets. However, the use of spreadsheets lacks standardization and could increase the risk of incomplete documentation. Other challenges were noted that stem from the use of multiple platforms that are not fully integrated with one another. The engagement team heard that a lack of standardization over the years has led to inefficiencies in how data is managed and processed. Furthermore, interviews indicated a duplication of effort from uploading information to both CMS and e-Access. It was also reported that tracking data across numerous platforms has occasionally resulted in a duplication of effort.

Additionally, CMS was used inconsistently in conjunction with e-Access, which created difficulties in finding and accessing certain documents. Key inspection documents that were meant to be saved directly in CMS were not, but were later retrieved in e-Access. This is a risk that impacts accessibility and reliability of data. It is worth noting that DNCFR directors identified the consistent use of IM tools as a priority.

Similarly, the integration of compliance verification information was identified as a recommendation in 2022 in the OAG audit. The MAP included a commitment to establish data management practices, including a data governance framework to support integrated management of compliance data, as well as a review of existing tools to identify potential areas for improvement. The MAP also included a long-term action to assess an integrated system approach to accurately capture and manage the information, understand the effectiveness of the compliance processes, and improve them as required. The expected completion dates are December 2024 for reviewing existing tools and fiscal year 2024–25 for assessing an integrated approach for information management.

The engagement team also noted some coordination issues in a related Internal Audit, Ethics and Evaluation Division (IAEED) case study of a DNCFR licensee. In the case study, the licensee failed to submit a key document for CNSC specialist review. The omission was not caught by the CNSC project officer, who is ultimately responsible for monitoring licensee submissions, or by the CNSC technical specialist. IAEED follow-up noted that for each technical area, there are many different technical documents to track in terms of licensee submissions. To the IAEED’s knowledge, there is no standard way in which to track these submissions as they are not part of any of the IM systems used. With the many different tasks that project officers are required to perform, there is a risk that document submissions could be missed or fail to be routed through the review process. Corroborating this observation, DNCFR specialists interviewed noted that project officers sometimes miss key licensee submissions or forget to request specialist review when it is required. This can lead to either incomplete information being provided or specialists working at the last minute to conduct a review. Specialists have reported mitigating this problem by tracking deadlines themselves, which is duplicating efforts.

In the benchmarking exercise, the engagement team made note of one Canadian federal department’s approach to this issue. Its IM system has document tracking built into its workflow. Once the regulated company submits documentation, a notification is sent to the employee responsible for the file. This functionality ensures that key document submissions are received by appropriate staff and do not get overlooked.

There is an opportunity to use a standardized tool and develop a standardized process to track important licensee submission documents and reminders for SME review. This may help ensure that key compliance documentation is received and distributed to the appropriate staff, and that the resulting task, such as a specialist review, is completed on time and not missed. In addition to fostering greater collaboration among the parties involved, this may mitigate the risk that the CNSC is not completing its compliance verification activities in an effective and timely manner. It is expected that the completion of the above-noted MAP, and the CNSC Digital Strategy, currently in its implementation phase, will help address these issues. Specifically, the introduction of new and integrated tools from SharePoint should streamline information management and data processing going forward. The anticipated SharePoint implementation date for ROB is March 2025.

The benchmarking exercise identified 4 organizations with an efficient and integrated software system for IM purposes (refer to figure 5 below). These benchmarking findings are timely for the CNSC, as the organization is currently implementing Horizon 1 of its Digital Strategy, which aims to enable an agile digital workplace. This includes the delivery of new task management, document creation and workflow automation capabilities.

Figure 5 - Information management by other international nuclear regulators and Canadian federal government departments

• One international nuclear regulator developed its information system as part of a strategy to digitize key tasks, increase the accessibility and visibility of information, and enhance the efficiency and effectiveness of its regulatory team. Inspection data in this system was controlled and aggregated in one place so that inspectors could see trends in compliance. There was also a secure link where licensees could seamlessly upload files for inspectors.
• One international nuclear regulator developed an in-house digital tool to plan, conduct, report on and document its inspection activities. It provided a web-based online tool for concurrent input by multiple inspectors working on a team inspection.
• One Canadian federal department built its information system using an integrated, modularized workflow approach for several activities, including, but not limited to, compliance verification, remediation and other enforcement activities. The information system supported planning, conditional oversight and data analysis.
• One Canadian federal department developed an in-house information management tool that supports risk assessment activities, compliance and related information tracking.

6.4.2. Collaboration for Knowledge sharing

Effective collaboration is in place between DNCFR personnel and other operations staff, such as technical specialists. Interviews with DNCFR personnel demonstrated that collaboration among staff and between different operational areas was effective, positive and productive. This is also evidenced by the number of forums established to support information sharing and collaboration. For example, FAC team consultations take place at regular intervals in the inspection process to obtain feedback from SMEs and site-specific experts. FAC teams were highlighted as a best practice at the CNSC, as they leverage diversity of experiences to arrive at better decisions and regulatory outcomes. The Inspector’s Forum provides a formal space in which inspectors can exchange best practices and lessons learned. In addition, the Operations Management Committee, which is also a key governance body, provides briefings on key projects and initiatives that involve operations staff. Additionally, collaboration takes place through informal communication channels, such as email, instant messaging and informal conversations, which are important means for collaboration.

Some staff interviewed noted that there were best practices from other divisions and directorates that could be considered by DNCFR. For instance, the use of PowerPoint decks at divisional meetings to identify lessons learned and the use of an auto-populating inspection matrix template were 2 suggestions from project officers and SMEs to improve efficiency and effectiveness.

6.4.3. Conclusion

Overall, the use of IM tools across DNCFR is generally consistent, and information, for the most part, is accessible, reliable and accurate. Collaboration among DNCFR and key stakeholders is effective, and multiple channels are established to foster these connections.

Some duplicative efforts and documentation inconsistencies were identified, particularly with information in CMS and the use of e-Access. If IM tools are not used in a consistent manner and information is spread across a variety of platforms, inefficiencies or errors in the compliance verification process may result. This can lead to more duplicative efforts, leaving less time for other compliance verification activities, or compliance verification activities being missed without the use of a controlled process.

It would be advantageous for the CNSC to consider the information management best practices identified in this report and the feedback from DNCFR staff when completing the existing MAP and implementing the CNSC Digital Strategy initiatives.

Recommendation

3. DNCFR should ensure that licensee submissions are consistently and effectively tracked, leveraging available tools.

6.5 Effectiveness – Performance Measurement, Monitoring and Reporting

Performance measurement is a systematic approach used to assess the efficiency and effectiveness of projects, programs and initiatives in an organization to ensure that desired outcomes are achieved.^{Footnote 7} The engagement expected to find appropriate performance measures linked to planned results that are continuously monitored and periodically reported on.

Key findings

The engagement found that the performance of DNCFR’s compliance verification activities is consistently monitored against planned results and periodically reported to management and an adequate process to manage the performance data related to activities.

DNCFR generally has a high completion rate of the inspections it plans but could improve its rate of meeting service standards.

However, performance metrics used for monitoring planned compliance verification activities are insufficient to measure effectiveness. There are opportunities to expand the coverage of the indicators and the quality of data collection.

It appears that the CNSC’s compliance verification program influences the safety culture of licensees, and a majority of licensees mostly agree that regulatory requirements for nuclear fuel cycle activities are clear. There are opportunities for improvement identified by licensees that are worth exploring further, such as the frequency of inspections.

6.5.1. Evidence of Performance Metrics

The documentation review and interviews demonstrated that an adequate process is in place to store, collect and analyze DNCFR performance data related to activities. This process tracks activities throughout the year and takes a collaborative approach, involving the DNCFR monitoring officer and stakeholders from the Strategic Planning Directorate (SPD), which is situated within the Regulatory Affairs Branch. It is evident from stakeholder interviews that this process is well understood by staff. The performance of DNCFR’s compliance verification activities is consistently documented, monitored against the plan and reported to the Management Committee.

Three of the 4 performance metrics for planned compliance verification activities focus entirely on the completion status of planned inspections within each directorate. The metrics consist of inspections in progress, deferred, cancelled and completed. This information is presented to the Management Committee every quarter and includes a combined total of the completed reactive and unannounced inspections each year. It would be beneficial to categorize these totals separately for greater clarity and planning purposes. The fourth performance metric tracks the percentage of inspection reports issued against the service standard timeline of 60 business days.

By focusing solely on the internal activities and outputs of the program, there is a risk that potential issues pertaining to inspection effectiveness, risk coverage or other operational matters may be overlooked.

Stakeholder perceptions on the validity and usefulness of the current performance indicators indicate that these indicators are useful for decision making in terms of adjusting the timing of the inspection plan. It was also confirmed that there are few metrics currently available to evaluate the performance of compliance verification activities. As a result, stakeholders may encounter limitations when attempting to use the data for planning activities or strategic decision making. This was elaborated on by one key stakeholder, who suggested that the existing indicators should be improved to be more risk-oriented and that other indicators could be tracked to reflect licensee performance. At the time of the engagement, there was no indication that such efforts would be made to enhance the metrics.

Having more performance indicators can help ensure completeness of information when making risk-informed compliance verification planning decisions. Decisions on risk assessment and prioritization of inspections are ultimately made by management with the support of CNSC staff. Much of the data used to inform these decisions originates from the licensee (e.g., annual reporting, event reports), who is ultimately responsible for the safety of its sites. With this approach, however, there is a residual risk that licensee reporting may not show underlying compliance issues.

The engagement team examined a case study that illustrates this point. In this case study, an event was reported to the CNSC that indicated systemic non-compliances had occurred at the licensee site. The non-compliances did not pose significant risk to public safety or the environment but did pose a risk to worker safety. This issue was not reflected in licensee reporting and was not caught by DNCFR through routine inspections and compliance verification. In the 2 years leading up to the event, there were safety near misses at this site that may have indicated an issue with the licensee’s safety culture. These safety near misses were reported individually to the CNSC. Formally tracking and monitoring performance trends could help give CNSC staff a more holistic view of licensees to help inform compliance planning.

Flexibility and discretion are important in making compliance verification planning decisions; however, enhancing the indicators and metrics tracked to include more insights on risk and licensee performance could help complement decision-maker discretion and provide evidence for compliance decisions. Having an evidence-based rationale is an effective knowledge management practice. Moreover, not having this information may negatively impact public trust or lead to increased scrutiny from key stakeholders such as the Commission, external auditors or the general public.

The benchmarking exercise conducted with 4 international nuclear regulators revealed some good practices pertaining to performance measurement. Three regulators have performance measurement frameworks that are outcome-based and grounded in licensee performance. Two regulators assess overall licensee performance quarterly, with a detailed assessment in the fourth quarter that assesses performance over the previous 12 months. The performance results are reported publicly. Another regulator has established performance indicators that directly link back to its strategy and are reviewed quarterly to determine whether the organization is ensuring that licensees are achieving routine compliance. Figure 6 provides an example.

Figure 6 - Performance measurement findings from international nuclear regulators

• One regulator defined performance indicators that linked back to its corporate plan using 5 strategic themes, with compliance inspection activities falling under the theme of “Influencing Proportionate Improvements”. There were 4 performance measures under this theme:
  • Regulatory activity drives demonstrable improvements and compliance across purposes.
  • Regulatory decisions are proportionate, balanced and unbiased.
  • A continuous self-improvement and learning culture.
  • Innovators are confident to test and deploy technology in a regulatory framework that embraces innovation.
• Performance was reported publicly in an annual report.

Additionally, one regulator identified risk-oriented key performance indicators covering a range of areas, captured in figure 7 below.

Figure 7 - Risk-oriented key performance metrics identified by one international nuclear regulator

• Number of significant events/incidents
• Number of findings above a “low safety and security significance”
• Number of deviations between the situation observed during the inspection and the regulations
• Number of anomalies or aspects warranting additional justifications
• Days lost due to work-related accidents
• Number or rate of safety rule errors
• Number of nuclear security incidents
• Number of regulatory issues

There is an opportunity to expand the breadth of the performance metrics to gain a better understanding of licensee performance and how key risks impact the division’s ability to achieve its mandate. This information may help inform future inspection planning and decision making. It may also be worthwhile to consider the performance measures deployed by other nuclear regulators to identify potential areas for improvement.

6.5.2. Achievement of Performance Activity Measures

Overall, DNCFR’s planned compliance verification activities are being achieved and measured, as demonstrated in the CNSC’s Corporate Performance Report. DNCFR completed 83% of its planned inspections in fiscal year 2023–24. The completion rate varied, with two sub-programs achieving at least 90%. There is no requirement to reschedule a deferred baseline inspection within a certain time frame so long as it is conducted within DNCFR’s 10-year baseline timeline and approved by the director. A breakdown by sub-program is captured in figure 8.

Figure 8 - Percentage of DNCFR inspection plan completion at Q4

Sub-program	Nuclear Processing Facilities	Uranium Mines and Mills	Nuclear Waste Management Facilities	Research Reactors	DNCFR total
Division	NPFD	UMMD	CNLRPD & WDD	CNLRPD & NPFD	-
2023–24	97%	90%	73%	68%	83%

The completion rate of 73% in 2023–24 for the Nuclear Waste Management Facilities sub-program was caused by 4 deferrals stemming from various factors, some of which were outside DNCFR’s control, such as waiting for specific regulatory document implementation, staffing changes, and incomplete remediation by the licensee. Similarly, the completion rate of 68% in the Research Reactors sub-program was caused by 4 deferrals and 3 cancellations resulting from factors such as weather and ability to access exterior features, waiting for specific regulatory document implementation, and cancellations.

Based on interviews and the document review, it is evident that the annual planning process helps to estimate the level of effort required to carry out compliance verification activities. Most divisions expressed that the workload from compliance verification activities is high, but achievable overall. On the other hand, the Wastes and Decommissioning Division (WDD) highlighted challenges regarding its high workload and staff capacity to achieve its competing priorities. According to WDD staff, the workload challenges stem from competing compliance and licensing priorities, the lack of trained inspectors, time spent training inspectors, and performing split duties as an inspector and SME. This was illustrated in the performance data, as available at the time of the engagement in 2023–24 (see figure 9 below).

Figure 9 - Number of DNCFR inspection reports issued in fiscal year 2022–2023

Division	WDD	CNLRPD	NPFD	UMMD	DNCFR total
Service standard timeline met	7	11	11	22	51
Service standard timeline unmet	7	9	9	7	32
Total reports issued	14	20	20	29	83
Percentage of reports that met service standard timeline	50%	55%	55%	76%	61.45%

In fiscal year 2022–2023, WDD issued 50% of its inspection reports within the service standard timeline of 60 business days. This service standard has a directorate target of 80%. The results were similar in the Canadian Nuclear Laboratories Regulatory Program Division (CNLRPD) and the Nuclear Processing Facilities Division (NPFD), both of which achieved 55% in attaining service standards. The rationales for exceeding the service standard timeline varied, with some pointing to administrative resource challenges, a limited number of inspectors, limited SME availability, and a significant spike in Commission-facing duties. In interviews with other DNCFR divisions, some staff noted that measures were being implemented to address high workloads, such as deferring an inspection if insufficient staff were available, or combining inspections where possible. There is an opportunity to assess whether such practices are feasible to address the workload challenges in WDD and whether contingency and resourcing is built into the division’s annual workplan. These supplemental efforts may help ensure that the workload distribution is more achievable for all divisions.

In addition, the engagement team observed a high number of position changes at DNCFR since the start of the engagement. For example, as of May 28, 2024, the director position had turned over in 3 of the 5 divisions, a new director general was appointed, and the Vice President of ROB was acting in the position. Additionally, of the 69 staff in DNCFR, 18 were acting, on assignment or on leave, representing more than one-quarter of DNCFR staff. While internal staff transfers may foster professional development, there is also a risk that such movement may contribute to workload challenges. Therefore, it is important to consider this when completing the annual planning exercise.

6.5.3. Program Effectiveness

The engagement found that most DNCFR licensees have a positive perception of the CNSC’s compliance verification program and consider it effective overall. Most DNCFR licensees view the CNSC’s inspections as somewhat or very influential to the safety culture of their organization and are of the opinion that inspections add value overall. Interviews also support this finding. Some opportunities for improvement were identified by DNCFR licensees in terms of the frequency and consistency of inspections.

The engagement team issued a survey to 25 DNCFR licensees (respondents) in April 2024 to evaluate the effectiveness of the CNSC’s compliance verification program. There was a response rate of 76% among those surveyed. The survey explored perceptions on various topics, including how regulated parties have adopted a safety culture, the usefulness and impact of compliance verification activities, best practices and areas for improvement, and factors facilitating or hindering achievement of inspection outcomes.

Most respondents^{Footnote 8} indicated that the CNSC’s compliance inspection activities are somewhat or very influential to their organization’s safety culture. A few respondents indicated that the CNSC’s feedback is highly valued by their organization, which has motivated them to implement required orders and recommendations immediately.

Most of the licensees surveyed indicated that the CNSC’s compliance verification activities demonstrate equal accessibility to all groups in their organization based on location, language, ethnicity, sex, gender, physical or intellectual ability, and/or other identify factors. This feedback is consistent with internal CNSC staff sentiments from the 2022–23 Public Service Employee Survey results on diversity and inclusion and aligns with the CNSC’s ongoing efforts to create an equitable, diverse and inclusive environment. In addition, the Human Resources Directorate actively monitors Gender-based Analysis Plus^{Footnote 9} (GBA Plus) metrics at a corporate level, as required.

The Corporate Planning Division, which falls under SPD within the Regulatory Affairs Branch, is the CNSC’s GBA Plus Responsibility Centre and, as such, reports on GBA Plus in the Departmental Plan and Departmental Results Report as required by the Treasury Board. According to Women and Gender Equality Canada, the role of a GBA Plus Responsibility Centre is to lead, support and monitor implementation of a GBA Plus framework, and to provide oversight, direction and promotion of GBA Plus across an organization. However, responsibility for GBA Plus extends across the entire organization and to all individuals, from the senior managers who endorse the policy or statement of intent, to the subject-matter experts, who are best positioned to apply GBA Plus to specific issues.

Specific metrics on equity, diversity and inclusion are not captured in DNCFR’s performance metrics, and it was noted during interviews that there is limited awareness of how GBA Plus relates to compliance verification programs. While there is no evidence that this has negatively impacted staff or licensees, there is still an opportunity to ensure that GBA Plus factors are formally considered during the CNSC’s compliance verification activities, such as reviewing DNCFR tools, templates and communication products for gendered language. Incorporating such practices may also help to ensure that the CNSC is meeting mandatory federal GBA Plus requirements.

In terms of overall program effectiveness, most (90%) respondents indicated that the CNSC’s compliance inspection activities add value to their organization by increasing the effectiveness of technical, compliance and safety activities. Respondents elaborated that regular, comprehensive inspections drive positive change within their organization and encourage staff at all levels to prioritize safety compliance in their day-to-day work. It was also expressed that inspections contribute to the continued improvement and growth of their organization through the provision of feedback and the sharing of best practices.

Safety inspections not only identify what needs to be improved, but also reinforce what is being done correctly

Almost half of respondents indicated that the CNSC’s compliance inspection activities and processes help increase the efficiency of their organization’s resources. The majority of these respondents specified that inspections highlight areas that need more focus, serve as a reminder for staff to remain attentive, and/or assist with the allocation of resources. Perceptions on factors that may hinder overall efficiency were also expressed. Some respondents noted that, although beneficial in the long term, compliance inspections require a significant amount of time and resources to plan, undertake and respond to. A few respondents suggested that a scalable, risk-based approach would improve organizational efficiency by reducing the time and effort required (e.g., low-risk facilities need less inspection), which aligns with the international benchmarking findings that regulators are moving away from reviewing everything on a periodic basis to a more risk- and intelligence-based review that has led to a more efficient use of resources.

Other areas for improvement were identified and various suggestions on enhancing the delivery of compliance inspection services were made. Some of the suggestions included:

• prioritizing the consistency of inspections carried out among licensee sites to promote efficiency
• establishing clear regulatory requirements without room for interpretation
• ensuring that inspection teams are experienced and well-informed on safety culture (e.g., technical competence, appropriate qualifications)
• ensuring that the frequency of inspections is relative to the risk of each site (e.g., low-risk sites require less frequent inspections), with some SCAs requiring more/less focus than others; corresponding resources should be adjusted based on that level of risk
• allowing smaller research facilities to replace reports with an annual site visit owing to limited capacity
• sharing best practices found during similar inspections from other licensed sites, which would be valuable for licensees

According to the DNCFR Performance Information Profile (PIP), one of its intended outcomes is that stakeholders understand regulatory requirements. The understanding of regulatory requirements is critical for delivering on the CNSC’s mandate to regulate the use of nuclear energy and materials to protect health, safety, security and the environment. According to the licensee survey results, most (84%) respondents indicated that they “completely” or “mostly” agree that regulatory requirements for nuclear fuel cycle activities are clear (see figure 10 below).

Figure 10 - Extent to which licensee respondents agree that there are clear regulatory requirements for nuclear fuel cycle activities under the licensee agreement

Text Version:

Extent to which licensee respondents agree that there are clear regulatory requirements for nuclear fuel cycle activities under the licensee agreement

Completely Agree	Mostly Agree	Slightly Agree	Slightly Disagree	Mostly Disagree	Completely Disagree
16%	68%	11%	5%	0%	0%

Total of respondents that mostly agree or completely agree = 84 percent.

Although a majority of licensees mostly agree that regulatory requirements are clear, a small segment of respondents do not, which may present an opportunity to target communications to stakeholders so that all licensees can “completely agree” that requirements under their licensee agreement are clear.

6.5.4. Conclusion

Performance metrics for planned compliance verification activities and outputs are clearly defined, measured against planned results, and periodically reported to management. The metrics are primarily activity-based and do not relate to risks or the effectiveness of inspection performance. Not utilizing an effectiveness lens when measuring performance may increase the likelihood that key risks are not captured or sufficiently monitored, which in turn may hinder effective future risk-informed inspection planning and decision making. Results from the benchmarking activity highlighted some best practices in performance measurement that the CNSC should consider assessing in order to address some of the risks noted.

Although planned compliance verification activities and workloads are achievable and measurable within DNCFR, one division cited workloads as a significant challenge because of competing priorities and available staff capacity.

In terms of overall effectiveness, most DNCFR licensees noted that the CNSC’s compliance verification program influences the safety culture of their organization, and a majority of licensees mostly agree that regulatory requirements for nuclear fuel cycle activities are clear. Some opportunities for improvement pertain to the frequency and consistency of inspections.

Recommendation

4. DNCFR should expand the breadth of the compliance verification performance metrics to ensure that relevant risks are identified, monitored over time, and used to inform compliance verification planning decisions. This should include:

1. data trending
2. scrutiny of a licensee’s internal problem verification and reporting mechanisms
3. consideration of additional risk factors, such as those identified in the benchmarking exercise
4. consultation with SPD (GBA Plus Responsibility Centre) in order to incorporate a GBA Plus lens in compliance verification activities, where relevant

7. Overall Conclusion

DNCFR’s risk-informed inspection process is supported by a comprehensive governance framework, and DNCFR personnel are knowledgeable about the documented guidance in place to support risk-informed compliance verification planning and conduct. The use of information management tools across DNCFR is generally consistent, and information is generally accessible, reliable and accurate. Collaboration within DNCFR and among key stakeholders is effective, and multiple platforms are in place to guide this. Performance metrics for planned compliance verification activities and outputs are clearly defined, measured against planned results, and reported to the Management Committee on a regular basis. In terms of overall program effectiveness, most DNCFR licensees note that the CNSC’s risk-informed compliance verification program positively influences the safety culture of their organization and that they find value in the delivery of inspection activities.

Areas for improvement include consolidating documented guidance for ease of use, documenting additional guidance on unannounced inspections, and documenting a clearer rationale for risk-ranking activities. From an information management perspective, there are opportunities to reduce duplicative efforts and inconsistencies, while ensuring the integration of some of the information management tools currently in use. Compliance and collaboration can be improved by better tracking key licensee submission documents and ensuring that they are distributed to the appropriate staff as needed. There are also opportunities to expand the scope of the performance metrics to support planning and decision making, while examining workload concerns more closely to determine whether assigned resources are commensurate with the expected workplan each year.

The successful implementation of the recommendations identified in this report, along with the completion of other initiatives already underway by management, will strengthen DNCFR’s risk-informed compliance verification processes. A management action plan to address the recommendations of this joint audit and evaluation can be found in section 8.

8. Observations, Recommendations and Management Action Plan

Observations	Recommendations	Management action plan	Target completion date
1. The guidelines for constructing the DNCFR baseline plan are dispersed among a variety of documents. This may pose challenges when conceptualizing the process and/or training new staff. Many of the procedural documents were last updated in 2018 and 2019, and one document reviewed appeared to be obsolete. It would be beneficial to review the procedural documentation to ensure that the content is still relevant and up to date.  (Audit)	2. DNCFR should update the process documentation in order to: include more documented guidance and context for inspection cancellations and deferrals consolidate its compliance verification procedural documents ensure that the content is current and relevant	Management agrees with this recommendation. MPSSD will examine DNCFR documentation by conducting a self-assessment review of all procedural documentation tied to compliance in order to ensure that the content is still relevant and current; make updates where needed; add guidance for inspection cancellations and deferrals and reactive inspections; and consolidate the documentation, where beneficial. Key implementation steps: June 30, 2025: Conduct self-assessment, report findings to DNCFR management, and get endorsement to action recommendations. September 30, 2026: Implement recommendations of the self-assessment. Action owner: DNCFR (coordinated by MPSSD)	September 2026
1. Through benchmarking, it was found that other federal government departments deploy distinctive models to establish systematic risk-ranking processes in their organizations. Each risk model considers factors that are specific to each of the regulated parties, rather than applying a generic approach. Similarly, all international nuclear regulators use a risk-informed inspection process, whereby the level of regulatory attention applied to the licensee is based on multiple factors, such as the level of hazard and risk posed by the facility or activity, and the licensee’s history of safety and security. The benchmarking exercise also revealed that most benchmarking participants perform unannounced inspections and identified various benefits of doing so. Although unannounced inspections are uncommon within DNCFR, they were also identified as an opportunity for improvement by the IRRS and as a good practice within DPRR. (Audit and evaluation)	DNCFR should conduct an analysis of the current risk-informed compliance verification planning process to identify how it can leverage best practices used in other organizations and within the CNSC, including, but not limited to: implementing the use of a risk- and intelligence-based planning approach to focus more on higher risk inspections, which promotes increased resource and outcome efficiency increasing the use of unannounced inspections to strengthen and complement existing compliance verification practices	Management agrees with this recommendation. MPSSD will conduct an exercise to compare the current risk-informed compliance verification planning process to best practices used in other relevant organizations or found in international guidance and commit to incorporating improvements into the existing process where relevant. The exercise will also consider ways to improve DNCFR’s existing process to emphasize/promote increased attention on higher risk inspections and promote the use of unannounced inspections where relevant. The outcome of this exercise will feed into commitments made under MAP 1. Key implementation steps: June 30, 2025: Complete the exercise (to complement the self-assessment identified in MAP 1), report findings to DNCFR management, and get endorsement to action recommendations. September 30, 2026: Implement recommendations as part of commitments to address MAP 1. Action owner: DNCFR (coordinated by MPSSD)	September 2026
3. During the audit testing of sampled inspections, the engagement team found inspection documentation to be, in most cases, accessible, reliable, accurate and complete. The major exception to this finding pertained to the lessons learned step in the inspection process within CMS, which was largely incomplete and inconsistent for the sampled inspections. There is a step in the inspection process to review lessons learned from previous inspections before planning. Only one completed inspection from the sample had the lessons learned tab populated in CMS. (Audit)	3. DNCFR should ensure that lessons learned from inspections are consistently documented in the appropriate information management system and that a process is developed to regularly analyze knowledge and share it with other stakeholders as appropriate.	Management agrees with this recommendation. DNCFR will communicate expectations to consistently use CMS for documenting lessons learned and develop guidance for sharing this knowledge as appropriate. As MPSSD develops and rolls out a new digital tool, it will include this functionality as a requirement in order to ensure that inspections consistently document lessons learned. Action owner: Director General, DNCFR (coordinated by MPSSD)	December 2024
4. The engagement team noted some coordination issues in a related IAEED case study of a DNCFR licensee, in which the licensee failed to submit a key document for CNSC specialist review. The omission was not caught by the CNSC project officer, who is ultimately responsible for monitoring licensee submissions, or by the CNSC technical specialist. IAEED follow-up noted that for each technical area, there are many different technical documents to track in terms of licensee submissions. To IAEED’s knowledge, there is no standard way in which to track these submissions. (Audit)	4. DNCFR should ensure that licensee submissions are consistently and effectively tracked, leveraging available tools.	Management agrees with this recommendation. In the short term, DNCFR will implement a standardized process for tracking licensee submissions. In the longer term, the new MS 365 tools will be evaluated for the tracking of key licensee submissions. Action owner: DNCFR (coordinated by MPSSD)	March 2025 March 2027
5. Three of the 4 performance metrics for planned compliance verification activities focus entirely on the completion status of planned inspections within each directorate. The fourth performance metric tracks the percentage of inspection reports issued against the service standard timeline of 60 business days. Having more performance indicators can help ensure completeness of information when making risk-informed compliance verification planning decisions. Decisions on risk assessment and prioritization of inspections are ultimately made by management with the support of CNSC staff. Much of the data used to inform these decisions originates from the licensee, who is ultimately responsible for the safety of its sites. With this approach, however, there is a residual risk that licensee reporting may not show underlying compliance issues. Additionally, while specific metrics on equity, diversity and inclusion are not captured in DNCFR’s performance metrics, it was noted during interviews that there is limited awareness of how GBA Plus relates to compliance verification programs. (Audit and evaluation)	5. DNCFR should expand the breadth of the compliance verification performance metrics to ensure that relevant risks are identified, monitored over time, and used to inform compliance verification planning decisions. This should include: data trending scrutiny of a licensee’s internal problem verification and reporting mechanisms consideration of additional risk factors, such as those identified in the benchmarking exercise consultation with SPD (GBA Plus Responsibility Centre) in order to incorporate a GBA Plus lens in compliance verification activities, where relevant	Management agrees with this recommendation. MPSSD will conduct a review of DNCFR’s compliance verification performance metrics to ensure that they are relevant, up to date, and incorporate the factors highlighted in the recommendation. Key implementation steps: September 30, 2025: Review available performance metrics, identify relevant metrics, and seek DNCFR management endorsement. September 30, 2026: Implement use of new performance metrics. Action owner: DNCFR (coordinated by MPSSD)	September 2026

9. Appendices

Appendix A: Audit Criteria and Evaluation Indicators

Governance

Objectives	Criteria/Indicators	Data source/method
Audit sub-objective 1.1: A governance framework is in place to ensure adequate oversight of risk-informed inspection planning.	1.1.1: Responsibilities and accountabilities are clear and communicated with respect to risk-informed inspection planning activities across the CNSC, specifically between ROB and TSB. 1.1.2: There is clear governance with respect to identifying reactive inspections.	-

Risk management

Objectives	Criteria/Indicators	Data source/method
Audit sub-objective 2.1: There are defined processes and practices to plan and prioritize inspections using a risk-informed approach.	2.1.1: There are processes in place to support risk-informed inspection planning across the CNSC, incorporating key factors such as licensee performance.	-
Audit sub-objective 2.2: Mechanisms are in place to ensure that inspection planning and the conduct of inspections are consistent with the associated level of risk.	2.2.1: Risk-informed inspection plans are periodically reviewed to ensure relevancy and accuracy of risks. 2.2.2: A review and approval process is in place for modifying existing risk-informed inspection plans. 2.2.3: There are processes in place for identifying and subsequently documenting risks and/or triggers for reactive inspections. 2.2.4: The CNSC conducts inspections that are consistent with established plans, priorities, risks and with its own procedures. 2.2.5: Tools and guidance are provided to support risk-informed inspection planning. 2.2.6: Tools and templates (e.g., inspection guides) are available to support staff when developing multi-year risk-based inspection plans and conducting inspections

Operations, information and knowledge management

Objectives	Criteria/Indicators	Data source/method
Evaluation question 1: Is the delivery of compliance inspection services effective?	EQ 1.1: Effectiveness of program delivery, best practices and areas for improvement Views on collaboration Between and among directorates, with specialists, with other branches (e.g., ROB) EQ 1.2: Views on capacity of resources Unplanned based on licensee performance Directorate workloads Addressing backlogs EQ 1.3: Views on how GBA Plus has been incorporated into programs EQ 1.4: How is safety culture included in the compliance verification of licensees and is it effective?	Document review Internal interviews Benchmarking Survey of licensees
Audit sub-objective 3.1: Management has a formal approach to information and knowledge management to ensure knowledge continuity.	3.1.1: Information management tools are consistently used across directorates, are well-documented, and deliver accessible, consistent, reliable and accurate information. 3.1.2: Processes and procedures have been developed, implemented and communicated to support collaboration.

Effectiveness – Performance measurement, monitoring and reporting

Objectives	Criteria/Indicators	Data source/method
Evaluation question 2: Is the program set up to report on progress and performance in a way that tells the full story of program success? Are changes required to current indicators, and/or options to add new ones? Are changes required to the collection, storage and tracking of data?	EQ 2.1: Perceptions on validity and usefulness of current performance indicators EQ 2.2: State of quality, consistency, coverage and timing of data collection and reporting EQ 2.3: Systems and processes in place to store, collect and analyze performance data EQ 2.4: Limitations encountered during the collection, storage, analysis, tracking and use of performance data EQ 2.5: Perceptions on possible new indicators and opportunities for improvement to processes EQ 2.6: Indicators currently being tracked by the program (covered by audit sub-objective 5)	Document review Internal interviews Benchmarking
Audit sub-objective 4.1: Management has identified appropriate performance measures linked to planned results.	4.1.1: Performance metrics for planned compliance verification activities are clearly defined and documented to ensure that risk-informed inspection plans and related activities are carried out successfully. 4.1.2: Planned compliance verification activities are achievable and measurable.	- or N/A
Audit sub-objective 4.2: Management monitors actual performance against planned results and adjusts course as needed.	4.2.1 Results of performance measurement are documented, are reported to required authority levels, and factor into decision making.	- or N/A

Effectiveness – Achievement of outcomes

Objectives	Criteria/Indicators	Data source/method
Evaluation question 3: To what extent has the program made progress towards achieving its compliance inspection intended outcomes?	Outcomes produced related to (see logic models and PIP indicators): Outcome 3: Risks are identified, monitored and controlled Performance targets have been met Perceptions on factors facilitating or hindering the achievement of outcomes	Performance data review Internal interviews Survey of licensees

Appendix B: Inspection Process Flowchart

Text Version:

The chart describes the step-by-step DNCFR compliance verification process in 3 phases where different steps are recommended based on yes or no questions, and performance considerations are noted.

Phase 1: Create Baseline Plan

• Start
• Risk profiles are established for each facility
• 10-year facility-specific compliance plan is developed
• POs consult with technical specialists and FAC team

Question 1: Was DG approval received?

• If your answer is “Yes”, go to the first step of Phase 2: Annual Planning:
  • Baseline plan forms the basis of the annual plan
• If your answer is “No”
  • Revise plan per DG feedback

Performance consideration: This process only considers inherent risks of the facility operating in a compliant manner and does not take into account licensee performance

Phase 2: Annual Planning

• Baseline plan forms the basis of the annual plan
• POs consider licensee performance and consult with FAC team and licensee

Question 2: Is there new information?

• If your answer is “Yes”, go to the last step of Phase 2: Annual Planning:
  • Annual Plan
• If your answer is “No”
  • Revise plan based on feedback

Performance consideration: Licensee performance is one of the factors considered in annual planning.

Phase 3: Within Year

• Compliance Verification Activities performed according to Annual Plan
• Information collected through compliance verification

Question 3: Are there compliance issues?

• If your answer is “Yes”
  • PO, Monitoring Officer coordinate to add reactive compliance verification activities
  • Director Approval
  • Compliance verification activities carried out commensurate to risk
• If your answer is “No”, go directly to the last step of Phase 3: Within the year:
  • Compliance verification activities carried out commensurate to risk

Performance consideration: Licensee performance can trigger reactive activities in-year.

Appendix C: Summary of Benchmarking Results

As part of the Joint Audit and Evaluation of the Risk-Informed Compliance Verification Processes, the IAEED conducted a benchmarking exercise against 8 organizations to identify best practices used for compliance verification activities. The organizations varied in size, from approximately 250 to 10,000 employees, and consisted of 4 international nuclear regulators and 4 federal government departments. The following is a summary of the results.

Topic	Summarized findings
Risk identification practices of other international nuclear regulators	One international regulator moved away from reviewing everything on a periodic basis in 2018 to promote the efficient use of resources, especially for low-risk areas. The regulator now uses a risk- and intelligence-based review. One international regulator uses 3 levels of regulatory attention assigned to each nuclear licensed site for safety, security, safeguards and transport. The intent of the attention levels is to better target regulatory effort based on risk and intelligence, using judgment across a range of indicators. The 3 regulatory attention levels are: Level 3: Routine Level 2: Enhanced Level 1: Significantly enhanced Two international regulators use a sampling approach rather than 100% inspection. This sampling approach is informed by risk and with the underlying premise that the licensee is responsible for controlling hazards and risks associated with its undertakings. One international regulator applies the principle of proportionality when determining its actions, so that the scope, conditions and extent of its regulatory actions are commensurate with the human and environmental protection implications involved. One international regulator’s regulatory oversight is informed by the complexity and potential harm posed by the licensed activity or facility. This means that those posing lower risk will be subject to less oversight and scrutiny.
Risk identification practices of other Canadian federal government departments	One department developed an internal science-based risk assessment model that drives its inspection plan, focusing on 3 key elements: inherent risk, regulatory control requirements, and the compliance history of the establishment. This helps determine the frequency of inspections for each of the establishments. Those establishments with the highest risk are inspected more frequently. One department has a risk model that assesses the risk or “consequences” for all of the regulated organizations it evaluates, using a tailored approach for each organization. Risk factors include product or service provided, related hazard, and proximity of the potential hazard to people, water, habitat and so on. The regulated companies are then ranked on a numeric scale from 1 to 9. One department generally applies a numeric scale (usually from 1 to 5) to represent the degree of perceived risk within the targeted entity. Some programs use advanced machine-learning algorithms to generate their rankings, while others rely on expert opinion and judgment. One department deploys risk profiling to inform inspection prioritization using data collected from previous inspections, licensing applications and international partners. Various factors are taken into consideration when risk profiling a site, with the profiles differing based on factors such as location and compliance history.
Unannounced inspections by other international nuclear regulators	Two international regulators perform unannounced inspections in response to a trigger (e.g., whistleblower information). One international regulator conducts unannounced inspections only on nuclear power plants. One international regulator conducts unannounced activities on a routine basis. To improve the efficiency of inspection trips and to reduce agency costs, they may contact the licensee in advance to verify that the inspection can be performed effectively and efficiently at the intended time.
Unannounced inspections by other Canadian federal government departments	One department conducts unannounced inspections on an ad-hoc basis if an inspector is in the vicinity of a potential issue. In addition, random inspections that are unannounced are practised by some programs in this department for quality control of the risk-based program. One department often conducts unannounced onsite compliance inspections without notice in response to complaints. One department noted that unannounced inspections can be done and are conducted in response to a concern or a complaint. The preference is to avoid them in order to ensure that the regulated party has the staff and paperwork available at the time of the inspection. One department conducts some unannounced inspections on an infrequent basis. One department has documented guidance on the benefits of unannounced inspections and when they may be appropriate. Some benefits include: the opportunity to observe “typical behaviours” when not being watched onsite by the regulator the ability to witness conditions at the workplace during the normal course of business providing a useful measure of the effectiveness and sustainability of licensee programs
Information management by other international nuclear regulators	One international regulator developed its information system as part of a strategy to digitize key tasks, increase the accessibility and visibility of information, and enhance the efficiency and effectiveness of its regulatory team. Inspection data in this system was controlled and aggregated in one place so that inspectors could see trends in compliance. There was also a secure link where licensees could seamlessly upload files for inspectors, including information about incidents, regulatory issues, assessments and inspections. The tool is part of its corporate strategy to become a more modern regulator, improving its ways of working and achieving an improved digital solution for its core business areas. One international regulator developed an in-house digital tool to plan, conduct, report on and document its inspection activities. It provided a web-based online tool for concurrent input by multiple inspectors working on a team inspection.
Information management by other Canadian federal government departments	One department built its information system using an integrated, modularized workflow approach for several activities, including, but not limited to, compliance verification, remediation and other enforcement activities. The information system supports planning, conditional oversight and data analysis, and captures lessons learned. Key periodic report submissions required from regulated companies/licensees are tracked and captured in the information system. One department developed an in-house information management tool that supports risk assessment activities, compliance and related information tracking. Key statistics, such as peaks in non-compliance and the number of inspections conducted in a particular area, can be tracked with this tool.
Performance reporting by other international nuclear regulators	Three international regulators publicly report the percentage of inspections completed across all business lines on an annual basis. One international regulator has defined performance indicators that link back to its corporate plan using 5 strategic themes, with compliance inspection activities falling under “Influencing Proportionate Improvements”. There are 4 performance measures under this theme: Regulatory activity drives demonstrable improvements and compliance across purposes. Regulatory decisions are proportionate, balanced and unbiased. A continuous self-improvement and learning culture. Innovators are confident to test and deploy technology in a regulatory framework that embraces innovation.
Metrics tracked by other international nuclear regulators	Three international regulators track additional indicators besides the percentage of inspections completed, including: Number of significant events/incidents Number of findings above a “low safety and security significance” Number of deviations between the situation observed during the inspection and the regulations Number of anomalies or aspects warranting additional justifications Days lost due to work-related accidents Number or rate of safety rule errors Number of nuclear security incidents Number of regulatory issues

Appendix D: Acronyms

Term	Meaning
CMS	Case Management System
CNSC	Canadian Nuclear Safety Commission
CNLRPD	Canadian Nuclear Laboratories Regulatory Program Division
DNCFR	Directorate of Nuclear Fuel Cycle and Facilities
DPRR	Directorate of Power Reactor Regulation
FAC	Facility Assessment and Compliance
GBA Plus	Gender-based Analysis Plus
IAEA	International Atomic Energy Agency
IAEED	Internal Audit, Ethics and Evaluation Division
IM	information management
IRRS	Integrated Regulatory Review Service
MAP	management action plan
MPSSD	Major Projects and Strategic Support Division
NPFD	Nuclear Processing Facilities Division
OAG	Office of the Auditor General
PIP	Performance Information Profile
RIB	Regulatory Information Bank
RICV	risk-informed compliance verification
ROB	Regulatory Operations Branch
SA	self-assessment
SCA	safety and control area
SPD	Strategic Planning Directorate
SME	subject-matter expert
TSB	Technical Support Branch
UMMD	Uranium Mines and Mills Division
WDD	Wastes and Decommissioning Division