Many banks and other financial institutions are struggling to stay current with anti-money-laundering (AML) regulations. Regulators have assessed several high-profile enforcement actions, fines, and penalties against financial organizations with lax controls over money laundering. Some of these actions were the result of an institution's failure to appropriately apply the concepts of a model risk management framework to design, execute, and maintain the models it deployed to manage AML risk.

As the financial institutions industry has evolved — offering new high-risk products, acquiring new types of customers, and adapting to frequently changing money laundering requirements — banks increasingly rely on complex models to meet the challenges of AML compliance.

The Office of the Comptroller of the Currency (OCC) and the Federal Reserve Board issued important regulatory guidance, "Supervisory Guidance on Model Risk Management," on April 4, 2011. According to the guidance, the term "model" refers to "a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates."¹

Financial institutions relying on bank-specific models should use what regulators refer to as a "model risk management framework" for developing, implementing, and using these models. This framework enables banks to predict and identify risk more accurately and, therefore, make better top-level and line-of-business decisions based on model results.

To truly reduce risks inherent in AML models, a comprehensive and sustainable AML program should include the following critical components:

1. Model inventory and assessment;
2. Model development, implementation, and use;
3. Model validation;
4. Model tuning and optimization;
5. Model governance; and
6. Model foundation.

In this article, each of these six components is discussed in turn.

Meeting business needs and regulatory expectations

Initially, banks interpreted the broad multi-agency guidance to apply only to their credit and financial risk modeling. Subsequently, however, regulators have made it clear that the guidance applies to AML compliance. Hence AML models, such as transaction monitoring systems, customer risk-rating programs, and watch-list filtering systems, are being held to the same standard.

The guidance provides banks with a basic framework for deploying an enterprise-wide model risk management program. Examiners expect banks to use the framework when designing, implementing, and improving all models, including operating models used for AML compliance. Examiners are looking for a formalized, prescriptive methodology dictating the three basic components of the model risk management framework that the guidance identifies and requires:

1. Model development, implementation, and use;
2. Model validation; and
3. Model governance.

These expectations obligate banks to modify their approach to AML compliance in order to comply with the new industry standard.

In addition, regulators expect organizations that use AML models to provide evidence that the model risk management framework in place effectively identifies and manages the risk arising from the bank's reliance on models. Evidence might include, for example, documentation that the bank has identified where its data resides and who is responsible for the quality of that data meeting model governance requirements.

Financial institutions are struggling with how to adapt existing AML compliance programs to a model risk management framework. Many already have enterprise-wide policies and procedures for financial models but might not have extended a similar approach to AML-specific compliance components. Some banks are finding that reworking or enhancing existing AML model risk management processes to comply with supervisory guidance can be complicated and costly. Finally, in many cases, banks are realizing they have in place few of the model risk management components necessary for an effective AML program.

Defining a model risk management framework

Banks should build their AML models using the three requirements in the guidance but also with three additional critical components of effective model risk management (Exhibit 1): a model inventory, model tuning and optimization, and a model foundation. The model foundation components — business and regulatory alignment, supporting documentation, enabling technology, and project management — are supporting elements that further the effectiveness of AML models overall.

The determination that these particular components are critical to AML model risk management is based on an informal analysis by Crowe Horwath LLP of regulatory guidance, examination findings, dialogues with regulators, and client insights. Institutions that implement and optimize the six components explained here can more effectively build a comprehensive AML program that meets business and regulatory needs.

Exhibit 1 Source: Crowe analysis

1. Model Inventory

   A model inventory takes stock of the implementing components — primarily the people, processes, and technology — that support AML models (Exhibit 2). The inventory of existing models for AML compliance at any single financial institution may be large and can include both manual and automated processes and technology. Once an inventory is completed, the organization should assess the enterprise risk of each item in the inventory in order to manage operational risks commensurate with the perceived risks in the model.

   

   Exhibit 2

   Completing an AML model inventory requires taking the following three steps:

   1. Enterprise identification. Existing models that support AML compliance in the organization are identified.
   2. Model risk assessment. The model's enterprise risk is assessed. The results of the assessment then drive the level and scope of the model risk management activities applied to each model.
   3. Accountability. The organization clarifies who is ultimately accountable for controls, compliance, and oversight of the models identified in the inventory.

2. Model Development, Implementation, and Use

   An AML model must be well documented and tailored to the unique risk profile of the organization (Exhibit 3). Due to the varying risk profiles and business requirements of different financial institutions, each AML model must be customized to meet the specific needs of the institution. When banks design models for AML programs, relevant stakeholders — including AML executives, IT personnel, and affected line-of-business leaders — should assess the design structure, implementation approach, and use of output from the AML model for efficacy and regulatory compliance.

   

   Exhibit 3

   1. Model Development and Implementation: Model definition and requirements. According to the joint guidance, "An effective development process begins with a clear statement of purpose to ensure that model development is aligned with the intended use." Banks must document model definitions and requirements for meeting business and regulatory needs. This development process involves the bank's identification of the various sources of its functional data. The process also includes defining and documenting performance and technical requirements, and the result serves as the baseline justification for the model.
   2. Model Development and Implementation: Model design. An AML model must be designed based on the documented objectives of the model and functional and technical requirements. Any limitations or merits of the model must be documented and considered as the model is developed. Model limitations — which could be tied to model inputs, the processing component of the model, or the way the model output is created — will be an important consideration when model risk is assessed.
   3. Model Development and Implementation: Data management. AML compliance relies heavily on data quality, and data is never perfect. Efforts must be made to assess the completeness, accuracy, and relevance of input data for the model.
   4. Model Use: Analysis of results. According to the guidance, "An understanding of model uncertainty and inaccuracy and a demonstration that the bank is accounting for them appropriately are important outcomes of effective model development, implementation, and use." AML executives must analyze model results relative to limitations and design assumptions in order to assess the potential performance of a model before other stakeholders use the results.
   5. Model Use: Continuous feedback and improvement. To verify that the AML model is being used in a manner consistent with business and regulatory needs, institutions should implement a feedback mechanism to learn from and improve the process during the development, implementation, and post-implementation of the model. For a process of continuous improvement, management must empower users of AML models to provide input and feedback to the model designers and managers.

3. Model Validation

   Ongoing evaluation of models is required to confirm that results are accurate and controls are adequate. The "Supervisory Guidance on Model Risk Management" defines model validation as "the set of processes and activities intended to verify that models are performing as expected, in line with their design objectives and business uses. It also identifies potential limitations and assumptions, and assesses their possible impact."

   While the concept of validation is not new, the guidance expands the expectations for an effective validation review. The guidance states, "All model components, including input, processing, and reporting, should be subject to validation; this applies equally to models developed in-house and to those purchased from or developed by vendors or consultants."

   Model validation confirms that an institution's AML model is aligned with business and regulatory expectations and is properly executing on underlying risks. The approach to validation focuses on the following four components related to the conceptual design, system, data, and process for AML models:

   

   Exhibit 4

   1. Conceptual design validation. Do the conceptual design and capabilities of the AML model meet the identified business and regulatory needs? In assessing the conceptual soundness of a model, AML executives should consider key metrics and evidence supporting the ability of the model to accurately predict AML risks.
   2. System validation. All AML technology and automated systems implemented to support an AML model have inherent limitations. System validation independently confirms that the development, implementation, and ongoing use of AML technology are properly designed and integrated enterprise-wide to support the model.
   3. Data validation. Is the AML model capturing accurate and complete information? Data errors or irregularities impair results and might lead to the organization's failure to identify and respond to elevated AML risks.
   4. Process validation. Verify that the design and ongoing sustainability of the AML model's processes are resulting in output that is accurate, managed effectively, and subject to appropriate controls.

   AML model validation is often complex and resource-intensive. Testing is generally done by individuals who have not previously provided input on either the design or implementation of the model.

4. Model Tuning and Optimization

   Tuning and optimization is the ongoing process of testing and enhancing the AML model. Model tuning is an important aspect of developing and implementing an AML model as well as its ongoing management and sustainability. Following are the three primary elements of a comprehensive model-tuning program:

   

   Exhibit 5

   1. Gap analysis. Gap analysis helps ensure that all of the risks the bank faces are being monitored at an appropriate level for AML purposes. For example, gap testing might identify new products or services that pose significant risk but are not part of the overall monitoring program, or the testing might identify specific high-risk customer types that are being incorrectly stratified. To confirm that an organization's AML risks are being covered appropriately, the organization or third-party vendor must remediate any gaps or limitations identified during this analysis. Furthermore, the organization should put into place a process for assessing gaps periodically.
   2. Prescriptive methodology. Optimizing AML models requires a consistent methodology founded in quantitative analysis. Prescriptive methodology is a formal documentation of the comprehensive approach the bank is using to optimize the model. The methodology must also define change-control procedures and the initiating events that require subsequent tuning. Examiners now call not only for the periodic assessment of model parameters but also for the definition of specific events — such as a change to an organization's risk profile due to an acquisition — that warrant tuning.
   3. Documentary evidence. Examiner and auditor expectations call for an audit trail of changes in the AML model's parameters during the tuning process. Documentary evidence is used to demonstrate to regulators and auditors that the AML system is aligned to the prescriptive methodology for managing AML activities. Without supporting documentation, the prescriptive methodology might not stand up to a regulatory examination. The bank must maintain the documentary evidence for the life of the AML model.

5. Model Governance

   All financial institutions that rely on models for AML compliance should implement an appropriate governance program. According to the guidance, "Even if model development, implementation, use, and validation are satisfactory, a weak governance function will reduce the effectiveness of overall model risk management."

   An organization's governance policies, procedures, and processes should support controls and oversight to manage an effective and sustainable AML program. AML models should be implemented from end-to-end, starting with the board of directors and senior management, who design and oversee operating models; and including the line-of-business managers, who are responsible for implementation; third-party vendors or IT specialists, who implement technology components; and compliance risk managers, who must manage the risk inherent in the model. Effective model governance requires the following elements:

   

   Exhibit 6

   1. Senior management and board involvement. Senior management and the boards of directors must set the direction and oversee the policies of a model risk management framework. AML models that are managed at the leadership level are more likely to be supported by well-thought-out policy and procedures as well as comprehensive methodologies that are executed consistently throughout the enterprise and managed through continuous improvements.
   2. Policies and procedures. At a minimum, the cornerstones of the guidance — development, implementation, and use; validation; and governance — must be addressed in formalized and enterprise-wide policies. Implementing procedures should dictate the organization's prescriptive methodology for managing the risks of models relied upon for AML compliance.
   3. Roles and responsibilities. Clear lines of reporting identify who owns model risk management processes and affirms the control structure that supports compliant AML procedures. When the bank lacks the knowledge to manage these functions, an individual should be responsible for augmenting the organization's expertise in order to meet the bank's model risk management standards.
   4. Enterprise risk management and reporting. Organizations with an enterprise risk management function should define risks, events, or changes in the organization that could affect business processes. An effective model risk management framework promptly identifies such risks and assesses their potential impact on bank models. Clear lines of authority and reporting should be established to oversee how a model's performance might change as a result of changes in the organization.
   5. Independent audit and testing. Auditing and self-testing effectively should challenge the model's conceptual design, data reliability, and risk management controls. The role of auditing and self-testing is not to duplicate model risk management activities but rather to assess the effectiveness of a model risk management framework in meeting AML business and regulatory needs.

6. Model Foundation

   The model foundation is the final component critical to an effective AML model risk management program. A strong foundation adds structure, consistency, and efficiency to support program compliance. Following are the four elements of a robust model foundation:

   1. Business and regulatory alignment. Regulators may deem a model risk management program to be ineffective if business needs are not aligned to regulatory standards. Misalignment could result in a bank's failure to appropriately manage the risks inherent in AML models and thus lead to poor decision-making and even regulatory fines and penalties.
   2. Supporting documentation. Detailed document management and retention are necessary for all areas of model risk management, including but not limited to policies and procedures, periodic model validation results, detailed tuning and optimization analysis, and model inventory and risk assessment.
   3. Enabling technology. Technology helps AML executives to manage the breadth of requirements for an effective model risk management program. Banks must define their approach to using technology based on their size, regulatory standards, and their model risks. From simple spreadsheet maintenance systems to an enterprise-wide model risk management software solution purchased from a third-party vendor, technology use varies by institution.
   4. Project management. Various internal and external stakeholders and third-party vendors often come together to develop and implement AML models. Experienced project management personnel are critical to effective resource management and successful model implementation.

Conclusion

Minimizing AML compliance risk in an era of mandated model risk management standards requires a financial institution to design, implement, test, and improve its AML models on an ongoing basis. The framework outlined here is designed to lead banks through the critical components of building an effective and compliant AML model risk management program.

Institutions that do not adopt a documented, consistent model risk management framework for AML programs risk incurring regulatory actions, fines, and other penalties. All aspects of model risk management require banks to provide robust supporting documentation that demonstrates to regulators and auditors that their AML models are effective and aligned to business and regulatory needs. Using a project management structure overseen by senior management contributes to consistency within the organization and the integrity of its AML models.

Additional tools and strategies are available to help institutions with the daunting requirements of the guidance. Given the complexities of model risk management, many banks are turning to technology available in the marketplace that can help them deploy a systematic and consistent approach. Using technology also helps with the documentation, quantitative analysis, and tuning procedures that federal examiners have come to expect.

Brookton Behm, CAMS, a principal, Crowe Horwath LLP, Grand Rapids, Michigan, USA, brookton.behm@crowehorwath.com

John Epperson, CAMS, Crowe, Oak Brook, Illinois, USA, john.epperson@crowehorwath.com

Arjun Kalra, CAMS, Crowe, San Francisco, California, USA, arjun.kalra@crowehorwath.com

1. Board of Governors of the Federal Reserve System and Office of the Comptroller of the Currency, "Supervisory Guidance on Model Risk Management" (OCC 2011-12 and SR 11-7), April 4, 2011, http://www.occ.gov/newsissuances/bulletins/2011/bulletin-2011-12.html and http://www.federalreserve.gov/bankinforeg/srletters/sr1107.htm.