Get Your Model Risk House in Order
CECL | Model Validation | MRM | MRM Framework
This week marks one year since life as we knew it was altered so drastically, and we were introduced to new terms like "zoom," "socially distant," and "N95." But another anniversary is upon us, one that comes with a bit less fanfare but has had a profound impact on our lives, even if we don't realize it. Ten years ago, on April 4, 2011, Model Risk Management (MRM) was introduced to the banking world and "effective challenge," "ongoing monitoring," and "rigor" became a part of the lexicon of model owners, risk managers, auditors, and examiners.
ALCO is the lifeblood of your organization, and the decisions that you make there have wide-ranging implications. So it stands to reason that you put as much emphasis on the reliability of your risk models and the quality of the model inputs as you do on the information they provide.
The complexity and number of models you rely on are continuing to increase, their utilization for decision-making purposes is expanding, and your potential exposure for errors from these models has never been higher. Do you fully understand the consequences your institution will face if these models don't perform as intended or are misused? The very guidance for which we are celebrating the tenth anniversary outlines the exposures facing your organization if model risk does not take on a more prominent focus:
"The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports. Model risk can lead to financial loss, poor business and strategic decision-making, or damage to a banking organization's reputation."
- SR 11-7: Guidance on Model Risk Management
Model validations, while often viewed as a regulatory requirement, are the most effective means for financial institutions to detect errors or deficiencies in modeling processes or utility – but not all validations are created equal. A "check-the-box" or compliance-based approach can be successful in comparing your process against suggested guidance and identify shortcomings. However, will that level of rigor give you the confidence that the model is performing as well as it should be (or as well as it can be)? The confidence that the model inputs are reasonable and reliable? The confidence that model outputs are being appropriately utilized for effective strategic decision making? In short, are you confident in your risk management process? More importantly, can you afford not to be confident?
In a recent survey conducted by DCG to financial institutions below $10 billion in assets, 85% of respondents said their MRM process was only somewhat developed, while over 90% said that during a recent exam, MRM had at least been discussed. The focus on MRM for mid-size and community institutions will continue to grow as Current Expected Credit Loss (CECL) becomes more prominent. CECL is a new, more complex model with a significant accounting and regulatory impact that will most assuredly draw increased attention, looking to see how your model risk management function is involved in the implementation, testing, and ongoing monitoring of this critical model.
Those who have an MRM framework that is still developing, or has yet to be created, may look at this as a formidable endeavor and continue to push it off into the future. The stakes are too high to just hope everything is OK. You wouldn't move your family into a new house without knowing it's safe, would you? Well, that is precisely the analogy my colleague Hannah Glasrud, one of DCG's leading MRM experts, uses to bring clarity to why model risk is such a critical function for organizations. For example:
These crucial steps along the model life cycle are designed to ensure the model works as intended and continues to perform appropriately. MRM should work to ensure this process is adhered to in order to identify and reduce the risk by working with the model owners and vendors. But if MRM is not in a state to determine your models' condition, where do you begin? The key is to prioritize and understand that MRM is not a "one-size-fits-all" process – find the right fit for your organization.
When implementing an MRM framework, we focus on the critical decisions that must be made to "right size" model risk management within your organization. These decisions are:
Governing Policy/MRM Framework
MRM Procedures
Model Identification and Inventory
Model Risk Rating Assignment
Validation Frequency & Rigor
Model Documentation
The first and most important decision is to take action and build out the framework. Then work on those that have the most impact on your organization's risk. From our experience working with our clients, we would prioritize:
The first and most important decision is to take action and build out the framework. Then work on those that have the most impact on your organization's risk. From our experience working with our clients, we would prioritize:
1. Building out a Model Identification Process and Inventory – you cannot effectively manage risk if you do not know all the models or tools that you have
2. Risk Rate each of your models – through the use of a scorecard or definition-based approach, risk ratings help identify and prioritize the riskiest and most critical models
3. Model Validation Frequency and Execution – based on assigned risk rating, the validation frequency and rigor will be determined; focus on high-risk models first (CECL, BSA, IRR, Liquidity) with validation framework to include evaluation of conceptual soundness, ongoing monitoring, and outcomes analysis
Once you have identified the "high risk" models and defined a standard validation schedule, the next phase will be to implement a process to mitigate the risks identified by the validations and evaluate each model’s performance on a regular basis. Focusing on governance, documentation, and controls will help to round out a solid and sustainable process.
Without an effective model risk management framework and robust ongoing monitoring in place, institutions are open to the possibility of costly errors, inaccurate results that drive misdirected strategy, and opportunity costs for actions not taken. Validation of your key risk models by a team of subject matter experts that can provide effective challenge and influence change is as essential as any strategic decision your team makes. CECL presents a significant regulatory impact, and the increase in fraud and cybercrimes has raised renewed focus on BSA/AML models, triggering more targeted exams that focus on how your model risk process is structured, particularly for these models.
Harkening back to Hannah's analogy of building out an MRM framework being like building a house, keep in mind that one doesn't take on building a house by themselves. It involves bringing in specialists in various fields, teams that work collaboratively, measuring twice and cutting once, and constant inspections and double-checking. In short, you don't have to build out your MRM framework on your own. DCG’s collaborative approach to MRM and validation provides access to former regulators, auditors, consultants, model developers, economists, professors, PhDs, documentation experts, and model risk management experts who apply the combination of "incentives, competence, and influence" in their effective challenge when validating your models.
It has been 10 years since model risk management was introduced to the banking community, yet it is still not incorporated into the processes of many organizations, and for those that have started the process, much work remains. While bankers have been slower to adopt MRM, technology, cybercrime, and worldwide pandemics have continued to advance, making the risks of continuing to delay too big to ignore.
Learn more about Model Risk Management.
ABOUT THE AUTHOR
Mark Haberland is a Managing Director at Darling Consulting Group. Mark has over 25 years providing balance sheet and model risk management education and consulting to the community and mid-size banking space. A frequent author and top-rated speaker on a wide array of risk management topics, Mark facilitates educational programs and workshops for numerous financial institutions, industry and state trade associations, and regulatory agencies.
Contact Mark Haberland: mhaberland@darlingconsulting.com or 978-499-8152 to learn more about the critical decisions in MRM implementations and to find out how DCG can help you establish or improve your model risk management process.
© 2021 Darling Consulting Group, Inc.
