We ask a lot of risk models – to provide consistent, timely, accurate information upon which to make key strategic decisions that can have long-lasting impact to the organization. But do we ask as much of the process to oversee the models themselves? If models go unchecked and misused, the repercussions could be significant.
Model risk has been around since models were invented. However, model risk management (MRM) is relatively new to the industry, and it brings with it some very useful guidelines that help ensure models work as well as they can, especially in periods of uncertainty when we need them the most. Key among these guidelines is the concept of “effective challenge.”
Effective challenge became part of the risk management lexicon at MRM’s onset. It represents the most significant point of divergence between comprehensive validations and mere audits, and has a profound impact on improving model performance. The Supervisory Guidance on Model Risk Management[1] defines effective challenge as:
“Critical analysis by objective, informed parties that can identify model limitations and produce appropriate changes. Effective challenge depends on a combination of incentives, competence, and influence.”
The three key elements of effective challenge drive a validator’s ability to identify how well a model is performing, detect model weaknesses, and provide guidance on improving overall model performance. Validations that incorporate effective challenge in the scope of the engagement ask the tougher questions, digging well below the surface to make a meaningful difference in how models are constructed and used.
Three Action Items
When planning the validation of any key risk models (e.g., interest rate risk, liquidity, CECL, BSA, etc.), consider these effective challenge action items:
Embed cultural incentives. The purpose of a validation is to help ensure the model works optimally and as intended for the size and complexity of the organization. It requires a collaborative approach, with stakeholders working together to enhance performance. Treating a validation as a means to highlight only what is wrong (a “gotcha” mentality) only serves to dis-incent model owners. Instead, emphasize potential model and process improvements, and demonstrate their connection toward overall strategic goals.
Arm the validator with a true understanding of model behaviors. It seems like a logical approach, but the more the validator knows about the model and software, the more beneficial the validation will be. With a true understanding of how models should perform as well as potential software limitations, the validator is enabled to perform a more fine-tuned analysis. Additionally, they are better equipped to offer guidance with implementing potential recommendations.
Exert influence to enact improvements. What good are recommendations for model improvements if they are never implemented or addressed? Engaging with those with the authority to effect change is critical to the actual impact of validation results. Foster connections organization-wide, emphasizing the importance of what was found, the strategic impact of recommended changes, and how to facilitate those improvements.
So many of an institution’s most important decisions are based on the results of key models. Make sure the next important decision you make involves getting the right validation of these models – with effective challenge – to provide confidence in the results.
[1] Supervisory Guidance on Model Risk Management OCC 2011-12 / Fed SR 11-7 / FDIC FIL-22-2017, April 4, 2011
For more Model Risk Management insights, click here.
ABOUT THE AUTHOR
Mark Haberland is a Managing Director at Darling Consulting Group. Mark has over 25 years providing balance sheet and model risk management education and consulting to the community and mid-size banking space. A frequent author and top-rated speaker on a wide array of risk management topics, Mark facilitates educational programs and workshops for numerous financial institutions, industry and state trade associations, and regulatory agencies.
Contact Mark Haberland: mhaberland@darlingconsulting.com or 508-237-2473 to learn more about effective challenge and Model Risk Management.
© 2023 Darling Consulting Group, Inc.
