It's time for your next model validation, but who should perform it - and how?
Regulators[i] make it clear that model validation is critical to internal control systems. But they are less clear as to who should perform these validations (an independent third party? internal/external audit? your model risk management team?), as well as what is considered adequate scope. As a result, many banks and credit unions make the all too frequent mistake of not engaging an experienced validator with the horizontal perspective necessary to go beyond a simple audit. This may not only render institutions vulnerable from a regulatory perspective, but they also miss key opportunities to build strategic value.
Most traditional “audits” of Interest Rate Risk (IRR) models consist of ensuring:
Data feeds tie to the model
The model’s projected first month of income is reasonably close to the ‘as of’ month’s actual income
Policy limits are included in reporting
No limits are being breached
Relying on decades of experience performing model validations, DCG believes that an effective validation transcends this traditional “tick and tie” checklist, and should also ensure:
A model’s logical and conceptual soundness
The efficiency and adequacy of controls surrounding the overall risk management processes
As risk management processes continue to evolve, regulators are looking less favorably on bare-bones validations, and rightly so. For an IRR model to not only function mechanically but also have a strategic focus, it’s time to insist on a more robust, all-encompassing validation.
Key components of an effective validation
DCG believes that the purpose of validations is to improve the model and process, and as such ensures our validation scopes include, at a minimum, a thorough assessment of:
Risk Management Policies: Are policies comprehensive and clear? Are risk metrics reasonable and in line with management and the Board’s risk appetite?
Governance: What internal controls surround the model? What type of model documentation exists? Who can make changes to the model?
Scenarios: Are the core scenarios appropriate? Does management review an array of alternative scenarios to enhance the understanding of the IRR profile?
Data Input/Quality: Are data inputs accurate and comprehensive? Is data transformation necessary due to deficiencies in the underlying data source, and if so, does the applied logic appear reasonable, and is it documented? Is the data being aggregated and if so, is it being done such that unique cash flow and/or repricing characteristics are not muted?
Category Structure: Is there sufficient granularity in the model structure to understand and accurately capture the underlying cash flow and repricing characteristics? Does the category structure tell the “story” of the balance sheet?
Assumption Development/Support: How are model assumptions developed? Who is involved in the assumption gathering process? Are assumptions used in the model reasonable and defensible? Are they based on institution-specific data? Industry data? Management estimates? Is the methodology appropriately documented?
Sensitivity/Stress Testing: Are assumptions sensitivity tested as a means of fully illustrating the potential impact they have on the interest rate risk profile? Are sensitivity tests reported and explained to members of ALCO and the Board?
Back-Testing & Outcome Analysis: Are model results (interest income, balance sheet volume, cash flow, etc.) back-tested and reviewed regularly? Are variances reviewed and documented?
ALCO Reporting: Does reporting material clearly provide model results, and are these results compared to all applicable policy limits? Are results effectively communicated/conveyed to all stakeholders? Does the reporting encourage proactive strategic discussion?
While it’s clear that a “model validation” can be broadly defined, it is important to understand your institution’s specific goals when engaging a validator. If your goal is regulatory compliance, an “audit” may have been sufficient in the past; today, the stakes are higher due to the increased model complexity and the need to make more impactful strategic decisions based on model output.
To ensure models and ALM processes are performing optimally, institutions should select a deeply experienced validator to get the most out of your validation and improve strategic performance.
[i] Board of Governors of the Federal Reserve System, SR 10-1: Interagency Advisory on Interest Rate Risk Management Board of Governors of the Federal Reserve System, SR 11-7: Guidance on Model Risk Management
Learn more about DCG's DCG’s comprehensive approach to model validation built on over 40 years of experience.
ABOUT THE AUTHOR
Mark Haberland is a Managing Director at Darling Consulting Group. Mark has over 25 years providing balance sheet and model risk management education and consulting to the community and mid-size banking space. A frequent author and top-rated speaker on a wide array of risk management topics, Mark facilitates educational programs and workshops for numerous financial institutions, industry and state trade associations, and regulatory agencies.
Contact Mark Haberland: mhaberland@darlingconsulting.com or 978-499-8152 to learn more about DCG's approach to model validations and Model Risk Management.
© 2022 Darling Consulting Group, Inc.
