top of page

Registration is now open for our 40th Annual Conference! 

About Us
Insights
Events
Data-Driven Solutions
Model Validation & MRM
Asset/Liability Management
  • Writer's pictureHannah Glasrud

Five CECL Model Risk Resolutions for 2023


Five CECL Model Risk Resolutions for 2023


How CECL Drove Model Risk Management into the Spotlight


CECL model development stole the show in 2022, but 2023 will be the first time many institutions will put the new model into production. Congratulations on making it this far on your CECL journey, and best of luck with your Q1 implementations!


Model Risk Management (MRM) is now rising into the spotlight as a byproduct of CECL. For those with an MRM framework in place, onboarding CECL and incorporating it into a broader risk management program should be a straightforward path. However, those without an MRM program are finding the onboarding process more difficult—they are missing crucial documentation and controls, among other broader risk-mitigating processes.

The benefits of implementing an MRM program may not have been apparent before. Still, as the first wave of CECL validations begins, many of the resulting findings could have been addressed had a working MRM program been in place before developing the model.


But do not be discouraged! Now is precisely the right time to forge ahead. You can help ensure a smoother journey and long-term benefits by:

  • Gathering supporting documentation for key management decisions, model assumptions, and the modeling methodology

  • Creating formalized and repeatable data and processing controls around the model

  • Ensuring the CECL model performs as intended and captures the underlying credit risk associated with lending activities through validation

  • Taking stock of your MRM progress and making it a priority to “right size” it for your organization in 2023

If these action items cause concern that you “aren’t ready for prime time,” don’t panic: it may indicate that you need to add a few CECL and MRM resolutions for 2023 to your list.


Resolution #1: Get Your CECL Model Validated


Model validation is a critical component of the MRM process. During validation, a qualified independent party should review the model and all its parts to confirm that it is working as intended and appropriate for use as an ACL model. For CECL, the scope and extent of the validation will depend on the methodology used, a portfolio's size and complexity, and the availability of information to validators.


During the validation, the model owner should be able to provide the validator with an end-to-end model run, model documentation, controls, and other governance-related material. Validators should review the input data, the processing component, and the model output and assess that it aligns with the model documentation and utilizes an appropriate approach for estimating a reserve. Additionally, model validators should review key modeling assumptions such as prepayment rates and average life estimates, for which the model owner should have supporting evidence and documentation. For example, a prepayment study may support prepayment rate assumptions.


If possible, the model owner should also supply the validator with testing materials such as assumption sensitivity testing, parallel runs with the ALLL model, and back-testing. Complete model validations will include a review of performance monitoring procedures that are in place.

Resolution #2: Create Remediation Plans for Validation Findings


If a CECL model has already undergone rigorous validation, the next step is to ensure that the institution’s team understands any findings and recommendations. The Model Risk Manager should track such findings, capturing management responses from the CECL model owner and formalizing a remediation plan to demonstrate that the institution is actively trying to fix deficiencies in the model and the modeling process. This is the first critical step in smoothing the CECL road ahead.

Based on DCG’s numerous validations of CECL models, common areas of improvement include:

  • Model risk management

  • Model governance and controls

  • Inadequate documentation and assumption support

The OCC[1] and the FDIC[2] have updated their examiner handbooks to include specifics related to model risk management. This includes creating a model inventory, risk rating models, and developing a policy to outline a program for the first, second, and third lines of defense.


Resolution #3: Make Sure CECL Is in Your Model Inventory


A model inventory is one of the first documents to develop for a Model Risk Management program. Each institution uses model inventories differently; the fields tracked will vary by use. Primary fields are collected for institutions that keep an inventory to track what models are in use, under development, or recently retired. For some models, there may be several variations of the same model with different underlying assumptions/data/uses that would merit separate validations and, thus, should be tracked as a separate model in the inventory.

The OCC has published a list of items that examiners may look for in the model inventory:

  • Model identifier

  • Model version

  • Whether the model was developed in-house or by a third party

  • Model dependency, describing whether the outcome of one model is used as input into another

  • Model owner and user(s) by title or group (e.g., chief compliance officer or compliance department)

  • Status of model (e.g., in development, production, decommissioned)

  • Approval date of model, or timeline for approval

  • Description of the purpose and products for which the model is designed

  • Description of actual or expected usage

  • Description of any restrictions on use or other controls (e.g., more frequent monitoring and appropriate benchmarking)

  • Type and source of inputs used by each model and underlying components of each model (which may include other models)

  • Description of the type of technology or approach used

  • Model outputs and their intended use

  • Identification of individuals responsible for various aspects of the model development and validation

  • Type (e.g., credit, compliance) and level of risk

  • Dates of completed and planned validation activities, ongoing monitoring frequency, and description of validation results/status (e.g., fit for purpose, approval), and changes made to the models that management deems material. E.g., if a bank makes any material change to a model, the model should be validated. That information should be captured in “planned validation activities” in the inventory

  • Description of any model issues or limitations

  • Summary of model issue status (e.g., work in progress, partially completed)

  • Description of any model overlays

  • Indication of whether models are functioning properly

  • Description of when the model was last updated

  • A list of any exceptions to policy

  • Time frame that the model is expected to remain valid

Source: United States, Congress, Office of the Comptroller of the Currency. Comptroller's Handbook, 1.0 ed., OCC, 2021, pp. 27–27. Model Risk Management.

The OCC has comprehensive expectations, whereas the FDIC has historically been less prescriptive. If yours is an FDIC-regulated institution, fewer fields may potentially be sufficient. An institution may track additional fields that it finds helpful for overseeing a risk management program, but those noted above may suffice to operate an effective risk management program.


The information collected from each model provides immediate insight into the model's purpose, use, basic construction, risks, and updates on primary risk management aspects such as validations, exceptions to policy, and an indication of the model's overall performance. By reviewing the model inventory, management can quickly identify which models are out of compliance with the risk management program, are developing issues, or may need expanded review to mitigate any emerging risks.


The model inventory is only valuable when it is up to date and contains accurate information from a practical standpoint. All models under development, in use, or recently retired should be tracked. The model inventory should be the responsibility of a sole position or person at the institution rather than a group or committee for accountability reasons. It is often easier to hold a singular person accountable for inventory maintenance than a group. CECL helped bring the importance of model inventories to the forefront.


Resolution #4: Risk Rate All Models (CECL Is a High-Risk Model!)


Once an institution has identified all of its models, it should prioritize them, noting which models pose the most model risk. This is done through a risk assessment process. When constructing a risk assessment process, several aspects of the model that affect its overall risk level must be evaluated. The risk assessment should consider the model's use and its materiality on the organization. The assessment should consider the amount of uncertainty generated by the model's inputs, outputs, or processing components. If the model is part of a suite of models, that may affect its overall risk to the institution. Other aspects to consider are the model's limitations, the frequency of use, the complexity of the model, the resource requirements of the model, and the expandability of the underlying theory/algorithm that drives the model.


There are multiple ways to create a risk assessment process. Some institutions use a definition-based approach by creating definitions for each risk level (typically high, medium, and low), highlighting aspects of the model that categorize it at each level. The institutions that implement this approach tend to use a committee to review each model, in conjunction with the model owner, to arrive at a final risk rating. The process is documented through minutes, so outsiders such as regulators or validators can understand how the rating was determined. Others decided to use a purely quantitative approach for risk rating, utilizing a scorecard or template. This approach takes the various risk categories (model complexity, use, operational impact, etc.) and applies ratings and weightings to arrive at a quantitative risk rating for each model.


Regardless of how an institution risk rates its models, the process should be easily understood and repeatable. The model risk management policy or procedures manual should describe in detail how the risk rating is performed, the criteria, and exception handling. Risk ratings help inform the frequency and depth of model risk management practices, such as the frequency and rigor of model validations, management oversight, and depth of the effective challenge. That said, CECL is a high-risk model, so validation should be on your radar for 2023!


Resolution #5: Create a Model Risk Management Program


Once models and their associated risks are identified, the institution can begin constructing a model risk management program proportional to that risk, the complexity of the institution, and its culture.


The policies (and, if applicable, accompanying procedures documents) should outline the governing structure, roles, responsibilities, model inventory standards, processes, and the risk rating process that has been determined up to this point. The policy and procedures should then describe the criteria and processes related to controls, data management, documentation, performance monitoring, development, implementation, model use, validation, exceptions, and reporting. A model risk management policy should include the definition of the model and model risk.


As mentioned earlier, those organizations that had an MRM program in place had an easier time implementing CECL. The standards in a proper MRM program help ensure proper model development, adequate documentation, testing, controls, and proper governance. These items are necessary to pass a model validation and effectively use the model’s results. MRM, when structured appropriately, can provide significant benefits for your institution.


BONUS Resolution: Don’t Be Afraid to Ask for Help


As CECL continues into its next phase, take a moment to evaluate where you are and what remains to be done. As always, DCG stands ready to help you along the journey, whether it’s helping to finalize CECL implementations, providing validation, or creating an MRM program right-sized for your organization.


For additional perspective on CECL and MRM, please visit DCG’s MRM webinar archive or our CECL resources.


For more information, contact Darling Consulting Group at: DarlingConsulting.com/contact

 

ABOUT THE AUTHOR

Hannah Glasrud

Risk Management Consultant, Darling Consulting Group

Hannah is a Risk Management Consultant at Darling Consulting Group, working primarily in the realm of model validation and model risk management consulting. Throughout her career, she has worked with various models, including CECL, DFAST, capital, credit, and default rate models. She also performs Model Risk Management process reviews and consulting engagements.


Hannah began her career at DCG in the Data Analytics Group as a business systems analyst, performing prepayment and deposit studies for community banks. From there, she was promoted to Quantitative Analyst, where she assisted in model validations and model risk management engagements.


Using her technical knowledge and coding experience, she has and continues to produce automated reporting tools for various areas within the company and performs code reviews for models written in SAS, R, and Python.


Hannah earned her Bachelor’s degree in economics from Boston University. She has also earned a MicroMasters Certificate from Georgia Tech in analytics: essential tools and methods.


 

© 2023 Darling Consulting Group, Inc.

Comments


DCG Insights

Stay up to date on the latest from DCG

bottom of page