Effective Challenge: Necessary Evil or Valuable Opportunity?

MRM Framework | MRM Performance Monitoring | Model Validation

Everyone has that one friend, relative or neighbor who always seems to ask the one question you don’t have an answer for. Though we appreciate and, in some cases may even love these folks, that lingering question can be pretty irritating. Even worse, it sometimes feels like a simple, off-the-cuff chat could easily morph into an extended interview that leaves you wondering if you even understood what you were talking about in the first place. While modelers and business line managers may not necessarily love their risk managers, examiners, or model validators, these are the folks who have been asking you about “effective challenge” in the modeling process (i.e., the one question you may not have an answer for) over the years. As it turns out, they have been doing you a favor.

Supervisory Guidance identifies effective challenge as a bedrock principle of model risk management and defines it as a “critical analysis by objective, informed parties who can identify model limitations and assumptions and produce appropriate changes.” But what does that really mean? It means you need someone to ask tough questions about the model you built (or the one you rely on) to make critical business decisions. In general, any financial institution that uses a model, no matter how simple or complex it is, should have a process for providing effective challenge to that model. The process is often performed in- house by individuals and/or committees with relevant expertise, independence and authority. This could be Audit, Risk Management, a Management Committee, or some combination of these groups. Some financial institutions also engage a third-party validator to provide an independent analysis of a model in support of the effective challenge effort. As expectations for model risk management have increased over time, so hav