Carolina Biagini - Senior Quantitative Analyst - Kevin D. Oden & Associates

Amul Bhatia - Partner - Kevin D. Oden & Associates

In last week’s post, we focused on establishing a strong foundation for a model risk management framework to determine roles, responsibilities, policies, and procedures. This week we will focus on properly identifying and maintaining your model inventory. Majority of banking activities are driven by automated processes, both internally developed and purchased from vendors. It is critical for Credit Unions (CUs) to understand when and where models are being utilized in their banking activities. This allows for proper compliance with regulatory guidance as well as improved utilization of new AI/ML systems. However, how does one manage risk around an AI/ML system that contains numerous processes underneath it? Too little risk management can lead to regulatory issues or financial losses due to not understanding where the system does not perform properly. Too much risk management can lead to wasted resources, increased costs that lead to less profits. CUs can establish a risk-based approach by formalizing a model definition and establishing a robust model identification and risk tiering process. Ultimately, a firm-wide model inventory supports model risk management throughout the model life cycle.

Model Definition

To identify models, you must have an idea of what you are looking for. So, the first step is to define what a model is. It is reasonable to start with a broadly adopted definition outlined by SR-11-7 guidance:

“Model refers to a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates.”[1]

The guidance goes on to say that a model consists of three components: an information input component, which delivers assumptions and data to the model; a processing component, which transforms inputs into estimates; and a reporting component, which translates the estimates into useful business information.

Though this is a good place to start for your MRM governance document, it is useful for all CUs, in particular smaller CUs, to augment the model definition with so-called “walking around questions.” These can be used to help potential model owners and users know the key differentiators between End User Computing Tools (EUCTs) or simply “calculators” and models.  The questions are the following:

• In addition to data, does the tool require assumptions as a key input? Could a different individual, business unit, or institution have different assumptions?
• Are those assumptions involved in combining the data? Are there various choices in the combination of the data or processing component that could vary from person to person or firm to firm?

Is there uncertainty in the output of the tool? That is, if it produces a (proposed) decision or a number, could someone starting with the same data arrive at a different decision or a different outcome?

These questions tackle three parts of a model and begin to clearly delineate between what could be a model and what may be an EUCT. For example, many calculations performed in the finance group fall short of being a model when viewed in light of these questions. Such as depreciation calculations which are mandated by finance accounting rules and do not require assumptions (e.g., possibly one-time accounting choices). Any firm would calculate the same depreciation on a piece of equipment purchased at the same time for the same amount with the same accounting regime. That is, there are no assumptions, no choices in the combination of the data, and no uncertainty in the outcome.   

We observe, very importantly, that these questions (and the model definition) do not say anything about where the tool is implemented. It is the concept that makes a model, not where it is implemented. So, a model can be implemented in Excel or nowhere at all!  If a person uses pencil and paper every time they “run” the model, it is still a model.  

Figure 1: Model Definition Tree

Source: KDOA

Now that we have ironed out the definition of a model, to meet our next set of risk appetite statement objectives we must identify and risk-assess (or rate) the models of the credit union.

Model Identification and Risk Rating

Model Identification Process

Model identification is an important starting point in the model lifecycle. In this stage, CUs of all sizes establish the process to identify all models (and non-models) at the institution. This process should be done firm-wide at least annually and when a potential model is planned to be developed or purchased.   

Key to developing a successful process is the following:

1. Clearly established roles and responsibilities (R&R)
2. Training
3. A good questionnaire (remember Voltaire!) covering model identification and potential risk ratings

A method for retaining and utilizing the information (a model inventory).

Let us take each of these in turn, with model inventory having a section of its own discussed later. First, we discuss R&R . An identification policy should at minimum define roles broadly across three groups: Enterprise Risk Management,  Model Owners, and Model Users. Best practice is to include a fourth group: Audit. The R&R assigned to these four groups should include (at minimum) the following:

• Enterprise Risk Management (ERM) develops the model definition and provides training on the model identification and risk rating process. ERM also Reviews and approves identified models and tools determined to be non-models. The head of MRM will also maintain the model inventory.
• Model Owners ensure models are properly identified, risk rated, validated, and used for their intended purpose. Compliance with the MRM policy is achieved by presenting tools to be assessed and assigning model users and competent individuals that will ensure the model remains in compliance with policy.
• Model User(s) are responsible for the appropriate use of the model and the escalation of any performance degradation. Additionally, they share responsibility to ensure the model is validated on a timely basis in compliance with this policy.
• Audit reviews the model identification approach designed and executed by ERM to ensure it meets requirements set out by the credit union’s policies and it aligns with regulatory requirements.

These R&Rs can also be associated with the three lines of defense framework. However, this can often be confusing, in particular at small institutions  where individuals wear many “hats” depending on their day-to-day activities and they are not sure which line they are in.  It is often clearer if R&R are defined in terms of the four principles players outlined above. A key to CUs having an effective model identification process (and more broadly an effective MRM program) is not focusing on whether individuals are in a certain “line of defense” at the start of the exercise. The focus should be on the R&R they need to play in the MRM process from a Model Owner, Model User, Oversight, and Audit perspective. The rest will sort itself out.

CUs can leverage existing model identification processes but augmented with specific considerations for AI/ML models such as key characteristics and capabilities. As this kind of models can be embedded within software applications and processes, identifying them may be difficult in certain circumstances. Clear and consistent standards of what constitutes an AI/ML model should be adopted across the firm. ERM should provide training to AI/ML model developers and users so that they fully understand what needs to be identified and reported.

Model Risk Rating

Once a tool is identified as a model by the institution it is important to measure the risk of the model to the CU. Like any other product used or distributed by the firm, the use (or lack-thereof) of models comes with risk that needs to be understood, ranked-ordered as best as possible, and mitigated to conform with the risk appetite of the institution. SR-11-7 defines model risk as:  

“The potential for adverse consequences from decisions based on incorrect or misused model outputs and reports. Model risk can lead to financial loss, poor business and strategic decision making, or damage to a bank's reputation”.

The two primary sources of model risk are due to:

• The model may have fundamental errors and may produce inaccurate outputs when viewed against the design objective and intended business uses, and
• The model may be used incorrectly or inappropriately.

A good starting point to rank-ordering model risk is to consider:

1. A simple three-tier approach – for example high, medium, and low-risk models
2. Separating (individual) model risk into three components: Inherent, Secondary (Intrinsic), and Residual risks.

We start by defining the components in (2) and then illustrate how they can be utilized to rank- order model risk in the spirit of (1).

Inherent Risk

 Inherent model risk, as per its name, is risk that is difficult to change or mitigate due to the ”nature” of the model. This difficulty may be due to the use of the model or the type of model, but the two largest components of Inherent risk are typically:

• The model’s role in critical decision making.
• Financial exposure.

Models that are used in critical decision making have the potential for high Inherent risk.  These are models that are used to make credit decisions, for pricing, to manage interest rates or the balance sheet, that have client impact, or are used to meet regulatory reporting requirements. 

Financial exposure is typically defined in terms of the size of the potential loss either due to direct financial loss (e.g., mispricing of credit) or in terms of poor business or strategic decision making (e.g., capital allocation).  This is most often the largest consideration in the Inherent risk category and can be categorized by the dollar exposure of the portfolio, products, or clients the model is used to manage. 

It is also important to consider the impact and degree of the model’s importance in critical decision making to assess its true financial exposure. Even when models are used to manage risk for similar portfolios at different credit unions, the risk to the credit union can vary depending on the role of that model in the decision-making process. The example that follows illustrates how the risk for two identical models can dramatically affect their “exposure” or materiality to the credit union:

1. Credit Union A: Uses vendor Consumer Credit Approval Model in an automated fashion to review and decide on consumer loans. The process is completely automated to have “borderline” declines reviewed by a credit officer for further review and potential approval.
2. Credit Union B: Uses vendor Consumer Credit Approval Model as one component of the credit officer’s review process for each loan application. The officer looks at ten risk factors where the model output is just one of those ten equally weighted factors, with most other factors being expert-judgment driven.

In these two very real examples the financial and reputational impact of the same model is vastly different based on its degree of use in critical business decisions.

It is worth noting that, increasingly, CUs are considering the knock-on effect of reputational damage, which could have long-term earnings impacts. In fact, reputational impacts from the misuse of models could present some of the most serious challenges to CUs going forward.  For example, recently, the reputational issues due to regulatory fall out from poor BSA/AML models or fair-lending practices have been problematic for institutions large and small.[2]  In some cases, these problems have even led to the regulatory-enforced delay of growth activity, like acquisitions.[3]  This is not to say that reputational impact should be the largest factor in determining the amount of risk associated with a model, but that it should not be forgotten in the process. For these reasons, credit unions also frequently consider two additional factors in the Inherent risk category:

• Non-financial exposure (including reputational risk)
• Regulatory risk 

Intrinsic (Secondary) Risk

The term can be defined slightly differently by market practitioners,  but there is increasing recognition at smaller institutions that the management of Intrinsic risk is critical to successful model risk management. Some of the items commonly considered in model Intrinsic risk are the following:

• Data or Inputs: Quality, stability over time, inclusive of output from upstream models and their risk tier.
• Complexity: Is this model well-known? Has it been peer-reviewed and used in the industry for years? Or not? Does the model rely on several different assumptions, each of which must tie together to make a logical whole? Or is it based on one relatively simple principle?
• Theory: Is the underlying theory well-known or intuitive? This also could have a “maturity” aspect. Has the model theory been in practice in the industry for a long time?
• Performance: Has the model proven accurate (predictive) in the past or in back testing? Have the results had wide dispersion?
• Implementation: Is the model easy to implement and run? Is it implemented in a system that has several control features, ranging from approved users, change controls, etc.?

Often the Inherent and Intrinsic components are combined into a single risk score which may collectively be called the  model risk ranking or rating. In either case, for many institutions this exercise leads to the final risk rating for the model.  A schematic of the proposed risk tiering outlined above is depicted in Figure 2, which follows a scoring methodology where model risk rating is assessed in terms of the two dimensions described above: inherent and intrinsic risk.

Figure 2: Risk Scoring Methodology

Source: KDOA

Residual Risk

Other institutions use the Residual risk to either derive a final risk ranking/score or used to complement the risk ranking. As just noted, the amount of risk associated with a model, without incorporating any mitigating controls, is referred to as either the model risk rating or score or, more correctly, as the sum of Inherent risk and Intrinsic risk.  As noted earlier, most of a model’s Inherent risk comes from its exposure to financial loss. However, a model which has a large exposure may be more or less risky than a comparable model with similar exposures due to other factors, including input or data quality (including reliability) issues, model complexity, or implementation of the model (Intrinsic risk).  Many of these Intrinsic (or secondary) risks can be “nurtured” or mitigated to reduce the total model risk. The net effect after mitigating these Intrinsic risks is a lower Residual model risk. That leads us to our final component definition. 

Even when model risk is large due to Intrinsic risk factors, there may be controls that reduce the overall risk of the model. What’s left when these controls are introduced and properly implemented is known as Residual risk. Most controls are placed around the Intrinsic risk factors but (dynamic) exposure controls can be utilized as well to reduce the Inherent risk at times. Typically, all these controls fall under the broad heading of governance. If there is a strong governance framework for models this, by its very nature, decreases model risk individually and collectively. Below we list the usual controls that are typically utilized to reduce Intrinsic risk.

1. Data/input: Review and cleansing of input data, including the definition, review, and removal and monitoring of the frequency and degree of outliers. 
2. Performance Monitoring: Clear guidelines related to good performance versus bad performance (or questionable performance). This is usually developed during model development and implemented to guard against performance degradation. Models with large exposure (inherent risk) that have large performance variance need to be monitored more frequently to minimize Residual risk
3. Usage Monitoring: Inappropriate use is one of the risk factors identified in SR-11-7 guidance.[4] Appropriate monitoring of usage and changes of usage with appropriate review before those changes take place can mitigate usage risk.
4. Reporting: The appropriateness, ease of use, and interpretation of the model outcomes in reporting are critical to appropriate and risk-controlled use.
5. Exposure control: In some cases, exposure can be dynamically decreased based on risk factors both internal and external.
6. Governance framework: At most institutions, the existence of a sound governance framework is recognized as a risk mitigant for some or all models.

The degree to which each of these components has documented, verifiable frameworks in place to control the risk presented can provide reduction in the Intrinsic risk of the model. This can be used to produce a total risk score or risk tier. Some of these elements can only be introduced after the model is in production (e.g., performance monitoring) and provide no risk relief when a model is initially validated.  

The following figure depicts a three-dimension tree approach proposed by Aruna Joshi (2017).[5] The first dimension assessed is the model’s use in terms of risk measuring, pricing, valuation, and support to critical business decisions or financial reporting. Then, the exposure level is assessed to reach a preliminary model Risk Rating. Finally, expert judgment will guide the final risk rating that will consider residual risk in the case of higher-risk tier models.

Figure 3: Risk Ranking Decision Tree

Source: Aruna Joshi (2017)

Model risk frameworks should be enhanced to incorporate AI/ML attributes. When assessing the intrinsic risk of AI/ML model the following aspects of model complexity should be considered: large volumes of structured and unstructured data, frequency of retraining, opacity of the algorithm, number of hyper-parameters, reliance on open-source code and interrelationships with other risks. Additional considerations to include in the risk assessment are related to ethical and social implications (potential for bias and lack of fairness). The risk of certain AI/ML models should be (re)assessed on an ongoing basis as the context, capabilities, benefits and potential impacts of the model evolve over time.

Model Inventory

Once the models are identified and risk rated, they need to be inventoried. You may ask, why does my credit union need an inventory, and does the system need to be complex?

The first answer is every credit union needs a model inventory. This is because the models in use at the firm, their performance, limitations, users, and even owners change over time. Also, there are required model updates and changes that need to be tracked to ensure timely resolution of issues and to identify model rollbacks when new model changes do not go as expected.  The model inventory is a dynamic risk management tool which evolves with the model risk cycle, the risk environment, and changes in the model and its usage.

A model should be included in the inventory when it is first proposed, whether it will be built in house or purchased from a vendor. This will enable the timely tracking of model development/acquisition milestones like documentation, model validation, and model monitoring. 

As for the second question, the complexity and the technology behind the inventory should be commensurate with the extent and risk of model usage at the institution. Most CUs can start with an Excel spreadsheet to prototype and manage the model inventory before building or buying more advanced tools. Remember: If you know the risks you want to manage and how to manage them, you are in a better position to evaluate third-party tools. This will prevent the wasteful out-of-system modifications when the tool does not work in line with the CU’s needs.

Though the fields of the inventory should fit the idiosyncratic needs of the credit union, the following field groups have become known as best practice: 

• General model information (model ID, version, name, status, etc.).
• Model development information (relevant dates, application environment, model type, model purpose, information about model owners and developers including name, business unit or group, whether internally or externally developed, model inputs and sources, model outputs, etc.).
• Model validation information including performance monitoring (model risk rating, validation status, approval conditions, model use limitations, data limitations, policy exceptions granted, model validator, whether internally or externally validated, relevant dates related to performed activities and incoming validation, annual review, and monitoring activities).
• Implementation information (dates, last and next review).
• Approved model uses (relevant dates, model users and business units/groups).
• Attestation from model owners and users as to appropriate use and policy compliance.
• Retirement information (dates, reasons).

It is expected that regulators will require most AI/ML models be included in MRM’s model inventory, given their historical posture toward the inclusion of new quantitative approaches. Given the dynamic nature of AI/ML models, ERM should periodically review the AI/ML model pipeline to confirm that the inventory remains accurate and complete.

Conclusion

Model risk increases with greater model complexity, higher uncertainty about inputs and assumptions, broader extent of use, and larger potential impact. Hence, identifying models used in every line of business or function, risk rating those models, and managing their individual and aggregated risks are essential components to a sound model risk framework. This article aims at guiding CUs establishing a risk-based management approach that starts by  a model definition consistently and broadly applied across the institution. It follows with clear guidance to implement a robust and comprehensive process to identify all models and non-models used at the CU. A model risk rating approach helps to allocate resources appropriately and to prioritize validation activities.  Ultimately,  a firm-wide and up-to-date inventory of all models used at the CU is crucial to assess and manage model risk in the aggregate.

[1] Supervisory Guidance on Model Risk Management, SR-11-7, Federal Reserve Board. April 2011..

[2] https://www.fdic.gov/news/press-releases/2020/pr20091a.pdf   One key component is the requirement to conduct “ongoing monitoring to identify and report suspicious transactions”, which may result in currency transaction reports (CTRs) and suspicious activity reports (SARs).  For many institutions this analysis is in large part performed by 3^rd-party models which are also utilized in fraud detection.

[3] https://www.spglobal.com/marketintelligence/en/news-insights/trending/oo7exse5idfrzbhdgd6ikg2 “Buffalo, N.Y.-based M&T Bank Corp. was forced to delay its acquisition of Hudson City Bancorp Inc. for over three years after regulators unearthed BSA issues in 2013”. This delayed merger has been a cautionary tale of the important of BSA/AML compliance.   

[4] Supervisory Guidance on Model Risk Management, SR-11-7, Federal Reserve Board. April 2011.

[5] Joshi, A. (2017). Managing Risk of Financial Models: A Smart and Simple Guide for the Practitioner.