Carolina Biagini - Senior Quantitative Analyst - Kevin D. Oden & Associates
Amul Bhatia - Partner - Kevin D. Oden & Associates
Introduction
In last week's post, we focused on developing a strong foundation for model risk management. This week we will focus on the application of model risk management and show how to build a strong program with proper model development, validation, and monitoring.
Model Development
At many institutions, model risk management starts after a model is developed or acquired from a vendor. This mindset poses real risks to the Credit Union (CU) and should be remedied by incorporating model development into the model risk management process. In other words, treating risk management as an afterthought is an error. It is important for every CU, regardless of its size, to understand the risks it is accepting when it proposes the development or purchase of a model. Therefore, the model development process must be a well-designed process that involves, at minimum, the following stakeholders: model risk management, the model owner, and ultimate model users.
Risk management plays a critical role in the model development and model review processes. The risk management team is the body responsible for identifying the risk practices apply to model management. Risk management will also outline the documentation necessary to ensure the model meets not just regulatory requirements but internal standards as well. If a third party (vendor model) is under consideration, third-party risk management will need to be engaged as another crucial player who will conduct some of the critical up-front negotiations with the vendor(s).
The development process itself
Existing models can be absorbed into the development process. However, we will focus on new models and changes to models. The first step should be identifying the business need for that particular model. The business leader should initially reach out to the model development team, which should then reach out to the risk management committee so that all components of model development or discussions around existing model modifications are properly and safely addressed. An effective model development process has at minimum the following components:
- Purpose and Objective Assessment: This involves the potential model owner or business leader, MRM, and the model developer assessing why the model is needed and its objectives. Note that this should happen before the developer begins working on the potential model or the vendors demonstrate their model. This can simply be a meeting where the model risk management team learns about the proposed model, begins to formulate a risk assessment of the proposed model, and can formulate a validation strategy inclusive of resources. This also commences the model inventory process, with a model ID assigned and model purpose/use field populated.
- Requirements or Expectations: Outlining the model requirements in sufficient detail to benchmark subsequent development or vendor capabilities is essential to ensuring the developed or purchased product is ultimately fit for purpose. Portfolio or business coverage, access or control requirements, and as much as possible performance standards should be considered. This will also help develop the validation strategy.
- Documentation Standards: Developed by MRM and potentially tailored to individual model types, consistent and strong documentation ensures business continuity as developers or users change firms or roles or as new users utilize internally built or vendor models.
- Development Guidelines: In addition to the components outlined above, it is good practice to have development standards. Essential to this will be testing. As SR-11-7 notes: “An integral part of model development is testing, in which the various components of a model and its overall functioning are evaluated to determine whether the model is performing as intended.” Testing is where #2 (Requirements and Expectations) are assessed and why it is important to outline them initially. Testing should check the model’s accuracy, its robustness and stability, and include the impacts of assumptions. Stress testing the model to understand its limitations is also a critical aspect of development testing. These development guidelines can be included in MRM’s documentation standards or published separately.
Importance of business continuity
Well-developed documentation is critical from both a business continuity and risk management perspective. An organization could spend years building out models; however, if the internal teams produce models with little or no documentation, business continuity is in jeopardy. The CU could use those models successfully for several years. However, at some point, the individual or group that built the models moves on, and the firm has no idea how to address problems should they arise because there is no documentation. Imagine a scenario where no CU employees know the codes that implement the models; thus, the institution must hire an individual or company that can reverse engineer the model, costing thousands of dollars, and countless hours. That error in judgment also exposes the CU to a barrage of regulatory and financial risks.
Using an outside vendor for model development
Before reaching out to outside vendors for model development or for a third-party model, the CU should first ask itself if it has the skill set internally to build the model. If the answer is yes, does that staff have the time to produce a model? If the CU does not have the staff, can it still conduct its developing in-house? The CU will need to hire the personnel with this expertise and bring them on board. All of this may be worth the cost because that model is going to demand support and maintenance over the years.
If the CU does not currently have the internal skill or staff to build a model and adding staff to build a model is cost prohibitive, the conclusion may be to use an outside vendor. But the question remains: What is the cost/benefit analysis of developing a model internally versus outsourcing the process?
Securing the services of an outside vendor, although cost effective, may introduce issues with sustainability and quality. How will the vendor meet all your standards? Increasingly, CU often overlook whether portions of a vendor’s model or the model itself may be proprietary. If that is the case, it is imperative that the CU understand that using a model with proprietary data or information does not relieve the CU of its responsibilities. If the model is ineffective and errors occur, you can sue the vendor, but your members and the regulators are focused on the CU, not the vendor. It is crucial that you still diligently manage the risk in the vendor relationship. That means ensuring the CU understands even the proprietary components of the model well enough so the documentation works. CU own the risk of the models they bring in-house. The vendors do not own that risk. Every model owner and user should acknowledge what risk they are owning with the model.
This is certainly not to suggest that using a vendor’s services is inappropriate. What we are emphasizing is that when selecting a vendor, it needs to be clear from the start that the vendor will be required to provide documentation that is in line with the CU’s internal requirements and needs, and that the vendor will be working with the CU’s risk management team. Also ensure the vendor can make required changes to the model going forward and, conversely, if the vendor makes changes to the model’s codes or other components, the CU is given notice and provided reasons why those changes are being made. Ultimately, the CU should evaluate if those changes make sense for the institution and its needs.
AI and ML models
AI/ML models should be developed in a well-controlled environment designed to encourage integrity, traceability and reproducibility of results. This implies controls embedded throughout the model’s life cycle, from data sourcing and pre-processing, model design and construction to implementation, performance assessment and ongoing monitoring. Developmental decisions and model choices should be properly documented and justified, including special AI/ML considerations such as calibration of hyper-parameters, trade-off between predictive accuracy and interpretability of model output, trade-off between algorithmic simplicity and computational intensity, trade-off between model bias and model fit, potential information security and privacy related risks. As in the case of traditional models, developers should perform appropriate tests and document the results.\
Model Oversight
Model Validation
Ensuring all models are validated and fit for use is of increasing concern to CU members, investors and regulators (and appropriately so). Furthermore, the standards around what is considered a strong validation are expanding as the use of models grows and the risks involved become increasingly clear. This is also leading to a more consistent approach to model validation and benchmarks for sound model validation.
One concern confronting small institutions trying to manage their risks appropriately is the frequency of validations. There is no short answer or predetermined time, but high-risk models should be validated and revalidated much more frequently, maybe annually in best practice. Regulators have made it clear that the scope, depth, and rigor of a validation should be commensurate with the scale and complexity of that model in the context of the individual firm. This applies to the frequency of revalidation as well.
As a simple example of how this frequency could differ by firm, we consider the same model alternately employed at a $250 billion asset institution for credit decisioning the entire portfolio and at a $10 billion asset institution where it complements expert judgment to risk manage 50% of a portfolio. The rigor of validation activities should be different, downsized, and rescoped for that smaller institution given the risk profile of the portfolio and the use of the model. In particular, the range of tests and the severity of issues would be different for larger institutions.
The challenge for smaller institutions as the model inventory grows is attracting and retaining the resources to effectively validate models of various types. The expertise requirements vary greatly depending on the model use. As an example, the skills to appropriately validate a retail credit model differ vastly from those required to validate a BSA/AML model or to validate a derivative pricing model. As a result, many smaller institutions need to engage third parties to properly validate some or many of their models. To handle this third-party arrangement effectively, the CU must ensure the third party has the requisite skill set to effectively validate the model. The CU should also baseline expectations in a statement of work (SOW) and expect regular check-ins as the validation progresses.
An effective validation framework should include three core elements:
- Evaluation of conceptual soundness, including developmental evidence.
- Ongoing monitoring, including process verification and benchmarking.
- Outcomes analysis, including back-testing.
Some of the process elements that have become “best practice” for model validations in the industry include:
- Pre-model development (revalidation) meeting
- Used to understand intended model purpose and requirements.
- Assess findings and observations from prior validations and their status.
- Discuss model changes, if any, since last validation.
- Documentation review with developers and business leads
- Answer documentation questions.
- Make owners aware of any deficiencies that may slow the validation process.
- Evaluation of conceptual soundness, including developmental evidence
- Implementation review
- Regular check-ins with developer and business
- Findings, issues, and recommendations
- Validation report
- Ongoing monitoring, including process verification and benchmarking.
The CU should not and cannot rely on the vendor to perform an independent validation and should ensure that any vendor is willing to comply with the CU’s model validation requirements, including sound documentation, performance monitoring, and independent review.
Annual Reviews
While it is crucial to ensure the model continues to perform as expected, continuously validating a model in production can be cumbersome and an unnecessary use of resources. An annual model assessment/review is a critical component of the ongoing validation process. These annual reviews involve a comprehensive evaluation of the models' performance, including checking for any changes in the underlying data, assumptions, or economic conditions that could impact the models' effectiveness.
By conducting annual reviews, CUs can identify and address potential model weaknesses or biases, thus reducing the risk of financial loss or regulatory penalties. Additionally, these reviews help ensure that the models continue to align with the institution's risk appetite and strategic objectives. Regularly updating and approving models through annual reviews also fosters greater confidence among stakeholders, including regulators and members, in the CUs risk management practices.
Model Monitoring
All models should be continually monitored for performance. A monitoring plan outlining expectations should be reviewed as part of model validation and approved by the independent model validation team or model risk manager. The metrics and reporting should initially be based on back-tests performed by the model developer and potentially repeated by the validator and updated over time. Importantly, the degree of monitoring in terms of frequency, resource allocation, etc., should be commensurate with the risk of the model and therefore driven by the risk rating.
The topic of model monitoring may conjure images of fancy systems and armies of teams watching flashing buttons on rows of monitors, but that is not necessary. Good model monitoring occurs over the life of each model, and it can be as simple as ascertaining the model’s performance in regular meetings with senior executives and model owners.
Effective model monitoring includes outlining performance expectations in terms of quantitative limits – for example, using a system of red, amber, and green is common. These should be reviewed on a regular basis (e.g., quarterly or semiannual), depending on the risk level of the model. Best practice is to establish escalation protocols to senior level committees and ultimately the board. However, board level escalations should be reserved for the highest risk models and degradation of performance to severe levels.
Setting up a monitoring framework for a model need not be complex to start. For example, take a credit decision model developed internally or by a vendor where back-testing has demonstrated 90% accuracy in differentiating good from bad credits (along with precision and other metrics). If the model returned 75-80% accuracy, it could be considered in the red or amber level, depending on the CU’s acceptable risk level. The CU would then review the actual credits developed over the past six months or a year, determine the accuracy of the model, and assess whether the model performance is red, amber, or green.
It is key that when the CU establishes model monitoring, there are appropriate actions associated with each level of performance. Those steps should be in writing and well documented in the model policy and procedure. However, as with any policy, there could be exceptions. The following figure describes the components of a strong model monitoring program.
Figure 1: Model monitoring program
Source: KDOA
AI and ML models
Validators should assess the rationale for the use of an AI/ML model as opposed to traditional techniques and whether the model specification is informed by domain expertise and aligns with the business purpose. When assessing the conceptual soundness of AI/ML models, EY (2020) suggests focusing on the following dimensions[1]:
- Data integrity: AI/ML models rely on large volumes of heterogeneous and high-dimensional data. Assessing data quality and appropriateness requires reviewing the entire data lifecycle, from sourcing and pre-processing to training, testing and deployment. Additionally, validators should consider internal policies and regulation related to data privacy, protection, and ownership.
- Feature engineering: Validators should review the process through which input variables are constructed from the raw data, including statistical analysis and business intuition applied to select features.
- Sampling bias and fairness[2]: Validators should evaluate model impact on stakeholders and where necessary, other control functions (such as compliance) may participate in the assessment.
- Hyper-parameter calibration: Validators should evaluate how different parameter settings impact the results of the model and the computational feasibility in production.
- Explainability: Understanding how a model produces outputs based on the input variables and being able to qualitatively interpret model outputs can become challenging for complex AI/ML models (neural networks and ensemble techniques) as the way outputs respond to inputs may be unclear and lack transparency. In other cases, it will be difficult to understand how inputs get transformed into outputs (traceability).
AI/ML model owners and developers should create a comprehensive ongoing monitoring plan to confirm that the model is operating as intended over time. This plan should cover model performance, stability and alignment with business purpose. Performance results should be evaluated in terms of sampling bias and fairness. Model users should play an important role when evaluating model performance, as they are able to observe performance over time and to effectively challenge model results.
Setting an appropriate monitoring frequency for the performance indicators can become challenging for models frequently retrained. Similarly, defining what constitutes a model change and how to assess it can be difficult given the dynamic nature of AI/ML models and the need to manage feature changes through frequent retraining.
Conclusion
The importance of model development documentation cannot be stressed enough. As previously claimed, well-developed documentation is critical from both a business continuity and risk management perspective. The standards for the documentation of vendor models should be the same as for models developed internally.
It should be stressed that model validation is critical to sound risk management whether the model is developed internally or by a vendor. A sound validation process involves a degree of independence from the model developer and model user. To be able to effectively challenge, validators need to have the requisite knowledge, skills, and expertise.
The risk of model performance not meeting internal and reported expectations is large and increasing as members, investors, and regulatory expectations grow. For this reason, developing a model monitoring program inclusive of expectations, roles, and responsibilities is so important for every institution, large and small.
[1] Agarwala, G., et al. (EY 2020). Supervisory expectations and sound model risk management practices for artificial intelligence and machine learning.
[2] Sampling bias occurs when AI/ML models incorrectly and systematically under/over-represent specific groups or classes from a population in a non-random manner. Fairness includes concerns for equality and equity by addressing issues such as harmful bias and discrimination, that can lead to disparate treatment.
If you have any questions or would like to get in touch, feel free to reach out – we're here to help!
Comments