Model Risk: it’s not just what you do, it’s the way that you do it

At its core Model Risk Management (MRM) enables firms to make good decisions based on the best possible information. Models and their use cases are dynamic and varied, and we should expect to see the full range of simplicity and complexity within any sizeable financial services organisation. For firms to be successful in managing model risk, it is considered paramount to ensure sufficient time is allocated in assessing the validity of insights derived from the models that ultimately underpin the value on resulting business decisions. 

Driven by continued increase of big data, access to working artificial intelligence (AI), advancements in generative AI and machine learning (ML), the need to manage model risk has come into focus. AI models, possibly more-so than traditional statistical and ML models, are more complex in design, in required infrastructure to operate, in how they are being used and controlled. With expanding possibilities in using such powerful tools the risk of misuse, of uncontrolled use or use that result in harm to customers increases the potential for significant risk to firm reputation. In short – the risks over model use are rising. In recognising this concern Regulators have begun to set out their expectations. The PRA model risk management principles for banks (SS1/23) comes into force on 17th May 2024. From discussions with the regulator, we understand more guidance will be forthcoming with specific publications for AI and ML as well as expected revised Stress Testing Guidelines. 

Whilst firms grapple to meet an increasing onerous set of requirements, model risk is not new. Such standards have been required for regulatory capital models for some time and as such model risk frameworks are widely embedded within most internal model-based organisations. What is changing is the scope of regulatory requirements which now reaches into non-prudential areas where models are relatively new and model risk definitions and controls won’t be understood or acknowledged. The principles introduced by the Prudential Regulatory Authority (PRA) for model risk management reinforce existing industry good practises. The intent of the regulation is primarily to produce the required focus from senior management and boards to improve firms’ culture in MRM.   

Without doubt firms have invested in capabilities and model risk frameworks to manage this risk. However, our recent industry roundtables with many UK lenders’ model risk oversight functions indicate they are having to continue to convince business management of the role, impact and risk models play in decision making and the merit of ensuring and assessing the reliability of their performance. SS1/23 highlights the importance of understanding third party models and deterministic quantitative models (DQM) together with models outside the traditional areas of credit and capital modelling, and it is an imperative for firms to create and embed a culture that is aware of the risk represented by such models and how they might impact the business they operate.   

From our own experience working with many lenders and discussing these matters, we have observed some helpful ways firms can achieve the desired model risk management culture and gain buy-in from senior management and the broader business:    

Gain clear recognition of model risk: The first step in embedding model risk management culture is recognition of its importance. Therefore, model risk management teams must create awareness. An effective way to do this is to invest in education of senior management and to continue that education at regular intervals during key moments of consideration. Firms have a reliance on the board to drive a principle based top-down model risk centric culture. This can be achieved through papers to senior committees but also more broadly through informal sessions as well as mandatory training. Materials do not need to be highly technical, so they are able to be comprehended and accessible to the intended audience for the purpose of executing their duties. The introduction of model risk champions in the business can also be an effective way to create localised specialists with related objectives.

Drive the right behaviours that manage model risk: Secondly, organisations should look to develop business incentives that align and produce the desired behaviours. This is challenging to balance commercial and risk objectives, but rewarding positive outcomes from strong controls, good culture, evidencing how understanding of models and limitations and assumptions strengthen the quality of governance are ways to reinforce this. SS 1/23 may lead to reviews and change to overarching frameworks and policy to manage model risk. Actions that align to the policy and controls that manage these risk that help to support commercial plans, should be a focal point of rewarding and incentivising. Improved reporting of model risk, with a robust model inventory helps to make this easier for how the business can operate and be governed. It’s often been said that ‘what gets reported gets done’ and highlighting successes and failures through reporting of risks by model owners helps create and support a transparent culture. Firms traditionally place the onus on first line to own risks, with management able to ‘risk accept’ model risks. A stronger control we are seeing introduced allows second line a mandate to remove from use models that are not fit for purpose. Having this capability incentivises the business to engage and address model issues in a timely manner.

Communicate openly on model risk: Communicate benefits and demonstrate the cultural change, again, and again, and again. A true change in culture cannot be achieved through a one-off exercise. People need to see the benefits and the change must be consistently applied so that people know that this is the new norm. Consistency breeds success and model risk practitioners must be committed to maintaining the message and culture over time. 

A demonstration of successful cultural change is an observable shift in behaviours. If the business can see consistency and tangible benefits through improved decisions and results, this will fuel proactive engagement. A genuine desire to understand model risk through effective validations will result in an investigatory mentality with senior management asking informed questions of their models, its inputs and application. Models will be challenged not just by model risk oversight practitioners but by those with value to add outside the traditional skills base. As an example, someone with a legal background may be better placed to validate a compliance model.​ 

The regulatory rules on model risk are not finalised, and a broad package of further supervisory requirements should be expected. Establishing a positive model risk culture is a key pillar to successful model risk management. 

For help with your Model Risk Management, S1/23 or to see what tools we have to aid you and your firm please contact 4most at info@4-most.co.uk.