Things to think of in model use – part 1 in a series of 2 blogs

Things to Think of in Model use – part 1 in a series of 2 blogs

In today’s financial industry widespread model use has been the standard for years. From increased competition, cost cuttings, modernization and a general tougher business climate including tighter regulations from authorities, also other industries have increasingly been relying on models in their daily work. Over the past few years there has perhaps been a slight shift back from maybe too complicated to understand model outputs relying on sometimes non-intuitive assumptions and to simply “pass the elevator test”. This blog shed some general light on model use.

What is a model?

Models are an abstraction and simplification of reality. They are next to never more complex than “the real thing”. Models attempt to describe the reality, but are NOT the reality. They should be understood as a guide to make decisions in an uncertain environment. Never mix up a model with the world itself in other worlds. In model use, it is thus essential to have a critical reflection about the assumptions used in model and be aware of model risk

I will remember that I did not make the world, and it does not satisfy my equations, though I will use model boldly to estimate values, I will not be overly impressed by mathematics I will never sacrifice reality for elegance without explaining why I have done so nor will I give the people who use my model false comfort about its accuracy.

Instead, I will make explicit its assumptions and oversights, I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension

Emmanuel Derman & Paul Wimott, 07 Jan 2009

A model should in general both offer:

Example of a model process:

The key question: among the myriad of influencing variables, what are the key material and primary risk factors and how do they influence the use of the model ?

Effective use is an ongoing loop of recurring tasks:

A typical model lifecycle:

… and eventually: scrap or replacement of model.

Objectives in the model lifecycle framework are an understanding of model risk sources and how to measure these. The consequences of using the model need to be assessed in light of the model risks, prior to use. Again: Never forget that model is not reality but a conscious simplified version of reality. Simplification consists in selecting key explanatory variables and identifying limits of the model.

Model design:

The key components of a model consist of input such as market data, key factors definitions, methodological and statistical assumptions as well as IT feeding processes. A calculation engine suited for the complexity and desired frequencies of the calculations then can output valuation levels, principal component analysis, probability distributions, mark-to-model vs mark-to-market values as well as the expected level of uncertainty. There should be a repeating model testing and reshaping process in place.

Model risk:

Different types of model risk are typically data problems, model misspecifications and/or inefficiency and wrong use (user risk).

Data problems:

For data problems, key considerations are relevance vs obsolescence of data, data corruption, data filtering i.e. signal vs noise, selection of appropriate data to minimize selection bias, data availability vs costs (information cost vs existence of proxies), appropriate feeding of the data in the calculation engine an well as uncertainty of data.

Model misspecification and inefficiency:

Specification and efficiency should be based on three key aspects: 1) appropriate assumptions based on empirical behaviors, 2) selection of variables with identification of correlations amongst them, 3) robustness over time of the explanatory power (R2), both in-sample and out-of sample qualities.

Wrong use:

Models need to be used for the right task otherwise its conclusions can be meaningless if it is not the case (for example using an equity risk model to predict risk in a fixed income portfolio). Further a review of the outputs and its analysis with a critical view is crucial. How sensitive is the model to assumptions and how accurate or correct are our inputted assumptions? The output should rather be a range than an exact figure. And finally the model output is not the reality but an estimate of reality should our assumptions hold.

The information, views, and opinions expressed in this blog are solely those of the author in person and generally do not reflect the views and opinions of SKAGEN Funds.