Operational risk modelling and organizational learning in structured finance operations: a Bayesian network approach

Abstract

This paper describes the development of a tool, based on a Bayesian network model, that provides posteriori predictions of operational risk events, aggregate operational loss distributions, and Operational Value-at-Risk, for a structured finance operations unit located within one of Australia's major banks. The Bayesian network, based on a previously developed causal framework, has been designed to model the smaller and more frequent, attritional operational loss events. Given the limited availability of risk factor event information and operational loss data, we rely on the elicitation of subjective probabilities, sourced from domain experts. Parameter sensitivity analysis is performed to validate and check the model's robustness against the beliefs of risk management and operational staff. To ensure that the domain's evolving risk profile is captured through time, a formal approach to organizational learning is investigated that employs the automatic parameter adaption features of the Bayesian network model. A hypothetical case study is then described to demonstrate model adaption and the application of the tool to operational loss forecasting by a business unit risk manager.

Keywords:

• banking
• finance
• operational risk
• probabilistic methods
• artificial intelligence
• Bayesian networks

Acknowledgements

The authors would like to acknowledge the financial support provided through two research grants awarded by the Department of Accounting and Finance, Monash University, and the Melbourne Centre for Financial Studies. The second author is also supported by an ARC Discovery grant for which he is grateful.

Notes

1 We use a manipulability account of causation whereby, if one intervenes within a system to change the state of an object A, and this results in a change in the state of another object B, then A's state is said to cause object B's state. However, if one intervenes within a system to change the state of an object B, but object A's state remains unchanged, then object B's state does not cause object A's state.

2 To distinguish between the marginal probabilities inferred from the network, and the conditional probabilities stored in the CPTs, marginal and conditional probabilities are denoted as Pr(…), and p(…), respectively.

3 Although the Bayesian network model development tool does come with automated network construction and parameter estimation functionality, absence of historical domain data makes these automated facilities unavailable.

4 The Bayesian network development tool used for the research was HUGIN Researcher^TM v7.0, published by HUGIN EXPERT A/S website: www.hugin.com

5 An extensive, detailed description document—containing all node definitions, states, probabilities, as well as experience and fading table settings—is available on request from the corresponding author.

6 The background transaction volumes (ie 1000 transactions) used in the elicitation questions exceeded the current transaction volume history of SFO. The motivation for designing the questions around such large hypothetical loan transaction volumes was to encourage domain experts to base their responses on their beliefs on the long-term averages within the domain.

7 At the time of the models development, 300 transactions was the risk manager's estimate of the number of transactions processed by SFO during its operational period.

8 Care should be taken when interpreting perceived deviations of the model's marginal probabilities from any observed domain events, as any apparent deviations may be due purely to random variation rather than from any misspecification of the node. This important point was highlighted by an anonymous referee in his/her comments on an earlier version of the paper.

9 The rational for the Dirichlet distribution is discussed in the parameter adaption section.

10 Owing to the proprietary nature of the parameter tuning facility, underlying implementation details are not available.

11 The term ‘d-connected’ refers to the relation between nodes within a DAG. If two or more nodes are d-connected, then changes in one nodes state will affect the marginal probabilities of states for the other nodes. In this situation we are referring to changes in parameter values. In this context d-connected means that if we consider the parameters to the node, to be actual auxiliary parent nodes, then those auxiliary nodes must be d-connected.

12 The proofs for Equations (2) and (3) can be found in Castillo et al (1995), with a detailed discussion of their application in Castillo et al (1997), van der Gaag et al (2007) and Kjærulff and Madsen (2008). A systematic approach to sensitivity analysis is also described in Bednarski et al (2004).

13 The adaption of the network model's CPT parameters involves using the sequential adaptation algorithm to incorporating new event data as required. A database was also developed to support the adaption process, capturing operational events relevant to the model's adaption. A document detailing the database design is available from the corresponding author.

14 The uncertainty levels are ‘very high’, ‘high’, ‘medium’, and ‘low’ uncertainty, to which the knowledge engineer assigned equivalent experience values of 50, 100, 200, and 400, respectively.

15 The data used in the hypothetical case study were produced by simulating event records from the initial a priori Bayesian network model after evidence on transaction characteristics, human errors, error types, and operational event failures had been entered into the network. Subsequently, to reflect the changing risk profile of SFO, 10 of the generated loan transactions were modified to record payment failure events (caused by transaction implementation errors and oversight control errors) thus resulting in an apparent increase in these events.

16 It was also model B₀ that was used to generate the hypothetical 116 event records for use in the hypothetical case study.

17 The number of 100 transaction events per history approximated the expected count over a single year of SFO loan transaction activity.