Multi-class misclassification cost matrix for credit ratings in peer-to-peer lending

Abstract

Online peer-to-peer (P2P) lending is a new form of loans. Different from traditional banks, lenders provide loans to borrowers directly through P2P platforms. Since many P2P loans are unsecured personal loans, credit rating of loans is vital to control default risk and improve profit for lenders and platforms. Standard binary classifiers are inappropriate in P2P lending because there are multiple credit classes and misclassification costs vary largely across classes in P2P lending. Though there are a few works that studied cost-sensitive classifiers in P2P lending, none of them have analyzed this issue from the perspective of multi-class classifications and measured misclassification costs of different credit grades using real losses and opportunity costs. The objective of this paper is to model credit rating in P2P lending as a cost-sensitive multi-class classification problem. We proposed a misclassification cost matrix for P2P credit grading with a set of equations and models to calculate the costs. An experiment using publicly available data from Lending Club was conducted to validate the usefulness of the proposed misclassification cost matrix. The results showed that the cost-sensitive classifiers can significantly reduce the total cost, which is essential for the survival and profitability of P2P platforms.

Keywords:

• Cost-sensitive learning
• P2P lending
• credit rating
• misclassification cost matrix
• data mining

1. Introduction

In the past decade, online peer-to-peer (P2P) lending, as a popular form of personal loan, has emerged in credit market. It transfers traditional way of face-to-face personal loans through online services (Bachmann et al., 2011). P2P lending is an electronic marketplace where individual lenders provide loans to individual borrowers. It is pervasive, convenient, efficient, and low-cost without the involvement of traditional financial institutions (Guo, Zhou, Luo, Liu, & Xiong, 2016).

Since the first lending platform Zopa was established in UK in February 2005, an increasing number of P2P lending platforms, such as Prosper, Smava, and Lending Club, have been developed all around the world (Ge, Feng, Gu, & Zhang, 2017) and accumulated data and management experiences. Comparing with traditional banking systems, P2P lending has some characteristics. First, P2P platforms facilitate transactions by connecting borrowers and lenders directly. Borrowers fill in electronic loan application forms, including amounts, terms, purposes, and personal information (such as age, job, address, and credit card). Platforms provide available financial situations and credit histories of borrowers to lenders, who will decide whether to grant a loan and an interest rate. Platforms use various approaches to help lenders set interest rates. Some platforms carry out an auction at which a borrower set her/his maximum interest rate and lenders give their bids (Galloway, 2009). Another approach is to assign interest rates automatically using borrowers’ credit grades, which are calculated based on borrowers’ characteristics (Collier & Hampshire, 2010). Generally, better credit grades are associated with lower interest rates. Second, P2P lending platforms charge service fees for transactions (Klafft, 2008), instead of charging borrowers higher interest rates than the cost of the money as traditional financial institutions. P2P lending process benefits both borrowers and lenders. While borrowers can borrow money at lower costs than traditional financial institutions, lenders can make more money than putting their money in banks. This benefit comes with the risk of borrowers’ defaulting on the loans because many P2P loans are unsecured personal loans and most lenders have little knowledge about credit risk management (Xia, Liu, & Liu, 2017).

To control default rates and risks, P2P lending platforms built classification models to evaluate credit risks of loans and borrowers and suggest appropriate interest rates for loan applications. The quality of credit classification models is vital to the credit risk management and sustainability of P2P lending platforms. Using experiences from financial institutions, P2P lending platforms adopt and develop classification algorithms to categorize borrowers into different credit grades based on their characteristics and credit history, and recognize potential borrowers who are likely to default (Lessmann, Baesens, Seow, & Thomas, 2015; Florez-Lopez & Ramon-Jeronimo, 2014; Marqués, García, & Sánchez, 2013).

Though it is a common practice in traditional credit rating to use standard cost-insensitive binary classification algorithms (Li, Kou, Peng, & Shi, 2017; Morente-Molinera, Mezei, Carlsson, & Herrera-Viedma, 2017), such as logistic regression, neural networks, and decision trees (Butaru et al., 2016; Luo, Wu, & Wu, 2017), they are not appropriate in P2P lending for the following reasons. First, there are more than two classes of credit grades in P2P lending and each credit grade implies a certain level of risk. Thus multi-class classification should be considered in P2P credit grading. Second, P2P loan data are imbalanced. The number of samples in different credit grades varies dramatically. For instance, the number of ideal borrowers in the best grade or high-risk borrowers in the worst grade is much smaller than the other grade groups. Third, misclassification costs are not uniform across classes in P2P lending. In general, the cost of classifying a loan with bad credit as a good one is usually greater than classifying a good one as bad (Chen, Ribeiro, & Chen, 2016). In a multi-class credit-grading scenario, classifying a sample of grade C into grade A is more costly than classifying B into A. Therefore, standard cost-insensitive multi-class classification, in which all errors have the same cost, is not suitable for credit rating in P2P lending.

Cost-sensitive multi-class classifiers fit well for credit rating in P2P lending. Cost-sensitive classifiers were developed for imbalanced data classification (Elkan, 2001; Hu et al., 2015; Sun, Shang, & Li, 2014). Various cost-sensitive classifiers have been proposed for credit rating (Bahnsen, Aouada, & Ottersten, 2015; Chao & Peng, 2018; Marqués et al., 2013; Sahin, Bulkan, & Duman, 2013). The goal of cost-sensitive classifier is to minimize total costs measured by a misclassification cost matrix (Guan, Yuan, Ma, Khattak, & Chow, 2017), which is not only necessary but also important for cost-sensitive classification problems.

Though there are a few works in P2P lending (Xia et al., 2017; Xu, Chen, & Chau, 2016) that studied cost-sensitive classifiers, none of them have analyzed this issue from the perspective of multi-class classifications and measured misclassification costs of different credit grades using real losses and opportunity costs associated with P2P lending. How to measure the misclassification costs of different credit grades is a useful but understudied problem. Serrano-Cinca and Gutiérrez-Nieto (2016) showed that loan profitability outperformed loan default probability in P2P lending, which proved the importance of considering both interest rates and the probability of default in P2P credit scoring.

Misclassification costs are losses of lenders’ earnings due to misclassifying credit grades of loans. It equals to the difference between the return of a loan when it is correctly classified and the return of a loan when it is misclassified as other credit grade. The difference can be one of the following situations: (1)(1) $$E{R}_i=(1-P{D}_i)\cdot (1+I_i)+P{D}_i\cdot (1-\mathit{Lgd})=1+I_i-P{D}_i\cdot (I_i+\mathit{Lgd})$$(1) If a loan is classified to a better credit grade with a lower interest rate, the risk to default of the loan is underestimated and the interest rate of the loan is set lower than it should be, which means that the interest maybe insufficient to cover the risk that the lender bears. The lender will lose potential returns that they could have gotten, including an unpaid risk that the borrower should pay for the higher-risk loan. (2)(2) $$P{D}^{\prime }{(j|i)}_{i>j}=P{D}_i+\beta \cdot (I_i-I_j)$$(2) If a loan is classified to a worse credit grade with a higher interest rate, borrowers might be scared away or it may increase their chance to default, which causes opportunity costs and financial losses to lenders.

The objective of this paper is to propose a multi-class cost matrix that measures misclassification costs of P2P credit grading by considering real losses and opportunity costs associated with P2P lending. We developed a set of equations and models to calculate misclassification costs. The parameters in the proposed equations and models are designed to calculate the cost matrix and support P2P lending platforms’ operations. A case study using data from Lending Club is conducted to demonstrate the performances of the proposed cost matrix using several well-known cost-sensitive classifiers. The results show that the proposed cost matrix can not only reveal the sources of losses caused by misclassifications, but also reduce the total costs for real-world P2P platforms, which is better than cost-insensitive classification algorithms.

The rest of this paper is organized as follows. Section 2 reviews related works. Section 3 proposes an abstract structure of credit grades, and misclassification costs which measure real financial losses in P2P lending. Section 4 analyzes the range of parameters in the misclassification cost matrix and explains their managerial implications. Section 5 conducts an experiment using data from Lending Club. Section 6 concludes the paper with limitations and future research directions.

2. Related works

The goal of most classifiers is to maximize accuracy and minimize misclassifications. Various classification methods have been proposed for credit rating and risk management (Santana, Lanzarini, & Bariviera, 2018; Huang & Kou, 2014; Kou, Peng, & Wang, 2014; Lanzarini, Villa Monte, Bariviera, & Jimbo Santana, 2017; Peng, Wang, Kou, & Shi, 2011; Wu & Kou, 2016;). Standard classifiers treat the costs of misclassifications the same, which is not true in real credit risk management (Fiore, De Santis, Perla, Zanetti, & Palmieri, 2017; Tapkan, Özbakır, Kulluk, & Baykasoğlu, 2016). Many researches support the use of cost-sensitive classifiers in credit rating. Sahin et al. (2013) proposed a cost-sensitive decision tree approach with varying misclassification costs. It is successfully used in credit card fraud detection to decrease financial losses. Alejo, García, Marqués, Sánchez, and Antonio-Velázquez (2013) improved the Multilayer Perceptron neural network using three misclassification cost functions and can be used to improve the prediction effectively in credit rating. Bahnsen, Aouada, and Ottersten (2014, Bahnsen et al., 2015) suggested example-dependent cost-sensitive methods and proposed logistic regression and decision trees for credit scoring.

Misclassification cost can be described by a cost matrix C = (c_ij)_n×n, where c_ij indicates the cost due to misclassifying an instance of class i as class j, and n is the number of classes (Domingos, 1999). In credit rating, the measurement of misclassification costs in C is not only a basic component of cost-sensitive classification, but also vital for high quality credit rating. Real financial indicators, like profit-based or financial loss-related measures, are well aligned with the objectives in credit rating (Maldonado, Bravo, Lopez, & Perez, 2017; Serrano-Cinca & Gutiérrez-Nieto, 2016; Verbraken, Bravo, Weber, & Baesens, 2014). Beling, Covaliu, and Oliver (2005) set the cost of a false negative to a loan’s interest rate charged to the customer int_r, the cost of a false positive to the loss given default Lgd, and both the costs of true positive and true negative are set to zero. Following this notation, this paper regards default loans as negative instances and good loans as positive instances. Table 1 shows the cost matrix (Beling et al., 2005).

Table 1. Cost matrix proposed by Beling et al. (2005).

		Prediction
		Positive	Negative
Actual	Positive	0	intr
	Negative	Lgd	0

Hand, Whitrow, Adams, Juszczak, and Weston (2008) proposed a cost matrix (Table 2) for credit card fraud detection. It represents the costs of misclassification by the administrative cost Ca, which is related to analyzing the transactions and contacting card holders. In the cases of false negative and true negative, the associated costs both equal to C_a because the card holder will have to be contacted. However, in the case of false positive, due to the fact that frauds are not detected, the cost is defined as a hundred times C_a.

Table 2. Cost matrix proposed by Hand et al. (2008).

		Prediction
		Positive	Negative
Actual	Positive	0	Ca
	Negative	100*Ca	Ca

Bahnsen, Stojanovic, Aouada, and Ottersten (2013) pointed out a limitation of the above cost matrices. Since losses of different frauds range from a few to a large amount, it is unrealistic to assume constant cost in false positive. In Bahnsen et al. (2013), a new cost matrix (Table 3) is proposed as a better representation of the actual costs, where the cost of false positive is replaced by the amount Amt_i of the transaction i.

Table 3. Cost matrix proposed by Bahnsen et al. (2013).

		Prediction
		Positive	Negative
Actual	Positive	0	Ca
	Negative	Amti	Ca

Bahnsen et al. (2014) proposed a cost matrix (Table 4) with example-dependent varying misclassification costs. For every borrower i, the costs of correct classifications are zero, and the cost of false positive is the losses if borrower i defaults which is proportional to his credit line Cl_i. The cost of false negative is r_i plus C^a, where r_i is the profit that can be earned from a good borrower, and C^a is related to uncertainty of next alternative borrower.

Table 4. Example-dependent cost matrix proposed by Bahnsen et al. (2014).

		Prediction
		Positive	Negative
Actual	Positive	0	ri+Ca
	Negative	Cli·Lgd	0

Xia et al. (2017) proposed a cost-sensitive boosted tree loan evaluation model to discriminate potential default borrowers in P2P lending. The model considers the imbalanced misclassification cost (shown in Table 5), based on the assumption that the cost of misclassifying a default borrower is larger than that of misclassifying a good one, i.e. C(0, 1) > C(1, 0). It adopts a cost matrix with two classes (Good and Default).

Table 5. Cost matrix in Xia et al. (2017).

		Predicted
		Good	Default
Observed	Good	0	C(1, 0)
	Default	C(0, 1)	0

Existing studies in cost-sensitive classification and cost matrices in credit rating focused on binary classification problems and traditional credit risk applications. Few, if there’s any, analyzed this issue from the perspective of multi-class classifications and in the context of P2P lending.

In cost-sensitive multi-class classification, the corresponding cost matrix C = (c_ij) is n-dimensional (Kou, Ergu, Lin, & Chen, 2016) where n is number of classes. For any $i,j=1,2,\dots ,n,$ the element c_ij indicates the misclassification cost that is caused by misclassifying an instance of class i as class j. The objective of this study is to determine the values of misclassification costs by incorporating profit losses and other indirect costs in P2P lending. A high-quality n-dimensional cost matrix can help to improve the performances of classifiers and reduce default losses in P2P lending.

3. Misclassification cost measures

This section analyzes the risks and profits in P2P lending, and proposes a multi-class misclassification cost matrix for P2P lending.

3.1. Modeling risks and profits in P2P lending

P2P lending platform provides services to lenders and borrowers to facilitate their transactions, and charges a proportion of profits earned by successful repaid loans as an essential part of services fee. One of the main services is to provide credit information of borrowers and suggestions (including credit rating and appropriate interest rates) to lenders. In fact, platforms and lenders share profits and risks.

For lenders, borrowers’ default would cause loses. In this paper, loss given default (Lgd) (Schuermann, 2004) is used to measure the proportion of a lender’s loss when a borrower defaults on a loan. Although Lgd is different for different loans, it is usually treated as a constant number for all loans to simplify models (Bahnsen et al., 2014; Beling et al., 2005). Lenders’ profits come from the interests of investments if borrowers fully repaid.

For platforms, default loans impact their profits directly by causing the loss of service fees and indirectly by damaging their reputations in quality of services on credit rating and setting interest rates. Thus, accurate credit rating is the foundation of healthy and profitable operations of P2P platforms. Consequently, an important requirement of credit rating in P2P lending is to measure the profitability of a loan (Xia et al., 2017), and the amount of loss in profit for lenders should be considered in misclassification cost matrix.

P2P platforms usually classify borrowers into multiple credit grades and determine hierarchical interest rates for them. We construct a structure of credit grades to measure the profits and losses in P2P lending (Table 6), based on the assumption that different credit grade i is associated with a certain probability of default (PD) PD_i. Therefore it is assigned an interest rate I_i to reflect the corresponding credit risk. Credit rating classifies loans into these grades, where ‘1(A)’ refers to the best credit, and ‘n’ refers to the worst credit risk. Generally, the worse the credit grade, the higher probability of default and the higher interest rate. That is, for any $i,j=1,2,\dots ,n,$ $i>j,$ so that $P{D}_i>P{D}_j$ and $I_i>I_j.$

Table 6. Structure of credit grades in P2P lending.

Credit grades	Probability of default (PD)	Interest rate (I)
1(A)	PD1	I1
2(B)	PD2	I2
3(C)	PD3	I3
…
n	PDn	In

When a lender lends a loan of credit grade i (i = 1, 2, …, n) with PD_i probability of default and I_i interest rate, the lenders’ expected return is: (1) $$E{R}_i=(1-P{D}_i)\cdot (1+I_i)+P{D}_i\cdot (1-\mathit{Lgd})=1+I_i-P{D}_i\cdot (I_i+\mathit{Lgd})$$(1) where Lgd is the loss given default which indicates the average loss rate of money when a borrower defaulting on the loan.

3.2. Misclassification cost matrix

Based on Table 6, we propose a misclassification cost matrix C for credit rating in P2P lending (Table 7), where the cost of correct classification is zero, i.e. c_ii = 0, i = 1, 2, …, n.

Table 7. Misclassification cost matrix C for credit rating in P2P lending.

		Prediction
		1(A)	2(B)	…	n
Actual	1(A)	0	c12	…	c1n
	2(B)	c21	0	…	c2n
	…	…	…	0	…
	n	cn1	cn2	…	0

Misclassification cost matrix C can be decomposed into two blocks C¹ and C², that is $C=\left(\begin{array}{cc}0& C^2\\ C^1& 0\end{array}\right).$ C¹is the lower triangular submatrix corresponding to the loans whose predicted credit grades (j) are better than their actual credit grades (i), i.e. i > j, and C² is the upper triangular submatrix corresponding to the loans whose actual credit grades (i) are better than their predicted credit grades (j), i.e. i < j.

3.2.1. Lower triangular submatrix C¹: Prediction (j) better than actual (i)

In the lower triangular matrix $C^1={(C_{ij}^1)}_{i>j},$ $C_{ij}^1$ is the cost of misclassifying a loan in grade i as grade j. As a result of misclassification, $P{D}_i>P{D}_j$ and $I_i>I_j,$ which means a borrower pays a lower interest rate than he/she should. Lower interest rate may affect borrowers’ potential defaults to a certain extent, which has been supported by theoretical and empirical evidences (Edelberg, 2006; Serrano-Cinca, Gutierrez-Nieto, & López-Palacios, 2015). Specifically, the relationship between interest rate and risk of default is positive in P2P lending. To describe this kind of changes, we revise the PD in a linear function related with the difference of interest rates: (2) $$P{D}^{\prime }{(j|i)}_{i>j}=P{D}_i+\beta \cdot (I_i-I_j)$$(2) where β is a revised coefficient of PD, indicating how the PD of a borrower changes with different interest rates. Obviously, $\beta <0.$

$P{D}^{\prime }(j|i)$ is the actual probability of default, after misclassifying a loan in grade i to grade j. The misclassification leads to a change of expected return for the lender: (3) $$E{R}^{\prime }{(j|i)}_{i>j}=1+I_j-P{D}^{\prime }(j|i)\cdot (I_j+\mathit{Lgd})$$(3)

The misclassification also leads to a loss in return and the cost is formulated as: (4) $$C_{ij}^1=E{R}_i-E{R}^{\prime }(j|i)=(I_i-I_j)\cdot \left[1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})\right]$$(4)

3.2.2. Upper triangular submatrix C²: Actual (i) better than prediction (j)

In the upper triangular matrix $C^2={(C_{ij}^2)}_{i<j},$ $P{D}_i<P{D}_j$ and $I_i<I_j,$ which means a borrower pays a higher interest rate than he/she should. This misclassification may lead to a withdrawal of loan application with a certain probability because the borrower can’t tolerate the high interest rate. This probability of application withdrawal is related to the difference of interest rates $I_i-I_j.$ To simplify the model, a linear function is used to calculate the probability that a borrower of credit grade i will give up application when misclassified as credit grade j (worse than grade i). (5) $${\text{Prob}}_{\mathit{give}-up}{(j|i)}_{i<j}=\alpha \cdot (I_i-I_j)$$(5) where α is borrowers’ churn rate (Zhu, Baesens, Backiel, & Vanden Broucke, 2018). It indicates how the probability ${\text{Prob}}_{\mathit{give}-up}(j|i)$ varies with the difference of interest rates $I_i-I_j.$

On the other hand, the probability of a borrower accepting interest rate I_j caused by misclassification is $1-{\text{Prob}}_{\mathit{give}-up}(j|i).$ In this case, the PD of the borrower increases with the higher interest rate. Equation (6)(6) $$P{D}^{\prime }{(j|i)}_{i<j}=P{D}_i+\beta \cdot (I_i-I_j)$$(6) is used to calculate the actual PD $P{D}^{\prime }{(j|i)}_{i<j}$ when i < j. (6) $$P{D}^{\prime }{(j|i)}_{i<j}=P{D}_i+\beta \cdot (I_i-I_j)$$(6)

For a lender who accepts the misclassified loans when i < j, the expected return is: (7) $$E{R}^{\prime }{(j|i)}_{i<j}=1+I_j-P{D}^{\prime }(j|i)\cdot (I_j+\mathit{Lgd})$$(7)

Thus, the misclassification cost $C_{ij}^2$ can be measured as: (8) $$C_{ij}^2=E{R}_i-(1-{\text{Prob}}_{\mathit{give}-up}(j|i))\cdot E{R}^{\prime }(j|i)$$(8)

It is rewritten as: (9) $$C_{ij}^2=(1-{\text{Prob}}_{\mathit{give}-up}(j|i))\cdot (E{R}_i-E{R}^{\prime }(j|i))+{\text{Prob}}_{\mathit{give}-up}(j|i)\cdot E{R}_i$$(9)

On the right hand side of Equation (9)(9) $$C_{ij}^2=(1-{\text{Prob}}_{\mathit{give}-up}(j|i))\cdot (E{R}_i-E{R}^{\prime }(j|i))+{\text{Prob}}_{\mathit{give}-up}(j|i)\cdot E{R}_i$$(9) , the first term indicates the loss in profit from transactions, and the second term ${\text{Prob}}_{\mathit{give}-up}(j|i)\cdot E{R}_i$ indicates the opportunity cost caused by application withdrawals due to misclassifications.

This model does not consider the idle investment due to borrowers’ termination and the uncertainty of next alternative borrower, which were discussed in Bahnsen et al. (2014). If a borrower terminates the application, the transaction will not happen, and the money of a lender invested will be lent to an alternative borrower. Credit rating for the alternative borrower is also a classification problem using the proposed misclassification cost matrix.

In summary, based on a structure of credit grades in P2P lending, we designed a misclassification cost matrix to measure real business losses, which are calculated using Equations (4)(4) $$C_{ij}^1=E{R}_i-E{R}^{\prime }(j|i)=(I_i-I_j)\cdot \left[1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})\right]$$(4) and (9)(9) $$C_{ij}^2=(1-{\text{Prob}}_{\mathit{give}-up}(j|i))\cdot (E{R}_i-E{R}^{\prime }(j|i))+{\text{Prob}}_{\mathit{give}-up}(j|i)\cdot E{R}_i$$(9) . In addition, we analyzed the parameters (Lgd, β, and α) from the perspective of both cost-sensitive classification and operations in P2P lending.

4. Parameters analysis

4.1. Loss given default: Lgd

We adopt a quadratic programming to deduce loss given default (Lgd) backward from observed PDs, interest rates, and actual return rates: (10) $$\begin{array}{c}\underset{\mathit{Lgd}}{\min }\sum _{i=1,2,\dots ,n}{(E{R}_i-A{R}_i)}^2\\ s.t.0<\mathit{Lgd}<1\end{array}$$(10) where ER_i indicates the expected returns of lenders who lend to borrowers in credit grade i. $A{R}_i$ is the actual return rate of investing loans to borrowers in credit grades i, which considers potential losses due to defaults and is calculated by P2P platforms. The Lgd value is calculated by minimizing the sum of squared differences between expected returns ER_i and actual returns $A{R}_i$ on different grades. This fits the realistic environment and reflects the average percentage of losses due to defaults.

4.2. PD’s revised coefficient: β

β is PD’s revised coefficient in Equation (2)(2) $$P{D}^{\prime }{(j|i)}_{i>j}=P{D}_i+\beta \cdot (I_i-I_j)$$(2) . It indicates how the PD of a borrower changes with different interest rates she/he bears. There are two reasonable assumptions about the revised PD in practice. First, the higher the interest rate is, the higher the probability that a borrower defaults. Second, a lower interest rate will not completely convert PDs of lower-credit-grades borrowers into those with better-credit-grades. In other words, when i > j, even if a borrower of credit grade i lowers her/his PD from PD_i to $P{D}^{\prime }(j|i),$ $P{D}^{\prime }(j|i)$ is still larger than PD_j due to the lower interest rate I_j. That is, $P{D}_j<P{D}^{\prime }(j|i)<P{D}_i.$ According to Equation (2)(2) $$P{D}^{\prime }{(j|i)}_{i>j}=P{D}_i+\beta \cdot (I_i-I_j)$$(2) , it is equivalent to: $$P{D}_j<P{D}_i+\beta \cdot (I_i-I_j)<P{D}_i$$ because $I_i-I_j>0,$ (11) $$\frac{P{D}_j-P{D}_i}{I_i-I_j}<\beta <0,\text{for}\text{any}i,j=1,2,\dots ,n\text{and}i>j$$(11)

In the view of cost matrix, for any i, j = 1, 2, …, n, if i > j, then $C_{ij}^1>0.$ According to Equation (4)(4) $$C_{ij}^1=E{R}_i-E{R}^{\prime }(j|i)=(I_i-I_j)\cdot \left[1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})\right]$$(4) , we get: (12) $$\begin{array}{c}(I_i-I_j)\cdot \left[1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})\right]>0\\ 1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})>0\\ \beta >\frac{P{D}_i-1}{I_j+\mathit{Lgd}},\text{for}\text{any}i,j=1,2,\dots ,n\text{and}i>j\end{array}$$(12)

Moreover, for any i, j, k = 1, 2, …, n, if i > j > k, the cost due to misclassifying a loan of credit grade i into k should be larger than misclassifying it as j. That is, $C_{ij}^1<C_{ik}^1$ so $$(I_i-I_j)\cdot \left[1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})\right]<(I_i-I_k)\cdot \left[1-P{D}_i+\beta \cdot (I_k+\mathit{Lgd})\right]$$ $$(I_i-I_j)\cdot \beta \cdot (I_j+\mathit{Lgd})-(I_i-I_k)\cdot \beta \cdot (I_k+\mathit{Lgd})<(I_j-I_k)\cdot (1-P{D}_i)$$

If $(I_i-I_j)\cdot (I_j+\mathit{Lgd})-(I_i-I_k)\cdot (I_k+\mathit{Lgd})>0,$ $$\beta <\frac{(I_j-I_k)\cdot (1-P{D}_i)}{(I_i-I_j)\cdot (I_j+\mathit{Lgd})-(I_i-I_k)\cdot (I_k+\mathit{Lgd})},$$ that is, $\beta$ is less than a positive number, which have been covered by $\beta <0$ in formula (12)(12) $$\begin{array}{c}(I_i-I_j)\cdot \left[1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})\right]>0\\ 1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})>0\\ \beta >\frac{P{D}_i-1}{I_j+\mathit{Lgd}},\text{for}\text{any}i,j=1,2,\dots ,n\text{and}i>j\end{array}$$(12) .

If $(I_i-I_j)\cdot (I_j+\mathit{Lgd})-(I_i-I_k)\cdot (I_k+\mathit{Lgd})<0,$ (13) $$\begin{array}{c}\beta >\frac{(I_j-I_k)\cdot (1-P{D}_i)}{(I_i-I_j)\cdot (I_j+\mathit{Lgd})-(I_i-I_k)\cdot (I_k+\mathit{Lgd})},\\ \text{for}\text{any}i,j,k=1,2,\dots ,n\text{and}i>j>k\end{array}$$(13)

Similarly, because $C_{jk}^1<C_{ik}^1,$ (14) $$\begin{array}{c}\beta >\frac{(I_j-I_k)\cdot (1-P{D}_j)-(I_i-I_k)\cdot (1-P{D}_i)}{(I_i-I_j)\cdot (I_k+\mathit{Lgd})},\\ \text{for}\text{any}i,j,k=1,2,\dots ,n\text{and}i>j>k\end{array}$$(14)

4.3. Borrowers’ churn rate: α

When a loan of credit grade i is misclassified into grade j ($\forall i,j=1,2,\dots ,n,i<j$), a borrower will give up his loan application with a probability ${\text{Prob}}_{\mathit{give}-up}(j|i).$ α is borrowers’ churn rate which indicates how the probability ${\text{Prob}}_{\mathit{give}-up}(j|i)$ varies with the difference of interest rates $I_i-I_j.$ Naturally, $0<{\text{Prob}}_{\mathit{give}-up}(j|i)<1$ so that: (15) $$\frac{1}{I_i-I_j}<\alpha <0,\text{for}\text{any}i,j=1,2,\dots ,n\text{and}i<j$$(15)

We analyze the parameter α in the view of cost matrix. Then α holds: (16) $$C_{ij}^2>0,\text{for}\text{any}i,j=1,2,\dots ,n\text{and}i<j$$(16) (17) $$C_{ij}^2<C_{ik}^2,,\text{for}\text{any}i,j,k=1,2,\dots ,n\text{and}i<j<k$$(17) (18) $$C_{jk}^2<C_{ik}^2,\text{for}\text{any}i,j,k=1,2,\dots ,n\text{and}i<j<k$$(18)

Specifically, the cost of misclassifying a bad loan into a better one is larger than the other way around. That is, (19) $$C_{ij}^2<C_{ji}^1,\text{for}\text{any}i,j=1,2,\dots ,n\text{and}i<j$$(19) where $C_{ji}^1$ indicates the misclassification costs of misclassifying a loan (of grade j) to a better credit grade i, and $C_{ij}^2$ indicates the misclassification costs of misclassifying a loan (of grade i) to a worse credit grade j.

5. Experiment: A case study on lending club

5.1. Data collection

To validate the proposed measure of misclassification costs in a real P2P platform, the empirical study utilizes data collected from Lending Club (Lending Club, 2017), which is the largest P2P lending platform in U.S and the data is publicly available.

We collected the data of loans on Lending Club from the first quarter of 2016 to the third quarter of 2017. The total number of loans in this data set is 759348 (as shown in Table 8). There are 128 features for each loan in the original data, including loan characteristics (such as loan amount, term, and purpose), borrowers’ financial situation (such as annual income and home ownership), and credit history (such as FICO score, the number of inquiries, the number of open credit lines, and incidences of delinquency). In particular, a variable named LC grade, which is the credit grade for loans assigned by Lending Club, is used as the class label for training and testing classification algorithms in the experiment.

Table 8. Number of instances in lending club data.

Period	# of instances
2016Q1	133889
2016Q2	97856
2016Q3	99122
2016Q4	103548
2017Q1	96781
2017Q2	105451
2017Q3	122701
Total	759348

The Lending Club data has been used in existing P2P lending researches (Serrano-Cinca & Gutiérrez-Nieto, 2016; Xia et al., 2017). Serrano-Cinca and Gutiérrez-Nieto (2016) provided some statistics of the data, such as the proportions of loans in different credit grades, the proportions of default, and the mean of interest rates for each grade. Table 9 summarizes the structure of credit grades of the Lending Club data, where the probability of default is estimated using the proportion of default loans to all loans in the grade.

Table 9. Structure of credit grades on lending club data.

Grades	% of loans	Probability of default (PD)	Interest rate (I)
A	32.33%	6.28%	7.42%
B	33.60%	11.54%	11.33%
C	19.97%	15.56%	13.94%
D	10.85%	18.46%	16.15%
E	2.56%	20.65%	17.78%
F	0.55%	25%	19.27%
G	0.15%	33.87%	21.03%

Table 9 supports our assumptions of the proposed models. First, there are seven credit grades (A–G) implemented on Lending Club, where A is best and G is worst. It confirms the use of multi-class classification in P2P lending. Second, the proportions of loans vary from 33.60% to 0.15%, which means that the data are highly imbalanced. Third, worse credit grades are associated with higher PDs and interest rates, which is a basic assumption of our model to measure misclassification costs using real losses in P2P lending.

5.2. Setting parameters and managerial implications

The interest rates in P2P lending are set based on the business environment, which can be measured using parameters, such as Lgd, β, and α. In the experiment, we set the values of parameters using the observed PDs and interest rates in Table 9. The parameters not only reflect the operating environment of Lending Club, but also provide guidelines for interest rates adjustments.

Furthermore, Lending Club provides actual average annualized returns for lenders, shown in Table 10. According to formula (10)(10) $$\begin{array}{c}\underset{\mathit{Lgd}}{\min }\sum _{i=1,2,\dots ,n}{(E{R}_i-A{R}_i)}^2\\ s.t.0<\mathit{Lgd}<1\end{array}$$(10) , the parameter Lgd is approximately 0.5 (0.50543), which means that about 50% of principal in investment will be lost if default happens.

Table 10. Actual annualized returns provided by lending club*.

Grades	A	B	C	D	E	F/G
Annualized return	4.38%	5.21%	5.35%	3.96%	2.99%	−1.5%

* The data is available at: https://www.lendingclub.com/info/demand-and-credit-profile.action.

To satisfy inequalities in (11)–(14), the range of parameter β is solved as −0.1116 < β < 0. To calculate the proposed model, we set β as the median of the range, that is β = −0.0558. Thus, the revised PD on Lending Club, shown in Table 11, can be estimated using Equation (2)(2) $$P{D}^{\prime }{(j|i)}_{i>j}=P{D}_i+\beta \cdot (I_i-I_j)$$(2) . Bold numbers in Table 11 indicate the inherent PDs for loans of credit grades. In fact, fluctuations of interest rates caused by misclassification can affect borrowers' default. The experiment proved that inherent credit ratings have more effects on the probability of default than the fluctuations of interest rates. This makes sense in P2P lending, where loans are normally small in size and repayment of unsecured microloans is more dependent on the willingness to pay than the ability to pay.

Table 11. Revised PD on lending club.

		Prediction
		A	B	C	D	E	F	G
Actual	A	0.0628	0.0650	0.0664	0.0677	0.0686	0.0694	0.0704
	B	0.1132	0.1154	0.1169	0.1181	0.1190	0.1198	0.1208
	C	0.1520	0.1541	0.1556	0.1568	0.1577	0.1586	0.1596
	D	0.1797	0.1819	0.1834	0.1846	0.1855	0.1863	0.1873
	E	0.2007	0.2029	0.2044	0.2056	0.2065	0.2073	0.2083
	F	0.2434	0.2456	0.2470	0.2483	0.2492	0.2500	0.2510
	G	0.3311	0.3333	0.3347	0.3360	0.3369	0.3377	0.3387

To satisfy inequalities in (15)–(19), the range of parameter α is −1.2916 < α < −0.8412. We set α as the median of the range, this is α = −1.0664. It indicates that a borrower's probability of terminating a loan application increases about 1.1% if the interest rate rises by 1%.

In the experiment, we calculate the parameters by analysis of models based on observed structure including probability of default and interest rates. In fact, the parameters are determined by behaviors of borrowers, which can be measured through investigation. It provides a guidance to check and adjust interest rates. If there’s a big difference between the values of parameters calculated and the results obtained from the real practice, the interest rates on P2P lending platforms should be adjusted, because PDs are given and generally stable.

5.3. Cost matrix

According to Equations (4)(4) $$C_{ij}^1=E{R}_i-E{R}^{\prime }(j|i)=(I_i-I_j)\cdot \left[1-P{D}_i+\beta \cdot (I_j+\mathit{Lgd})\right]$$(4) and (9)(9) $$C_{ij}^2=(1-{\text{Prob}}_{\mathit{give}-up}(j|i))\cdot (E{R}_i-E{R}^{\prime }(j|i))+{\text{Prob}}_{\mathit{give}-up}(j|i)\cdot E{R}_i$$(9) , we calculate the misclassification cost matrix C (Table 12), which is the main contribution of this paper. Elements in the cost matrix measure the loss of lenders’ expected return if a loan is misclassified. For example, in the cost matrix C, c₂₁ = 0.0333 means that a lender who provides a loan with a certain amount amt will loss 0.0333*amt in her/his expected return when a loan of credit grade B is misclassified as grade A. Meanwhile, the revenue of the platform reduces because of the misclassification of credit grades, where the costs should be minimized in cost-sensitive credit rating.

Table 12. Misclassification cost matrix on lending club.

		Prediction
		A	B	C	D	E	F	G
Actual	A	0	0.0089	0.0166	0.0241	0.0303	0.0365	0.0443
	B	0.0333	0	0.0073	0.0144	0.0203	0.0262	0.0336
	C	0.0530	0.0211	0	0.0070	0.0128	0.0184	0.0256
	D	0.0684	0.0376	0.0172	0	0.0056	0.0112	0.0182
	E	0.0789	0.0489	0.0291	0.0123	0	0.0055	0.0124
	F	0.0851	0.0568	0.0380	0.0222	0.0106	0	0.0070
	G	0.0856	0.0608	0.0443	0.0304	0.0202	0.0109	0

The largest cost in Table 12 is 0.0856, which happens when class G was misclassified as class A. It is align with the reality because this misclassification assigns the lowest interest rate to the highest credit risk group. And the smallest cost is 0.0055, which appears when class E is misclassified into F.

5.4. Sensitivity analysis

The parameters α and β were set as the median value of the corresponding ranges in the experiments. In reality, there is uncertainty on the parameters due to borrowers’ behaviors. This section uses sensitivity analysis to study how the uncertainty of input parameters affects the cost matrix in the output.

To facilitate observations of the cost matrixes under different parameters, cosine similarity is adopted to compare them with the cost matrix C = (c_ij) shown in Table 12, where parameters are taken as median (α₀, β₀). Given any −1.2916 < α₁ < −0.8412 and −0.1116 < β₁ < 0, the corresponding cost matrix D = (d_ij) can be obtained. Then the cosine similarity between C and D is calculated as follows: (20) $$\text{Cosine}\text{Similarity}(C,D)=\frac{\sum c_{ij}\cdot d_{ij}}{\sqrt{\sum c_{ij}{}^2}\cdot \sqrt{\sum d_{ij}{}^2}}$$(20)

In other words, the cost matrix C in Table 12 is considered as benchmark in calculation of cosine similarity. Figure 1 shows the similarity result when the parameters vary in their ranges. Obviously, the similarity is equal to 1 when α and β are median, because it is benchmark. To observe more details of the result, we take the parameters α and β respectively at 9 equidistant points within their ranges. Table 13 lists the cosine similarities on these discrete values of parameters. The element at the center of Table 13 indicates the situation of benchmark (Table 12). The lowest similarity appears at the corner of the range. When both α and β are taken as the maximum in their ranges, the fact that similarity equals 0.9451 still guaranteed a small difference between the corresponding cost matrix and benchmark.

Figure 1. Sensitivity analysis for cosine similarity of cost matrixes in terms of parameters uncertainty.

Table 13. Cosine similarities of cost matrixes when the parameters are discrete values.

Alpha \ Beta	−0.1116	−0.0977	−0.0837	−0.0698	−0.0558	−0.0419	−0.0279	−0.0140	0.0000
−1.2916	0.9619	0.9635	0.9651	0.9666	0.9681	0.9695	0.9708	0.9721	0.9734
−1.2353	0.9758	0.9770	0.9783	0.9794	0.9805	0.9816	0.9826	0.9836	0.9845
−1.1790	0.9874	0.9883	0.9891	0.9899	0.9906	0.9914	0.9920	0.9926	0.9932
−1.1227	0.9958	0.9963	0.9967	0.9971	0.9975	0.9978	0.9981	0.9984	0.9986
−1.0664	0.9998	0.9999	1.0000	1.0000	1.0000	1.0000	1.0000	0.9999	0.9998
−1.0101	0.9982	0.9979	0.9977	0.9974	0.9971	0.9968	0.9965	0.9962	0.9959
−0.9538	0.9897	0.9892	0.9887	0.9883	0.9878	0.9873	0.9868	0.9863	0.9859
−0.8975	0.9732	0.9727	0.9722	0.9716	0.9711	0.9706	0.9701	0.9696	0.9691
−0.8412	0.9483	0.9478	0.9474	0.9470	0.9466	0.9463	0.9459	0.9455	0.9451

The sensitivity analysis for the parameters shows that the proposed method used to calculate cost matrix is robust in the presence of uncertainty of parameters. Thus, it is reasonable to set the parameters as the median value of the ranges.

5.5. Cost-sensitive credit rating

After calculated the misclassification cost matrix, we can use cost-sensitive credit rating to classify the loans into credit grades (A, B, C, D, E, F, or G), using the collected Lending Club data.

Data preprocessing was performed first. We deleted some null features and irrelevant features (such as title and other date-type features). The number of features reduced from 128 to 74. We randomly sample 10% instances from 759348 loans to train and test classifiers.

In this experiment, we selected four well-known classifiers (Lessmann et al., 2015) and the corresponding Meta-cost-sensitive classifiers to compare their performances. The four classifiers are C4.5, Random Forest, Logistic Regression, and SVM. Then we adopted a Meta approach proposed by Domingos (1999) to make the four classifiers cost-sensitive by wrapping a cost-minimizing procedure around it. The proposed cost matrix is used to guide credit rating in cost-sensitive classification.

For experimental setup, the cost-insensitive and cost-sensitive classifiers were evaluated by 10-folds cross validation using Weka software (Frank, Hall, & Witten, 2016). The performances of classifiers were evaluated using accuracy, total cost, average cost, and cost saving rate. They were computed using the confusion matrix N (Table 14) in classification result and the cost matrix proposed in Section 3.

Table 14. Confusion matrix N*.

		Prediction
		A	B	…	G
Actual	A	n11	n12	…	n17
	B	n21	n22	…	n27
	…	…	…		…
	G	n71	n72	…	n77

* n_ij indicates the number of instances of ith grades classified into jth grades.

Accuracy is the percentage of correctly classified loans (Kou, Lu, Peng, & Shi, 2012) and computed as $\mathit{Accuracy}=\sum _i{n}_{ii}/\sum _{i,j}{n}_{ij}.$ Total cost is the sum of the costs of misclassified instances, and $\mathit{Total}\mathit{cost}=\sum _{i,j}{n}_{ij}\cdot c_{ij}.$ Average cost is cost per instance, that is, $\mathit{Average}\mathit{cost}=\mathit{Total}\mathit{cost}/\sum _{i,j}{n}_{ij}.$ Cost saving rate is the rate of decreased cost after using cost-sensitive classifiers.

The classification results were summarized in Table 15. Since traditional standard classifiers focus on optimizing the accuracy and ignoring the relationship between classes and different misclassifications, they have higher accuracies than their cost-sensitive counterparts, and the cost-sensitive classifiers reduced the total costs dramatically, which is a major concern of P2P lending platforms and lenders.

Table 15. Classification results using proposed cost matrix.

Classifier	Accuracy	Total cost	Average cost	Cost saving rate
C4.5	79.29%	328.1648	0.0043	13.12%
Cost-sensitive C4.5	77.96%	285.112	0.0038
Random forest	57.18%	730.5374	0.0096	25.48%
Cost-sensitive RF	40.66%	544.4132	0.0072
Logistic regression	61.12%	565.1794	0.0074	21.37%
Cost-sensitive LR	48.44%	444.4158	0.0059
SVM	52.95%	808.4923	0.0106	11.94%
Cost-sensitive SVM	48.48%	711.933	0.0094

It is really difficult, if not impossible, to achieve the highest accuracy and the lowest total cost simultaneously. Cost-sensitive classification normally sacrifices accuracy for lower total cost (Wang, Kou, & Peng, 2018). The objective of cost-insensitive classifiers is to maximize the total accuracy. But none of them can reach 100% accuracy in multi-class classification. Although misclassified instances may cause different costs according to the cost matrix, cost-insensitive classifiers treat them the same. This is the reason that cost-insensitive classifiers have higher accuracy and higher total misclassification cost. Cost-sensitive classifiers, on the other hand, try to minimize total costs caused by misclassification, and some misclassification errors with low costs are compromised to achieve this goal when training the classifiers. Thus, cost-sensitive classifiers reduce total cost of classification, but have lower accuracy than cost-insensitive classifiers. As shown in Table 15, the classifiers which achieved the highest accuracy and the lowest total cost are C4.5 and cost-sensitive C4.5, respectively. While the accuracy of cost-sensitive C4.5 is 1.33% lower than C4.5, cost-sensitive C4.5 reduced 13.12% total cost, compared to C4.5.

Since the proposed cost matrix measured the real losses of lenders, the total cost is more practical than the accuracy. Compared with the corresponding standard classifiers, the four cost-sensitive classifiers reduced 13%, 25%, 21%, and 12% total costs, respectively (shown as the column “Cost saving rate” in Table 15). In order to connect the classification results and business reality in P2P lending, the average cost indicates how much money is lost in a loan, on average, due to potential misclassification of credit grades when a classification algorithm is used. For example, if standard C4.5 is used to grade a loan, potential misclassification cost will cause 0.43% less returns for lenders, comparing to 100% accurate classification. And this cost dropped to 0.38% if cost-sensitive C4.5 is used.

In summary, using cost-sensitive classifiers for credit rating is able to reduce losses effectively in P2P lending, which is measured by the cost matrix proposed in this paper. It also shows the need to evaluate and develop more credit rating approaches with respect to minimizing financial losses for P2P lending platforms.

6. Conclusion and discussion

Online peer-to-peer (P2P) lending platforms provide convenient and low costs lending option to individuals and small businesses. Since many P2P loans are unsecured personal loans, the quality of credit risk classification is vital to P2P lending platforms. Traditional cost-insensitive binary classification is not appropriate in P2P lending because there are more than two credit classes in real-life P2P lending and misclassification costs are not uniform across classes. Cost-sensitive classifiers try to minimize total costs measured by misclassification cost matrixes. How to measure misclassification costs of different credit grades in P2P lending is a useful but understudied issue.

The objective of this paper is to model credit rating in P2P lending as a cost-sensitive multi-class classification problem. We first proposed a misclassification cost matrix for P2P credit grading that takes into account real losses and opportunity costs associated with P2P lending. A set of equations and models were developed to calculate the costs in the misclassification cost matrix. Then we analyzed the parameters in the proposed equations and models from the perspective of both cost-sensitive classification and business operations.

To validate the proposed misclassification cost matrix, an experiment using publicly available data from Lending Club was conducted. The results showed that standard classifiers have higher accuracy than cost-sensitive counterparts, but the cost-sensitive classifiers significantly reduced the total costs, which is essential for the survival and profitability of P2P lending platforms.

One of the limitations of this work is that it did not take administrative cost into consideration. The administrative cost is an important part of the total cost in traditional bank loans. We did not include the administrative costs in the total cost calculation because Lending Club does not provide such information. Since the administrative cost in P2P lending, comparing with traditional bank loans, is insignificant, the reduced total cost calculated in the experiment can still be used as a useful reference for P2P lending platforms.

This paper defined two variables, revised PD and a borrower’s probability of give-up, as linear function related with different interest rates. The actual situations may be more complex than these assumptions. One of the future research directions is to investigate how changes in interest rates affect human behaviors on default and giving up a loan. Though this paper focuses on misclassification costs in credit rating, the basic idea of the proposal is not necessarily restricted to the financial domain and can be applied to a wide selection of areas. For instance, misclassification costs of type I and type II errors are quite different in medical diagnosis. The difficulty lies in the definition of misclassification cost matrix for medical diagnosis. Another future research direction is to generalize the misclassification cost matrix for other multi-class cost-sensitive classification problems.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported in part by grants from the National Natural Science Foundation of China (#U1811462, #71771037, #71971042, #71725001, and #71910107002).