Performance Evaluation of Hybrid Machine Learning Algorithms for Online Lending Credit Risk Prediction

ABSTRACT

Peer-to-Peer systems are still in the early stages of development when it comes to the processing of credit and the appraisal of the risk associated with it. In this study, we used a hybrid convolutional neural network with logistic regression, a gradient-boosting decision tree, and a k-nearest neighbor to predict the credit risk in a P2P lending club. The lending clubs publicly available P2P loan data was used to train the model. In order to address the issue of data imbalance within the dataset, specifically between the non-defaulter and defaulter classes, the synthetic minority oversampling technique sampling approach is utilized. We developed the architecture of our hybrid model by removing the fully connected layer with the soft-max, which is the final layer of the fully connected CNN model and replaced by LR, GBDT, and k-NN algorithms. The experimental results show that the hybrid CNN-kNN model outperforms the CNN-GBDT and CNN-LR models based on the performance metrics accuracy, recall, F1-score, and area under the curve for both all input and important features. This shows that hybrid machine learning models effectively identify and categorize credit risk in peer-to-peer lending clubs, hence assisting in financial loss prevention.

Introduction

Peer-to-peer (P2P) lending was developed in 2005 and rapidly gained popularity across the globe as a result of the rapid development of financial technologies. It revolutionized lending by allowing direct funds transfer to individuals or businesses, bypassing the need for traditional financial institutions as intermediaries (Ko et al. 2022; Ren 2021). This innovative lending model primarily relies on internet matching services to connect lenders with potential borrowers (Lyócsa et al. 2022; Serrano-Cinca and Gutiérrez-Nieto 2016). Borrowers are required to submit personal data to the P2P lending platform to apply for loans. In contrast, the lenders select which loans to invest in after carefully evaluating the data provided on the P2P lending platform. P2P lending is a viable alternative for small businesses and some individuals with no credit history. Due to the absence of operating expenses associated with a physical location, peer-to-peer lending can provide borrowers with lower interest rates than banks. In 2018, Lending Club handled more than a million loan applications totaling the US $ 20 billion (Sharma, Li, and Ahmad 2021).

There are several well-known peer-to-peer lending sites offering different types of loans. Loop specializes in small business financing, Prosper focuses on personal loans, LendingClub provides personal and business loans, Upstart offers personal loans with innovative credit assessments, StreetShares tailors loans for small companies, and SoFi provides personal and student debt refinancing. These platforms connect borrowers with lenders, offering competitive rates and flexible terms to meet various financial needs.

However, the fundamental problem with P2P lending is information asymmetry because lenders only consider the data provided by borrowers when making loan choices (Chen et al. 2021). Moreover, peer-to-peer lending involves several risks, such as operational risks, legal risks, liquidity risks, and default risks (Ko et al. 2022). There are many risks associated with the lending system, but credit risk is the primary one. The platform suffers a financial loss because it is difficult to retrieve other assets from customers who have fallen behind on payments since many loans have no collateral.

Peer-to-peer lending entails the risk of borrowers defaulting on their loans or failing to fulfill their agreed-upon interest and principal payments. In the event of a borrower’s default, a P2P platform can assist lenders in recovering their funds and potentially pursue legal action against the defaulter. However, the platform’s capabilities are limited to these measures. There is no assurance that the borrowed funds will be fully repaid. Therefore, it is crucial to employ suitable methods to accurately predict borrower repayment in P2P lending, aiming to reduce the financial losses incurred by the lender.

CNN, a well-known and valuable image classification technique, has not been extensively explored for general data mining tasks (Kim and Cho 2019). In the present study, we propose a hybrid machine-learning approach to predict repayment in the context of peer-to-peer lending. Our proposed hybrid machine learning techniques incorporate convolutional neural networks combined with logistic regression (LR), gradient-boosting decision trees (GBDT), and k-nearest neighbor (kNN).

To separate the lending transaction data from multiple borrower characteristics, the convolutional layer is employed, while the pooling layers reduce the dimensionality of the feature maps. Consequently, the pooling layer serves the dual purpose of minimizing the number of parameters to be determined from the lending data and reducing the computational load within the network. The resulting feature maps, obtained through successive application of convolution and pooling layers on the lending data, are then flattened into a one-dimensional array and utilized as inputs for the logistic regression, gradient-boosting decision tree, and k-nearest neighbor algorithms.

The following sections of this paper are organized as follows. In section 2, the results of relevant studies using machine learning and deep learning algorithms for financial sector risk prediction are reviewed. Section 3 presents the preprocessing of the lending data and provides a detailed discussion of the methods employed, such as convolutional neural networks, logistic regression, gradient-boosting decision tress, k-nearest neighbors, and hybrid CNN-LR/GBDT/kNN. Section 4 discusses more on the experimental results and comparison of LR, GBDT, and kNN algorithms that are hybrid with CNN. The conclusion is reached in section 5.

Related Works

More intensive study has been attempted in the literature, despite the fact that recognizing and classifying peer-to-peer lending clubs should have a considerable economic advantage. A number of previous studies are compiled in this section.

In order to reduce the risk of default for peer-to-peer companies and encourage the healthy and sustainable development of the P2P industry (Gao and Balyan 2022), uses a model based on the LightGBM (Light Gradient Boosting Machine) algorithm to analyze a large number of sample data from Renrendai, which is a representative platform of the P2P industry. In order to provide the best default risk detection model, they investigate the integration of linear blending with the core LightGBM model. Their findings demonstrate that the LightGBM algorithm model has the best prediction performance in terms of several performance assessment indicators, with a prediction accuracy rate on the test set of 80.25%, accurately identifying more than 80% of users. The linear blending method and the integration of LightGBM produce results with precision values of 91.36%, recall values of 75.90%, and accuracy values of 84.36%. In comparison to conventional machine learning models like logistic regression and support vector machines, the established LightGBM algorithm can effectively identify the default of the loan business on the P2P platform. The LightGBM method can accurately asses a huge number of multi-dimensional data samples to determine a user’s default risk on P2P networks.

P2P lending systems are not yet strictly regulated by pertinent laws, nevertheless, their rate of development has considerably outpaced that of laws. P2P lending activities are nevertheless vulnerable to dangers as a result. In order to reduce the risks of default and asymmetric information on P2P lending platforms (Ko et al. 2022), suggests prediction models. In particular, they created complex processes to per-process massive amounts of data retrieved from Lending. A total of five AI models- decision tree, random forest, LightGBMM, artificial neural networks, and convolutional neural network, as well as three statistical models- logistic regression, Bayesian classifier, and linear discriminant analysis- were then utilized to analyze the data. Based on the state of their loans, lending club customers were grouped logically. To evaluate the models, they employed the AUC-ROC curve, the Kolmogorov-Smirnov chart, the student’s t-test, the confusion matrix series of metrics, and other metrics. Empirical testing reveals that LightGBM perfroms the best and is 2.91% more accurate than the other models, raising lending club’s revenue by nearly $ 24 million.

Credit risk evaluation is crucial for helping financial institutions define their banking policies and business plans. The emergence of social lending platforms over the past ten years has upended established credit risk evaluation systems. Without the assistance of financial institutions, lenders and borrowers can simply communicate with one another using these platforms. They specifically assist borrowers with the investment phase, allowing lenders of any number and size to participate. Yet, the inadequate experience of the lenders and the lack of knowledge about the credit history of the borrower might increase risks in social lending platforms, necessitating an accurate credit risk rating. Due to the fact that the credit risk assessment problem in financial operations is frequently characterized as a binary problem based on debt repayment, suitable machine learning algorithms can be used to overcome these issues (Moscato, Picariello, and Sperl 2021). suggest conducting a comparison analysis of some of the most well-liked credit risk assessment techniques in order to ascertain if a loan would be repaid in a P2P platform. A number of classifiers that are among the most popular in the study and are based on diverse sampling strategies are employed to address the issue of class imbalance. An actual data set comprising 877,956 samples was gathered from the social lending sector for experimental research. They come to the conclusion that their results are the most accurate in predicting the P2P credit risk.

To overcome the limitations of the peer-to-peer credit scoring model and more accurately assess credit risk (Turiel and Aste 2020), proposes applying logistic regression and support vector machine algorithms, along with linear and nonlinear deep neural networks (DNN), to lending data in order to replicate lender acceptance of loans and forecast the likelihood of default of issued loans. A suggested two-phase model forecasts loan rejection in the first phase and default risk for loans that are granted in the second. With a test set recall macro score of 77:4%, LR was determined to be the top performer for the initial phase. Only the second phase saw the application of DNNs, which showed the highest results with a test set recall score of 72% for defaults. This demonstrates how artificial intelligence can improve credit risk models already in use, cutting the default risk of loans granted by as much as 70%. The models were also used to analyze loans obtained just for small firms. When trained on the entire dataset, the model’s initial stage performs noticeably better. Instead, when trained on the small business subgroup, the second phase performs noticeably better. This raises the possibility of a mismatch between the screening methods used for these loans and the default prediction methods that should be applied.

(Bussmann et al. 2021) presents an agnostic, post-processing methodology aimed at enhancing interpretability of sophisticated machine learning models in credit risk management. The study includes a comparison between XGBoost and logistic regression models using ROC curves, with XGBoost demonstrating superior predictive accuracy. Variable importance is interpreted through the utilization of Shapley values and a Minimal Spanning Tree (MST), revealing clusters of defaulted companies and proximity to high-risk areas. Personalized explanations identify crucial variables for default and non-default cases, aligning with previous analyses. The research distinguishes between local and global explanations, wherein the latter highlights the top ten significant variables, such as leverage, EBITDA, and trade receivables. Overall, their findings underscore the benefits of explainable models in providing personalized explanations and identifying key variables for credit risk prediction. Their proposed methodology holds potential for assisting policymakers and regulators in comprehending the determinants of financial risks and forecasting default probabilities.

Peer-to-peer lending is a sort of financial innovation that provides businesses and individuals with loans directly. With the peer-to-peer lending system, there is a chance that the loan would be repaid late, costing the organization money. There have been numerous studies done to lower the risk of default by creating a classification model of default prediction that aims to improve accuracy. Yet, data asymmetry and underperforming categorization algorithms are the main issues with prediction (Muslim et al. 2023). aims to improve the accuracy of the default risk prediction by balancing the data and applying both the stacking model ensemble and the meta-learner. A new method of stacking ensemble learning that combines the XGBoost meta-learner algorithm with the kNN, support vector machine (SVM), and RF base-learner algorithms. Two datasets, the lending club loan data analysis dataset, and the online P2P lending dataset, were used to test the model. According to the evaluation findings, LGBFS-Stacking-XGBoost is the ideal model for both datasets. The accuracy was 99.98% for the online P2P lending dataset and 91.43% for the lending club loan data analysis dataset. Their findings show that the LGBFS-Stacjubg-XGBoost strategy can improve the prediction models precision.

Building the credit-scoring model for P2P loans required the use of a number of machine learning and artificial neural network (ANN) techniques, such as logistic regression, a support vector machine, a decision tree, a random forest, XGBoost, LightGBM, and 2-layer neural networks (Chang et al. 2022). Using feature engineering approaches, they obtain and sanitize Lending Club’s open P2P loan data. To find the most suited credit-scoring model in terms of training time and test performance, their study investigates various hyperparameter values for each technique using a grid search and cross-validation. They pre-trained all the data and obtained the feature importance using an XGBoost approach in order to uncover relevant default factors. The 16 features that were chosen may have economic ramifications for studies on P2P loan default prediction. Additionally, the empirical finding demonstrates that approaches for gradient-boosting decision trees, such as XGBoost and LightGBM, outperform methods for ANN and LR, which are frequently used for conventional credit scoring. The most effective strategy outperformed the others was XGBoost.

The existing studies focus primarily on the prediction performance of individual machine learning algorithms and their comparison to traditional credit scoring models. However, it is important to note that machine learning algorithms alone may not possess the inherent capability to effectively extract significant information from P2P lending data. The gap highlights the need for exploring hybrid machine learning algorithms that combine the strengths of different techniques. In this study, we proposed a hybrid machine learning algorithms such as CNN-LR, CNN-GBDT, and CNN/Knn for credit risk prediction in P2P lending club.

In our proposed approach, we employ convolutional neural networks as feature extractors to capture important information from P2P lending data. The input data is processed and converted into a one-dimensional array through the flattening layer. Subsequently, we remove the final fully connected layer as well as the soft-max layer. To perform prediction, we stack logistic regression, gradient boosting decision tree, and k-nearest neighbor algorithms after the flattening layer in the model architecture. Addressing this gap can contribute to a more comprehensive understanding of credit risk in P2P lending platforms and facilitate the development of more accurate and interpretable credit risk assessment models.

Methodology

The primary goal of this study, as shown in Figure 1, is to predict defaulting borrowers on the P2P network. This will be accomplished using hybrid machine learning algorithms.

Figure 1. Workflow of our (Ileberi, Sun, and Wang 2022).

Data Preprocessing

The data for this study was obtained from the lending club’s accessible P2P loan data. This dataset contains all of the information obtained by the lending club platform. The identity of the borrower, the purpose of the loan, the borrower’s loan history, loan status, annual income, interest rate, and other factors are the main characteristics.

Non-defaulter and defaulter are the two categories of the target variable loan status. A total of 188,846 loans were made, of which 151,871 were non-defaulters and 36,975 were defaulters.

Data from P2P lending clubs are primarily made up of less defaulter customers than non-defaulters. Logistic regression, gradient boosting decision trees, and K-nearest neighbor can all experience problems when learning from a very imbalanced training set, just like the majority of machine learning methods. The bulk of the samples for data that are imbalanced belong to the dominant group. To minimize the total prediction error rate and improve the model’s accuracy, the target variable must be balanced first.

Re-sampling is the process of frequently obtaining samples from the training dataset. These samples are then used to train a certain model in order to gather additional data regarding the fitted model. The targets are to enhance accuracy, identify inconsistency, and obtain more information about a sample. The Synthetic Minority Oversampling Technique, oversampling the minority (defaulter), and under-sampling the majority (non-defaulter) class are the most prevalent re-sampling techniques. In this study, the synthetic minority oversampling technique is utilized as a preprocessor to address the issue of an imbalanced dataset.

SMOTE, as shown in Figure 2, generates synthetic data using a k-nearest neighbor technique. Prior to determining the data’s k-nearest neighbors, it first randomly selects data from the minority class (Elreedy and Atiya 2019). Random data would be mixed to produce synthetic data when the k-nearest neighbor is chosen.

Figure 2. SMOTE: With current instances as a starting point, SMOTE creates new minority instances synthetically. For each class label and its neighbors, the algorithm selects a sample of the feature space. Then, new instances are created by fusing the characteristics of the target cases with those of their neighbors. As a result, the new examples are not identical to the existing minority cases. After incorporating the entire dataset, SMOTE enhances the proportion of merely the minority instances.

Logistic Regression

Logistic regression is a well-known, widely used, and frequently successful supervised machine learning technique (Caie, Dimitriou, and Arandjelovic′ 2021). Logistic regression is a variant of linear regression that can handle binary outcomes or binary classifications (Giudici 2018; Kurt, Ture, and Kurum 2008). It links a number of various independent variables. Logistic regression models are used to estimate the likelihood of an occurrence, such as whether or not a peer-to-peer lending club customer would repay the loan. The output of the logistic regression model is a probability that ranges from 0 to 1 (Theerthagiri et al. 2020), which shows whether a specific instance belongs to a particular class or not (Chen 2011). By analyzing borrower-specific variables, a logistic regression model calculates the probability of default, providing an estimate of the probability that a borrower will fail to meet their obligations (Giudici, Hadji-Misheva, and Spelta 2020).

The sigmoid function is used to translate the input data into a probability. The total of the weighted input values is transformed into values between 0 and 1 by the sigmoid function, as shown in Figure 3, often known as the logistic function. The following equations illustrate the hypothesis as the sigmoid function (Chen et al. 2022):

Figure 3. Sigmoid (logistic) function and its derivative (Zhang et al. 2020).

(1) $\mathit{\sigma }(\mathit{z})=\frac{1}{1+{\mathit{e}}^{-\mathit{z}}}$(1)

where $\mathit{z}$ represents the weighted sum of input values.

The value of $\mathit{z}$ in sigmoid function can be written as the following:

(2) $\mathit{z}={\theta }^T.\mathit{x}$(2)

where $\mathit{x}$ denotes the training data.

The output of the sigmoid function can be defined as

(3) $h_{\theta }(\mathit{x})=\mathit{\sigma }({\theta }^T.\mathit{x})$(3)

The performance of the model is summed up by the cost function. To put it another way, we evaluate the accuracy of the model’s predictions by looking at the cost function. The logistic regression’s parameters are learned using maximum likelihood estimation (Bussmann et al. 2021). The logistic regression’s cost function is provided by:

(4) $\mathit{J}(\mathit{\theta })=-\frac{1}{m}\sum _{\mathit{i}=1}^m[y^{(\mathit{i})}log(h_{\theta }(x^{(\mathit{i})}))+(1-y^{(\mathit{i})})log(1-h_{\theta }(x^{(\mathit{i})}))]$(4)

where $\mathit{x}$ denotes the training data, $\mathit{y}$ denotes the class or label of the training data, which is either defaulter or non-defaulter customers, and $\mathit{h}$ denotes the output of the sigmoid function previously illustrated.

In binary classification, the cost function can be expressed as follows to determine if a consumer of a peer-to-peer lending club will default or not:

(5) $\mathit{cost}(h_{\theta }(\mathit{x}),\mathit{y})=\left\{\begin{array}{c}-log(h_{\theta }(\mathit{x})),\mathrm{if}\mathit{y}=1\\ -log(1-h_{\theta }(\mathit{x})),\mathrm{if}\mathit{y}=0\end{array}\right.$(5)

Gradient descent is an iterative optimization method for finding the minimum of a differentiable function. Gradient descent aims to find the optimal weights that minimize the loss function of the model by adjusting the value of our weights so that it continuously converges to a minimal point.

In this method, we may discover the best option while minimizing the cost relative to the model parameters:

(6) $\underset{\theta }{\mathit{min}}\mathit{J}(\mathit{\theta })$(6)

Assume that there are $\mathit{n}$ features and parameters in the $\mathit{\theta }$ vector. The cost function can be minimized by using the gradient descent approach for each parameter, ${\theta }_j$.

(7) ${\theta }_j\leftarrow {\theta }_j-\mathit{\eta }\frac{\mathrm{\partial }}{\mathrm{\partial }{\theta }_j}\mathit{J}(\mathit{\theta })$(7)

where $\mathit{\eta }$ is the learning rate.

The derivative of the cost function $\mathit{J}(\mathit{\theta })$ is given by

(8) $\frac{\mathrm{\partial }}{\mathrm{\partial }{\theta }_j}\mathit{J}(\mathit{\theta })=\frac{1}{m}\sum _{\mathit{i}=1}^m(h_{\theta }(x^{(\mathit{i})})-y^{(\mathit{i})})x_j^{(i)}$(8)

Placing equation (8)(8) $\frac{\mathrm{\partial }}{\mathrm{\partial }{\theta }_j}\mathit{J}(\mathit{\theta })=\frac{1}{m}\sum _{\mathit{i}=1}^m(h_{\theta }(x^{(\mathit{i})})-y^{(\mathit{i})})x_j^{(i)}$(8) in equation (7)(7) ${\theta }_j\leftarrow {\theta }_j-\mathit{\eta }\frac{\mathrm{\partial }}{\mathrm{\partial }{\theta }_j}\mathit{J}(\mathit{\theta })$(7) , we get

(9) ${\theta }_j\leftarrow {\theta }_j-\mathit{\eta }\frac{1}{m}\sum _{\mathit{i}=1}^m(h_{\theta }(x^{(\mathit{i})})-y^{(\mathit{i})})x_j^{(i)}$(9)

Gradient-Boosted Decision Trees

Gradient-boosted decision trees are a machine-learning method for improving a model’s prediction power over subsequent learning phases. In order to reduce the loss function, each iteration of the decision tree entails changing the values of the coefficients, weights, or biases assigned to each of the input variables being used to predict the target value. Gradient refers to the little adjustments made at each stage of the process while boosting is a method for accelerating the increase in prediction accuracy to an appropriate optimal level.

Gradient boosting decision trees, is an ensemble model that consists of decision trees trained in a sequential manner (Friedman 2001). During each iteration, GBDT learns by fitting the negative gradients, which are also referred to as residual errors, to train the decision trees (Ke et al. 2017).

Gradient-boosted decision trees are a popular technique to address prediction challenges in both the classification and regression areas (Wang et al. 2018). The method enhances learning by streamlining the goal and lowering the number of iterations required to arrive at an appropriately optimal solution.

Gradient boosting decision tree is one type of iterative decision tree technique (Li et al. 2022; Xuan et al. 2019). The predictions of various trees are combined by GBDT, which uses decision trees as its base learner (Zhang and Jung 2020). A new decision tree is trained at each stage to fit the residual between the current prediction and the observed data. Multiple decision trees make up the algorithm, and the outputs of all the trees are combined to produce the final outcome (Bentéjac, Csörgö, and Martnez-Muñoz 2021). The number of trees and the number of proportionally-sized leaves per tree are the two hyper-parameters that most strongly influence the structure of a GBDT (Zhang et al. 2019).

Consider a P2P lending club training data $\mathit{X}=\{X_i{\}}_{i=1}^n$, with $X_i\in R^D$ and their target variables $\mathit{Y}=\{y_i{\}}_{i=1}^n$ with $y_i\in \{0,1\}$, where 1 is a customer whose credit status is a non-defaulter and 0 is a defaulter customer. Each decision tree $\mathit{F}(\mathit{x})$ is customized as follows to maximize its performance (Si et al. 2017):

(10) $F_m(\mathit{x})=F_{\mathit{m}-1}(\mathit{x})+{\rho }_m{h}_m(\mathit{x})$(10)

where $\mathit{h}(\mathit{x})$ is the decision tree function, ${\rho }_m$ is the weight of the $m^{\mathit{th}}$ decision tree function $h_m(\mathit{x})$.

The goal is to choose a classification function $\mathit{F}(\mathit{x})$ to minimize the aggregation of some specified function $\mathit{L}(y_i,\mathit{F}(X_i))$ si2017gradient:

(11) $F_0=\underset{F}{\mathit{argmin}}\sum _{\mathit{i}=1}^n\mathit{L}(y_i,\mathit{F}(X_i))$(11)

The function estimation F is taken into account by gradient boosting in an additive form:

(12) $\mathit{F}(\mathit{x})=\sum _{\mathit{M}=1}^T{f}_m(\mathit{x})$(12)

where $\mathit{T}$ is the number of iterations.

In Figure 4, the operational principles of GBDT are depicted. Gradient boosting decision trees merge a number of weak learners into a single dominant learner. In this case, single decision trees are poor learners. Each tree works to reduce the error of the one before it since they are all connected in a sequence. The model is strengthened such that each new learner fits into the residuals of the previous stage by modifying the weak learners. The final model produces a potent learner by fusing the results from each phase. A loss function is utilized to find the residuals.

Figure 4. Gradient boosting decision tree structure (Deng et al. 2021).

K-Nearest Neighbors

The k-nearest neighbors (k-NN) method is a straightforward technique that may be applied to numerous practical issues in a variety of fields, including banking, healthcare, recommendation systems, and much more. It is a well-known non-parametric machine learning classification algorithm (Moosavian et al. 2013; Zhang et al. 2022). k-NN assigns the class label based on the majority of the k-nearest patterns in the data space (Kataria and Singh 2013; Kramer 2013).

There are just two options given to the user: (1) the distance metric, and (2) the k-neighbors to be used (Neath and Johnson 2010). The number of neighbors is often determined through cross-validation or by assessing the effectiveness of the classifier on a second, test data set (Neath and Johnson 2010).

The k-NN algorithm has the following steps (Taunk et al. 2019; Taunk et al; Zhang et al. 2022).

Step 1: Consider a lending club training dataset $\mathit{D}=\{(x_i,y_i)\},\text{breaki}=1,2,\dots ,\mathit{n}$ be a set of n-labeled samples.

Step 2: Let $\mathit{x}$ be the new instance, we need to be classified.

1. Choose the k-value for the neighbors.

2. Find the Euclidean distance between the k closest neighbors.

3. The k-nearest neighbors should be identified using the estimated Euclidean distance.

4. A class with the most nearby neighbors should be assigned.

Figure 5 illustrates the working strategies of the k-nearest neighbor algorithm. In the context of the P2P lending club dataset, which consists of labeled instances as either non-defaulter or defaulter, k-NN operates by calculating the distances between the new customer and each of the existing customers. The k customers closest to the new customer are then selected based on these distances. Subsequently, the label with the highest frequency among the selected k customers is chosen as the prediction for the new customer in the case of classification.

Figure 5. working strategies of the k-nearest neighbor algorithm.

Convolutional Neural Network

The components of 1D-CNN are 1-dimensional convolution layers, pooling layers, dropout layers, and activation functions for processing the 1-dimensional data (Qazi, Almorjan, and Zia 2022), as shown in Figure 7. The central component of a CNN is the convolutional layer, which is also where the majority of computation takes place. It needs input data, a filter, and a feature map, among other things. The convolution layer applies the filter to the input. The specific properties associated with the data points are displayed on the feature map that is produced as a result of performing the filtering procedure frequently. In the linear process known as convolution, a set of weights are utilized to control the input multiplication. The inputs are multiplied by the weights, also known as kernels, of the one-dimensional array, as shown in Figure 6. This procedure generates several values, each with a distinct value for each pass, and results in a feature map.

Figure 6. Convolution layer operation (1D) (Kuo and Huang 2018).

Figure 7. Pooling layer (1D) (Aurangzeb et al. 2021).

Each value is then sent to the Rectified Linear Unit activation function following the computation of the feature map.

The ReLU activation function can be mathematically defined as:

(13) $\mathit{f}(\mathit{x})=\max (0,\mathit{x})$(13)

where $\mathit{x}$ represents the input to the function. If the input is greater than zero, the function returns the input value itself $\mathit{f}(\mathit{x})=\mathit{x})$. However, if the input is less than or equal to zero, the function outputs zero $(\mathit{f}(\mathit{x})=0)$.

A pooling layer follows every convolution layer. More precisely, following the application of a ReLU to the feature maps produced by a convolutional layer. Each feature map is individually processed by the pooling layer, which creates a new set of the same number of pooled feature maps.

Pooling is used to apply a pooling process, which is similar to a filter, to feature maps. The filtering or pooling operation’s size is smaller than the feature map’s. Due to this, its primary purpose is to gradually lower the dimensionality or representation of the network in order to reduce the number of parameters and computations in the network and the feature map’s vector size is down-sampled without losing any information (Ragab, Abdulkadir, and Aziz 2020).

In the pooling operation, there are two often utilized functions (as shown in Figure 8): Average pooling is used to determine the average of the elements in the feature map area that the filter is covering. In contrast, max-pooling creates the most prominent feature in a specific area of the feature map.

Figure 8. Pooling layer (1D) (Aurangzeb et al. 2021).

Convolutional neural networks and the LR/GBDT/k-NN algorithm are used in this technique to create a credit risk prediction model for per-to-peer lending. The first phase involves employing the remarkable feature extraction capabilities of the convolutional neural network to extract features from the original lending club data and produce a new feature matrix (Berhane et al. 2023; Melese et al. 2023). Second, using the updated feature matrix as input data, LR/GBDT/kNN algorithms are used to construct the prediction model.

The lending transaction data is separated from the multiple borrower characteristics by the convolutional layer, and the pooling layers reduce the size of the feature maps. As a result, the pooling layer minimizes both the number of parameters that must be determined from the lending data and the amount of computation performed within the network. The feature maps obtained by repeatedly using numerous of convolution and pooling layers on the lending data are flattened into an one-dimensional array and utilized as inputs for the logistic regression, gradient boosting decision tree, and k-nearest neighbor algorithms (Berhane et al. 2023; Melese et al. 2023). So, the fully connected and soft-max layer of the convolutional neural network is eliminated and replaced by LR/GBDT/kNN in the architecture of our proposed models (as shown in Figure 9).

Figure 9. The architecture of our proposed model (Melese et al. 2023): Three classifiers—logistic regression, gradient boosting decision tree, and k-nearest neighbor—are layered after the flattened layer by removing the final fully connected and soft-max layer. The input features extracted by the CNN is feed into the logistic regression, gradient boosting decision tree, and k-nearest neighbor algorithms. k-nearest neighbor, decision trees with gradient boosting, and logistic regression have all been used for classification.

Results and Discussion

P2P systems are still in the early stages of development when it comes to the processing of credit and the appraisal of the risk associated with it. Several P2P systems lack accurate credit repayment prediction, which might be used to avoid financial loss. So, it is critical to predicting the credit default rate in P2P lending. In this study, a hybrid machine learning prediction approach for default on P2P credit is developed.

The P2P lending club dataset, which includes 18 input features and one target variable, was used to validate this study. Due to numerous missing values and outliers in the data, the dataset must undergo a data processing phase. The median and mode values are used to fill in missing values in the data. A row that has more than 30% of its values missing is removed.

An imbalanced dataset refers to a situation where the distribution of classification classes within the dataset is not equitable. In the case of P2P lending clubs, the data exhibits an imbalance, with fewer instances of defaulters compared to non-defaulters. To enhance the overall predictive performance and accuracy of the model, it is imperative to address this class imbalance issue.

To tackle the problem of an imbalanced dataset, the synthetic minority oversampling technique is employed as a preprocessor. SMOTE is a method that generates synthetic data to balance the distribution of the minority class. It utilizes the concept of k-nearest neighbors to create synthetic instances. Initially, random samples from the minority class are selected, and then the k-nearest neighbors of these samples are computed. The synthetic data points are subsequently generated by interpolating and mixing the features of the randomly selected samples with those of their k-nearest neighbors.

By using the k-nearest neighbor approach, SMOTE effectively augments the minority class, thereby mitigating the class imbalance. The synthetic data produced through this process assists in providing more representative training examples for the minority class, improving the model’s ability to learn and make accurate predictions.

In our model, the CNN architecture comprises multiple layers, including convolutional, dropout, pooling, and dense layers. The first convolutional layer utilizes 32 filters with a kernel size of 3, activated by the rectified linear unit (ReLU) function. This layer convolves over the input data, generating feature maps. A subsequent convolutional layer follows with 64 filters of size 3, extracting higher-level features. Further refining and extracting complex features, a third convolutional layer with 128 filters is employed. To address overfitting, the model uses dropout regularization and max-pooling. A dropout layer with a 0.5 rate randomly deactivates 50% of neurons during training, promoting generalized learning. Max-pooling reduces spatial dimensions while retaining crucial information by selecting the maximum value within each pooling window, enhancing network efficiency and translation invariance.

To achieve the desired outcomes, the CNN undergoes 35 training iterations using the Adam optimizer, ReLU activation function, max-pooling, padding set to “same,” and a dropout rate of 0.5. In our experiments, the CNN served as a feature extractor. After the features have been extracted and flattened into a one-dimensional array, the logistic regression, gradient boosting decision tree, and k-nearest neighbors classifiers are sequentially applied for prediction. In this process, the fully connected layer, along with the subsequent soft-max layer typically used in the fully connected CNN architectures, is removed. Instead, the flattened layer directly serves as the input for logistic regression, gradient-boosting decision trees, and K-nearest neighbors classifiers.

When training the GBDT model, we took into consideration the learning rate and the n-estimator parameters, which determine the number of decision trees and tree depth. The model’s predictive performance is enhanced through multiple learning iterations. The n-estimator indicates the number of trees utilized by the model, while the learning rate determines the speed at which the model incorporates new data. To minimize the loss function, the decision tree iteratively adjusts the coefficients, weights, or biases assigned to each lending data input variable used for predicting the target value, distinguishing between defaulters and non-defaulters.

In order to assess the performance of the models, it is necessary to employ suitable evaluation metrics. Basic metrics such as accuracy, precision, recall, F1-score, and AUC are commonly utilized to measure the effectiveness of hybrid machine learning algorithm models. These metrics provide valuable insights into the performance and predictive capabilities of the models.

Accuracy alone is inadequate for evaluating the performance of the model due to the imbalanced nature of the datasets from the peer-to-peer lending club. Thus, we employ a different evaluation metric, called the confusion matrix, to assess the effectiveness of the hybrid machine-learning algorithms. The confusion matrix provides a summary of the classifier’s classification performance with respect to the given test data (Ting, Vijayakumar, and Schaal 2010). It facilitates the visualization of the outputs of a classification task by arranging the various prediction outcomes and findings in a tabular format. When evaluating the performance of the default prediction model for lending club customers, four distinct results are available.

1. True Positives (TP) – There are situations where a prediction made by the model comes true, such as when a customer experiences non-default and the prediction made by the model also occurs.

2. True Negatives (TN) – instances where the model claims that nothing has happened and nothing has actually happened, i.e., where the customer has not defaulted and the model has not predicted defaulting customers.

3. False Positives (FP) – A customer doesn’t have non-default, but the model forecasts default in cases when the algorithm states something has happened when it hasn’t. Type I errors are the ones that fall under this category.

4. False Negatives (FN) – There are circumstances where the model offers no promises, such as when something has already occurred, such as when a customer goes into default but the model didn’t predict default. This defect, known as a Type II error, is extremely dangerous.

Precision is a measure of the proportion of positive class predictions that actually belong to the positive class. The ratio of true positives (TP) to the sum of true and false positives (TP + FP) is one way to describe it.

(14) $\mathit{Precision}=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FP}}$(14)

Recall, also known as the True Positive Rate (TPR), measures how often positive outcomes were correctly predicted relative to all other positive outcomes.

(15) $\mathit{Recall}=\frac{\mathit{TP}}{\mathit{TP}+\mathit{TN}}$(15)

The F1–score, a machine learning evaluation metric, is used to evaluate the performance of a model. It takes into account a model’s recall and precision measures.

(16) $\mathit{F}1-\mathit{Score}=\frac{2\mathit{TP}}{2\mathit{TP}+\mathit{FP}+\mathit{FN}}$(16)

Table 1 presents a comprehensive analysis of the performance of the hybrid CNN-LR/GBDT/k-NN model for peer-to-peer lending prediction, utilizing all 18 input features. The CNN-LR hybrid model achieved an accuracy of 85.40%, correctly classifying around 85.40% of customers and demonstrating high precision (92.41%) in identifying defaulters. The CNN-GBDT hybrid model outperformed with an accuracy of 88.73%, achieving a precision of 94.65% and effectively identifying defaulters. The CNN-kNN hybrid model exhibited the highest performance, with an accuracy of 91.87%, precision of 91.68%, and strong discriminatory power (AUC of 91.86%). Overall, the Hybrid CNN-kNN model demonstrated superior performance in accurately predicting default/non-default status, while the CNN-GBDT hybrid model also performed well.

Table 1. Performance of hybrid CNN-LR/GBDT/k-NN (%): using all input features(18).

Performance Metrics	Accuracy	Precision	Recall	F1-score	AUC
CNN-LR	85.40	92.41	80.34	85.95	85.63
CNN-GBDT	88.73	94.65	83.23	88.57	89.16
CNN-kNN	91.87	91.68	91.16	91.42	91.86

According to the findings presented in Table 2, our hybrid models, namely CNN-kNN, CNN-LR, and CNN-GBDT, demonstrated enhanced accuracy when important features were selected. Specifically, the CNN-kNN model achieved the highest prediction performance with an accuracy of 97.60%. Following closely behind, the CNN-GBDT model achieved an accuracy of 91.60%, while the CNN-LR model exhibited an accuracy of 90.40%. These results highlight the effectiveness of our hybrid models in accurately predicting outcomes once essential features have been identified.

Table 2. Performance of Hybrid CNN-LR/GBDT/k-NN (%): using important input features(10).

Performance Metrics	Accuracy	Precision	Recall	F1-score	AUC
CNN-LR	90.40	96.58	84.96	90.39	90.77
CNN-GBDT	91.6	97.23	86.55	91.58	91.92
CNN-kNN	97.60	96.69	98.32	97.49	97.57

Furthermore, Table 2 also showcases that when important input features were utilized, the hybrid CNN-GBDT model exhibited the highest precision, achieving an impressive precision value of 97.23%. On the other hand, the hybrid CNN-LR model achieved a precision of 96.58% when important input features were employed. These results signify that the CNN-kNN, CNN-GBDT, and CNN-LR algorithms provide effective means for predicting credit risk in platforms used by investors and lending companies.

The Receiver Operating Characteristic (ROC) curve provides valuable insights into the trade-off between sensitivity (the ability of the classifier to correctly identify positive instances) and specificity (the ability of the classifier to correctly identify negative instances). It illustrates the performance of a binary classification model across various classification thresholds. By adjusting the classification threshold, the balance between these two measures can be altered.

Figure 10 shows the ROC curve of our proposed models, which provides a comprehensive evaluation of the classification model’s performance across all classification thresholds. Figure 10a displays the ROC curve of the hybrid models incorporating 10 crucial input features, while Figure 10b displays the ROC curve of the hybrid models incorporating all input features (18).

Figure 10. ROC curve of hybrid CNN-LR/GBDT/k-NN.

In conclusion, our evaluation of the model performance using two distinct datasets has demonstrated that the inclusion of important features significantly enhances accuracy, yielding a superior rate of 97.60%. Therefore, we confidently assert that hybrid machine learning algorithms are the most effective means for predicting default customers.

Table 3 presents the performance evaluation of logistic regression, gradient boosting decision tree, and k-nearest neighbors algorithms independently, without hybridization with convolutional neural networks. In light of the findings, it is evident that the performance of logistic regression, gradient boosting decision Tree, and k-nearest neighbors does not surpass that of our proposed methodologies (hybrid CNN with LR, GBDT, or k-NN).

Table 3. Performance of LR/GBDT/k-NN (%): using important input features(10).

Performance Metrics	Accuracy	Recall	F1-Score	F1-score	AUC
LR	81.09	83.14	82.99	83.06	81.82
GBDT	82.20	85.62	80.18	82.81	82.35
kNN	86.20	86.22	88.06	87.13	86.07

The model’s error rate is correlated with the value of k in the k-NN method, making it challenging to determine the ideal k value for achieving the highest accuracy. Figure 11a displays the error rate of the hybrid CNN-kNN model, revealing that the error rate increases as k increases.

Figure 11. Error rate of k-NN & feature importance.

The term “feature importance” describes methods that evaluate input features based on their effectiveness in predicting a specific target variable. According to Figure 11b, the key input features for determining credit risk in peer-to-peer lending are interest rate, grade, annual income, payment, and length of employment.

Finally, we made a comparison between our findings and the most recent findings. When compared to the state-of-the-art as shown below in Table 4, our suggested method outperforms in all metrics for identifying credit risk in P2P lending clubs.

Table 4. Performance of earlier, related work.

Authors	Methods	Accuracy	Precision	Recall	F1-Score
	Logistic regression	86.61	86.66	86.99	86.82
	SVM	87.38	84.50	91.52	87.87
Chang et al. (2022)	DT	87.58	84.12	91.76	87.77
	RF	87.90	84.90	91.40	88.03
	XGBoost	87.90	85.52	91.22	88.28
	LightGBM	87.58	85.18	90.98	87.98
	LR	64.62	77.14	41.54	54.0
	RF	74.36	85.19	58.97	69.70
Gao et.al. (Gao and Balyan 2022)	XGBoost	88.89	61.54	90.34	72.70
	LightGBM	87.82	70.26	90.34	78.10
	LightGBM + linear blending	84.36	91.36	75.90	82.90
	CNN	67.27	64.24	64.94	66.43
	ANN	66.85	67.91	64.50	65.95
Ko et al. (Ko et al. 2022)	LR	66.87	67.05	66.44	66.72
	Bayesian classifier	64.27	62.44	72.16	66.82
	LightGBM	68.57	69.73	65.66	67.62
	k-NN	86.69		87.78	92.86
	SVM	87.77		87.78	93.49
Muslim et al. (Muslim et al. 2023)	RF	87.78		87.90	93.47
	Stacking-AdaBoost	87.688		87.82	93.43
	Stacking-LightGBM	87.86		87.91	93.52
	Stacking-XGBoost	87.95		87.99	93.57
	LGBFS-Stacking-XGBoost	91.43		91.65	91.65

Conclusion

An essential factor that must be taken into account is the credit risk prediction for P2P lending. P2P lending organizations and other financial businesses would benefit from having a reliable method of predicting credit risk. Thus, having such a prediction would assist both the financial sector and society in preventing numerous economic crises. The absence of regulation and a reliable method to assess the degree of credit risk played a part in this horrible condition. As a result of such thought, society is pushing for some sort of prediction model to address the issue.

Machine learning and deep learning techniques are currently receiving significant attention, particularly for loan evaluation and credit risk prediction in financial institutions. Existing studies have primarily focused on the prediction performance of individual machine learning algorithms and their comparison to traditional credit scoring models. However, it is important to note that relying solely on machine learning algorithms may not effectively extract important information from P2P lending data. This highlights the need for exploring hybrid machine learning algorithms that combine the strengths of different techniques.

In this study, we developed hybrid machine learning algorithms, such as CNN-LR, CNN-GBDT, and CNN-kNN, for credit risk prediction in P2P lending clubs. To conduct this research, we utilize publicly available P2P loan data from the lending club, which provides comprehensive information obtained from their platform.

The target variable in our dataset is the loan status, which can be classified into two classes: non-defaulters and defaulters. However, it is important to note that the dataset exhibits class imbalance, meaning that the distribution of the classification classes is not uniform.

To address the issue of data imbalance between non-defaulters and defaulters, we employ the SMOTE sampling approach. This technique helps mitigate the imbalance and ensures a more robust analysis of the dataset.

In our proposed approach, we utilize convolutional neural network as feature extractors to capture important information from P2P lending data. The input data is processed and converted into a one-dimensional array through the flattening layer. Subsequently, we remove the final fully connected layer as well as the softmax layer. To perform prediction, we stack logistic regression, gradient boosting decision tree, and k-nearest neighbor algorithms after the flattening layer in the model architecture.

The experimental results show that the hybrid CNN-kNN model outperforms the CNN-GBDT and CNN-LR models based on the performance metrics accuracy, recall, F1-score, and area under the curve (AUC) for both all input and important features. The accuracy of the hybrid machine-learning model increased as we used important input features. This shows that hybrid machine learning models effectively identify and categorize credit risk in peer-to-peer lending clubs, hence assisting in financial loss prevention.

Based on the findings, it is advised that stakeholders and policymakers integrate hybrid machine learning models, such as the CNN-kNN model, into credit risk assessment processes in peer-to-peer lending. These models have proven superior in identifying and categorizing credit risk. Prioritizing the use of important input features is crucial to enhance prediction accuracy and strengthen risk assessment frameworks. Insights from the hybrid models also call for the reinforcement of risk management strategies to mitigate potential financial losses. Additionally, stakeholders and policymakers should regularly monitor and adapt their approaches to ensure optimal credit risk prediction and prevention of financial crises. These recommendations aim to guide stakeholders in leveraging the advantages of hybrid machine learning models and enhancing credit risk management practices in peer-to-peer lending.

Considering the method of comparing models in terms of interpretability using Shapley or Shapley-Lorenz values Giudici and Raffinetti (2021), is an excellent addition to our future work. By incorporating these measures, we can assess the importance of individual predictors in each model and compare their contributions. This will provide valuable insights into the interpretability and understanding of the models’ decision-making processes. Using Shapley values or Shapley-Lorenz values will enable you to quantify the relative importance of each predictor, highlighting the variables that have the most significant impact on the predicted outcomes. This approach will enhance the transparency and explain-ability of our models, allowing for a more comprehensive analysis and interpretation of the credit risk assessment. By considering this method, we can provide a more holistic evaluation of the models, considering both their predictive performance and their interpretability. This will contribute to a better understanding of the factors driving the credit risk assessment and facilitate more informed decision-making in the P2P.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Supplemental material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/08839514.2024.2358661