Data driven based fuzzy logic inference algorithm for quality of experience modeling for video streaming in LTE network: Case of Addis Ababa City LTE network

Abstract

Nowadays, video streaming has become one of the most dominant services due to increasing interest in watching online television programs and video-on-demand activities. Providing this service requires high quality, high speed and high-capacity network infrastructures. In this work, we propose a data-driven-based fuzzy logic inference algorithm for quality of experience (QoE) modeling of video streaming services in Addis Ababa City LTE network. The proposed fuzzy logic inference system model is used to measure the user perception from the key quality of indicator (KQI) parameters. The model is essential to replace conventional subjective measurement techniques that are costly and inefficient. In addition, the proposed fuzzy logic inference system model is helpful for business decision making, network planning and optimization activities. To analyze the performance of the proposed model, we consider main KQI parameters such as video streaming start success rate, video streaming start delay, video streaming play disconnection rate, video streaming stall frequency and video streaming stalled time rate. We perform numerical simulation to analyze the proposed model and to validate the impacts of main KQI parameters on the quality of experience in LTE video streaming services. The simulation results show that video streaming stall frequency and video streaming start delay rate play a major impact on user perception by 33% and 25%, respectively. Besides, validation of the results shows that the proposed fuzzy logic inference system model is accurate, consistent and linear compared to currently existing models.

Keywords:

• LTE
• key quality indicator
• mean opinion score
• quality of experience
• quality of service
• video streaming

PUBLIC INTEREST STATEMENT

Video streaming has become one of the most dominant services due to increasing interest in watching online television programs and video-on-demand activities. Providing this service requires high quality, speed and capacity network infrastructures. In this work, we develop data-driven based fuzzy logic inference algorithm for quality of experience modeling to video streaming services in Addis Ababa City LTE network. The proposed algorithm helps to measure the user perception from key quality of service parameters for video streaming services in LTE network. To analyze the performance of the proposed algorithm, we consider main key quality indicator parameters such as video streaming start success rate, video streaming start delay, video streaming play disconnection rate, video streaming stall frequency and video streaming stalled time rate. The proposed model is essential to replace conventional subjective measurement techniques that are costly and inefficient. It also helps for business decision making, network planning and optimization activities.

1. Introduction

According to the recent Cisco visual networking index, video traffic will quadruple and account for 82% of all IP traffic at the end of 2022 (Cisco, 2019). The forecast says that nearly half of all devices and connections will be video capable (Cisco, 2019). Hence, modeling, analyzing and characterizing video streaming algorithms and systems have become a hot area of research in both the academies and industries (G´omez et al., 2012; ITU-T, 2008). A video streaming can be either an integral part of a browser, plug-in, separate program or dedicated device. Video files may also include in embedded players. There are two types of streaming video content distribution such as on-demand and programmed-time streaming (ITU-T, 2008). On-demand streaming allows the viewer to choose what content to play at any time. This type of streaming increases the bandwidth costs due to the necessity of establishing a new network stream for each player. Whereas, programmed-time streaming requires a channel for a viewer at specific time that is programmed by the streamer. This type of streaming minimizes bandwidth costs as it broadcasts at one time. The quality of experience of the users can be measured from Quality of Service (QoS) parameters (Cisco, 2019; G´omez et al., 2012; ITU-T, 2008; Bajpai and Schonwalder, 2015).

Quality of experience (QoE) in mobile networks is defined by International Telecommunication Union (ITU) as the degree of delight of the user applications or services (Cisco, 2019; G´omez et al., 2012). QoE shows the overall acceptability of an application or service when perceived subjectively by the network users. It includes the complete end-to-end system effects such as client, terminal, network, services infrastructure and others. In this regard, QoE modeling aims to achieve high user expectations and requirements on network reliability and cost-effective communication, access to anytime and anywhere services, and interworking across multiple operator domains (G´omez et al., 2012). Whereas, due to the move toward converged all IP wireless network environments, modeling end-user QoE has become a challenging task. Besides, the overall acceptable QoE standard is influenced by user expectations and contexts. Therefore, QoE modeling is considered to be a multidimensional construct, encompassing both objective (performance-related) and subjective (user-related) aspects (Huang et al., 2018). Thus, to incorporate these multidimensional constructs, QoE has been modeled by using both the quality of service (QoS) which is primarily a technical, objective and model-oriented concept; and by using the user experience (Planel-Ratna & Juwaheer, 2021), which is generally considered as a more user-oriented concept. The former approach focuses on the impact of network and application performance on the perception of the user quality; while the latter approach primarily deals with the individual user’s experience derived from encounters with the system, and it is affected by expectations, prior experiences, feelings, thoughts and contexts (Planel-Ratna & Juwaheer, 2021).

Quality of service (QoS) shows the service providers ability to guarantee a certain level of consistent performance in the network to a data flow by means of monitoring and controlling. By using the key QoS parameters, QoE can be modeled subjectively and objectively (G´omez et al., 2012; Lo Duca & Marchetti, 2020). The subjective measurement is the mean opinion score (MOS) of the users perception. The MOS value is dependent on user factors such as user expectations, security requirements, demography, access types, user mobility and location, social contexts and others. In this regard, the MOS measurement has a 5-point scale score such as 5-Excellent, 4-Good, 3-Fair, 2-Poor and 1-Bad (G´omez et al., 2012). Whereas, subjective-based QoE measurement is costly, limited to a certain sample level, has insufficient system knowledge and difficult to take real-time measurements (G´omez et al., 2012), (Pal & Triyason, 2018).

However, objective based measurements estimate the QoE by using a parametric model and without the involvement of the users’ opinion. The parametric model is a function of the network-level end-to-end QoS parameters (BarakoviT & Skorin-Kapov, 2013). Objective-based QoE measurement is essential for user-centric service provisioning that help to deliver a service by considering the interest and satisfaction level of customers. Therefore, this approach helps the service provider to identify the customers’ interest and satisfaction level before delivering a service on market. The objective measurement provides several advantages for both the users and the service providers. It helps to predict QoE of a specific service from the most significant key quality indicator (KQI) parameters. Objective-based QoE modeling is network centric and can be measured objectively (G´omez et al., 2012; Pal & Triyason, 2018; Nasralla et al., 2018).

In this work, we develop a data-driven-based fuzzy logic inference algorithm for quality of experience (QoE) modeling for video streaming services in Addis Ababa City Long-Term Evolution (LTE) network. The model helps to measure the user perception from quality of service (QoS) parameters for video streaming services in Addis Ababa city LTE network. To develop the proposed model, first real network traffic data is collected from selected experimental customer survey and network management system (NMS) key quality indicator parameters. The experimental customer survey data is collected through the Internet by inviting individual LTE customers to watch online broadcasting television programs and then fill the questionnaires prepared on Google Sheet. The NMS data is collected from QoS monitoring platform in Ethio-Telecom QoS monitoring section. Then, extracting and categorizing the KQI parameters have been done to map the KQI parameters to the QoE features. Regression and correlation analysis is done to remove redundancy and to identify the relationship between each KQI parameter with the QoE. The coefficient influence analysis and weighting coefficients is used to show the impact of each KQI parameter on the QoE (ITU-T, 2008; Alakhras etal., 2020).

Then, the proposed data-driven-based fuzzy logic inference algorithm is developed. Finally, the performance of the proposed model is analyzed and validated by using experimental customer survey subjective test. The subjective test is done to analyze the relationship between the KQI parameters and QoE at different network performance and customer situations. The test considers the subjective context, expectation, terminal device performance, network quality and user experiences. Besides, the accuracy and consistency of the model has been validated through simulation. The results show that the proposed data-driven QoE model for video streaming services in Addis Ababa City LTE network is highly accurate, consistent and linear. The model is sufficient to evaluate the user satisfaction level and to evaluate the impact of KQI parameters on the user perception.

The rest of the paper is organized as follows. In Section 2, summary of the review of recent related works is provided. The system model and algorithm formulation are shown in Section 3.

Simulation results are discussed in Section 4 and conclusions are drawn in Section 5.

2. Review of related works

A survey of state-of the-art research work that address the field of QoE management, which include three management aspects such as QoE modeling, monitoring and measurement, and adaptation and optimization are shown in (BarakoviT & Skorin-Kapov, 2013). The authors identify and discuss the key aspects and challenges that need to be considered to model and design quality of experience of wireless networks. The authors in (Goel et al., 2016) show that end-to-end QoS and QoE measurements are very important guideline for network operators, developers, regulators, customers and researchers. Operators may use the data to identify network performance problems, developers may use the data to predict the future network, regulators may use to identify the best serving operator, customers may use to check the issue of quality within their own network or within the operators and researchers may use the data for academic study. The work in (Pedras & Sousa, 2017) proposed a novel QoE model for universal mobile telecommunication system (UMTS) voice services and long-term evolution (LTE) data services from QoS parameters. They employ machine learning algorithms and show that the results have a good correlation and small error variation.

The authors in (Huang et al., 2018) provide a survey of the state-of-the-art on data-driven approach for QoE modeling assessment. The result shows that QoE evaluation based on data-driven approach are significant in three aspects. First, with the age of big-data, operators can know more about the users behavior and interests. Second, QoE assessment based on data-driven approach uses real data from networks instead of test data from laboratories. Third, this approach can help to evaluate the QoE of the users without knowing anything about them.

The authors in (BarakoviT & Skorin-Kapov, 2013) propose objective QoE measurement models based on QoS parameters. They present the state-of-the-art on mobile QoE modeling and outline some of the key challenges. A fuzzy logic inference-based scheduling algorithm for multi-user scalable video streaming over orthogonal frequency division multiple access (OFDMA) channels in proposed in (Omiyi et al., 2020). The video streams are precoded using a scalable video coding (SVC) scheme. The algorithm prioritizes the transmission to different users by considering video content and channel conditions. A QoS-aware, content-aware and device-aware nonintrusive medical QoE prediction model over small cell networks is proposed in (Rehman et al., 2019). The proposed prediction model utilizes a Multilayer Perceptron (MLP) neural network to predict the QoE. It also acts as a platform to maintain and optimize the acceptable diagnostic quality through a device-aware adaptive video streaming mechanism. The model is trained for an unseen dataset of input variables such as QoS, content features and display device characteristics, to produce an output QoE value. The work in (Wang et al., 2017) proposed a data-driven QoE analysis framework for video streaming. They show that a small set of QoS parameters play an important role to determine the video streaming MOS values. They show that the weights of initial buffering latency, stalling times and stalling ratio is 69.01%. The work in (Umoh & Asuquo, 2018) proposes a fuzzy logic model to assess the QoS in multimedia transmission. It evaluates the major QoS parameters such as delay, jitter and packet loss. Then, they use these parameters as inputs to evaluate the QoS parameter requirements of each network. The model is used to detect and restore any QoS deterioration in real-time and thus provides a good and efficient QoS model for customer satisfaction.

In general, the above works show the different approaches and algorithms used for QoE modeling from real-time data. Whereas, the accuracy, consistency and linearity of the proposed models still needs further improvement and customization based network types, geographical, environmental and customers contexts. Hence, based on the reviewed literatures, in this work, we develop data-driven-based fuzzy logic inference algorithm for quality of experience modeling for video streaming in Addis Ababa City LTE networks.

3. System modeling and algorithm development

The methodology that we follow for data-driven-based QoE system modeling and algorithm development is shown in figure 1. As stated in Section 1, first the LTE video streaming data is collected from experimental customer survey and Ethio-Telecom ISP datasets.

Figure 1. Methodology and algorithm development.

The NMS system datasets has an 1 hour KQI parameters collected from Addis Ababa LTE network video streaming services. An 8-month data, from 5 October 2018 to 5 July 2019, with a total size of 5,760 datasets have been collected. The customer survey data is collected by using subjective test on a streaming video in Addis Ababa LTE network.

The data analysis is performed with main KQI parameters, which have high impact on QoE of video streaming services. These include video streaming start success rate (SSR), video streaming start delay (SD), video streaming play disconnection rate (PDR), video streaming stall frequency (SF) and video streaming stalled time rate (STR; Lo Duca & Marchetti, 2020).

The SSR indicates the rate at which video streaming files are successfully played on a webpage after a user clicks the play button. It is the ratio of video streaming start success to video streaming start request (Wang et al., 2017). The SSR is affected by connect success rate, get response success rate and initial buffer success rate as (Technologies Ltd, 2015; Wang et al., 2017)

(1) $\mathrm{S}\mathrm{S}\mathrm{R}\left(\mathrm{\%}\right)=\frac{\mathrm{V}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{n}\mathrm{g}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{f}\mathrm{u}\mathrm{l}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}}{\mathrm{V}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{s}}$(1)

The PDR indicates the rate at which the downloading of a video streaming file stops or restarts before the video buffer is fully occupied due to the client or server fails to process the video service efficiently or the network transmission quality is poor (Technologies Ltd, 2015; Wang et al., 2017). The PDR is given by (Technologies Ltd, 2015; Wang et al., 2017)

(2) $\mathrm{P}\mathrm{D}\mathrm{R}\left(\mathrm{\%}\right)=\frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{w}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}\mathrm{e}\mathrm{d}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{o}\mathrm{s}\mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{u}\mathrm{p}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{w}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}\mathrm{e}\mathrm{d}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{o}\mathrm{s}}$(2)

The SD is the amount of time a user waits until a video streaming file is played after clicking the play button on a webpage. This KQI parameter depends on the end-to-end delay or latency. It is the sum of video streaming response delay, DNS query delay, TCP connection delay, get response delay and initial buffer delay as (Technologies Ltd, 2015; Wang et al., 2017)

(3) $\mathrm{S}\mathrm{D}\left(\mathrm{m}\mathrm{s}\right)=\mathrm{R}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{e}\mathrm{d}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{y}+\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{r}\mathrm{y}\mathrm{d}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{y}+\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{y}+\mathrm{b}\mathrm{u}\mathrm{f}\mathrm{f}\mathrm{e}\mathrm{r}\mathrm{d}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{y}$(3)

The STR indicates the percentage of the total stall time out of the total length of the played video streaming file (Technologies Ltd, 2015; Wang et al., 2017). This KQI depends on the TCP download retransmission rate, the stalled duration (min), play times, video streaming download throughput and average download delay as

(4) $\mathrm{S}\mathrm{T}\mathrm{R}\left(\mathrm{\%}\right)=\frac{\mathrm{V}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{d}\mathrm{d}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{P}\mathrm{l}\mathrm{a}\mathrm{y}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}\ast \mathrm{A}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{d}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{y}}$(4)

The SF is the number of times a video streaming file stalls within 1 minute when the file is being played. It depends on the stalled count, play times, video streaming download throughput (kbps), TCP download retransmission rate (%) and average download delay (min) as (Technologies Ltd, 2015; Wang et al., 2017)

(5) $\mathrm{S}\mathrm{F}\left(\mathrm{\%}\right)=\frac{\mathrm{S}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}}{\mathrm{P}\mathrm{l}\mathrm{a}\mathrm{y}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}\ast \mathrm{A}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{d}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{y}}$(5)

3.1. Categorization of KQI Parameters

Categorizing KQI parameters used to map the impact of each parameter on the QoE features. In this work, the selected five KQI parameters are mapped with selected video streaming QoE features. The results from linear regression of the data are used to identify the relationship between the subjective MOS value and the KQI parameters (Wang et al., 2017). The KQI parameters are considered as independent input variables, and the QoE is considered as dependent output variable for the regression analysis. Based on the regression results, the relationship between the KQI and QoE is formulated as

(6) $y=0.809+0.113{\mathrm{x}}_1-0.216{\mathrm{x}}_2-0.081{\mathrm{x}}_3-0.303{\mathrm{x}}_4$(6)

where y denotes the output QoE value and {x₁, x₂, x₃, x₄} represents the input SSR, SD, PDR, SF and STR values, respectively.

3.2. Feature selection

We employ Pearson correlation to find the linear relationship between KQI parameters and the QoE. Pearson correlation helps to reduce the dimension of the dataset by removing the redundant parameters (Pedras & Sousa, 2017). If r represents the Pearson linear correlation, N denote the number of samples in the dataset, and x and y denote the mean value of KQI parameters and QoE, respectively; then, the Pearson correlation is defined as (Pedras & Sousa, 2017)

(7) $r=\hspace{0.17em}\frac{{\sum }_{i=1}^N\left(X_i-x\right)\left(Y_i-y\right)}{\sqrt{{\sum }_{i=1}^N{\left(X_i-x\right)}^2}\sqrt{{\sum }_{i=1}^N{\left(Y_i-y\right)}^2}}$(7)

where X_i, Y_i represents the KQI and QoE data sets (ITU-T, 2008).

3.3. Coefficient influence analysis

We employ coefficient influence analysis with four-way analysis of variance (ANOVA) method and multiple regressions to analyze the impact of each feature on the QoE of video streaming. Before the ANOVA analysis, the KQI parameters are normalized to have zero mean and unit standard deviation. Selecting the target range depends on the nature of the data. The general expression is given by (Huang et al., 2018)

(8) $\bar{x}=\frac{x-\mathrm{m}\mathrm{i}\mathrm{n}\left(x\right)}{max\left(x\right)-\mathrm{m}\mathrm{i}\mathrm{n}\left(x\right)}$(8)

where x is the data value, min(x) is the minimum value of the dataset and max(x) is the maximum value of the datasets. Independent weighting method can be used to determine the weight of each KQI parameter coefficient and to identify the impact of each KQI parameters on the QoE features. The weight can be calculated as (Huang et al., 2018)

$w_i=\frac{\zeta \left(i\right)}{{\sum }_i^M\zeta \left(i\right)}$

(9) $\zeta \left(i\right)=\sqrt{\frac{B_i}{{\sum }_i^m{B}_i}}$(9)

When B_i denotes the correlation coefficient and w_i denotes the weighted coefficient of each KQI parameter.

3.4. Data-Driven-based fuzzy inference algorithm for QoE prediction in LTE Video streaming

We propose data-driven-based fuzzy inference system (FIS) model as shown in figure 2 to evaluate the video streaming QoE in Addis Ababa LTE network. The model incorporates the FIS with membership functions that is developed from the datasets prepared by using multiple regression and weighted coefficients.

Figure 2. Proposed fuzzy inference system (FIS)-based QoE model for video streaming service prediction.

3.5. Subjective test for proposed model analysis and validation

Finally, to show the performance and accuracy of the proposed data-driven-based fuzzy inference algorithm for QoE prediction, we compare the results from the proposed model with the subjective test results. Compared to objective tests, subjective methods are more accurate to formulate the user opinion. An objective model may not predict an average subjective opinion more accurately than the test subjects. Hence, validation should be done to show if the output of objective models is well correlated with subjective results. In this case, the subjective measurement is considered as the ground truth for user QoE evaluation. Hence, we validate our data-driven fuzzy logic inference model by using the subjective methods with the collected survey data.

4. Simulation results and analysis

4.1. Analysis of measured survey data

As stated in Section 3, the data measurement is taken for Addis Ababa LTE network that consists of a total 329 LTE sites all over the city. The experiment includes all these sites considering ten zones at different time of the day. The collected data includes data from four time schedules such as morning, afternoon, evening and during the night. Whereas, The NMS system datasets has a one hour KQI parameters collected for the LTE network video streaming services. An 8-month data, from 5 October 2018 to 5 July 2019, with a total size of 5,760 datasets have been collected.

The results are generated based on the KQI parameters analysis and simulation from the proposed model. The results from real data analysis show the relationship between KQI parameters and their influence on the user perception. Whereas, analysis of results from the proposed data-driven-based FIS model is used to analyze the model accuracy, consistency and linearity.

First, by using the survey data, we calculate average MOS value of LTE video streaming. The average MOS value of subjective measurements obtained with experimental customer survey data is 3.92. Based on the results in (G´omez et al., 2012), a 3.92 MOS value is approximated as a good satisfaction level on MOS scale rating. And this result indicates that the video streaming services in Addis Ababa LTE network has a satisfaction level of good. Hence, Ethio-Telecom should do further analysis to identify the reasons of quality degradation and to develop a model that improve the user satisfaction level to level of excellent in MOS scale rating.

figure 3 (a)-(e) shows the histogram plot for SSR, SD, PDR, SF and STR results, respectively. The results show the mean and standard deviation of video streaming KQI dataset collected for 8 months. As shown in figure 3 (a), the SSR has an average value of above 75%, whereas Figure 3 (b)-(e) shows an average value of 10 ms SD, 8% PDR, 0.3x/min SF and 9% STR, respectively.

Figure 3. Histogram plot for SSR, SD, PDR, SF and STR.

From the histogram results and data analysis, we calculate the mean, median, standard deviation, minimum and maximum value of the KQI parameters, as shown Table 1. Based on these statistical parameters, it may be possible to say the QoS is good or bad. But, only this justification does not guarantee to know that the required QoE has been achieved or not. Also the impact of each KQI parameter on the QoE should be identified and analyzed. Besides, there should be an algorithm or a model that help to measure the QoE of video streaming from these KQI data. Hence, we perform further data analysis and algorithm development to identify the impact of KQI parameters on the video streaming QoE.

Table 1. Video streaming KQI data analysis result

Parameters	SSR (%)	SD(ms)	PDR (%)	SF (times/min)	STR (%)
Mean	75.2234	9712.8115	9.0646	0.4517	12.1665
Median	75.115	9323	8.52	0.35	10.57
Std. Deviation	6.85916	2633.07173	5.38347	0.44486	7.75679
Minimum	45.83	2200	0	0	0
Maximum	100	26,177	45.31	4.17	59.85

Correlation analysis is done to select the important features and to reduce the dimension by removing the redundant KQI parameters. The correlation result is used to identify the impact of each KQI parameter on the user perception. For subjective data, correlation has been done between KQI parameters obtained from participants response among themselves and with individuals opinion score. The results in Table 2 show that SF and STR are correlated by 80.8%. And this shows that they have redundant information. Therefore, one of them should be ignored. We ignore the STR value since it is less correlated with the MOS value. The result also shows that SD and SF have high correlation magnitude value; thus, they have high impact on the customer satisfaction.

Table 2. Correlation result of the subjective KQI parameters

	SSR	SD	PDR	SF	STR	MOS
SSR	1	−0.672	−0.622	−.562	−0.53	0.733
SD	−0.672	1	0.555	0.546	0.59	−0.79
PDR	−0.622	0.555	1	0.571	0.499	−0.683
SF	−0.562	0.546	0.571	1	0.808	−0.77
STR	−0.53	0.59	0.499	0.808	1	−0.75
MOS	0.733	−0.79	−0.683	−0.77	−0.75	1

The results from multiple regressions are shown in Tables 3, 4 and 5. The results in Table 3 validate that the data from the regression is consistent and accurately estimated.

Table 3. Summary of the regression result

Model	R	R^2	Adj. R^2	Std. Err.
1	0.969	0.94	0.94	0.0093841

Table 4. ANOVA result of the objective dataset

Model	Sum of Sq.	df	Mean Sq.	F	Sig.
Regr.	5.372	4	1.343	15,251.685	0
Res.	0.345	3914	0
Total	5.717	3918

Table 5. Coefficients of the regressed objective dataset

Model	Unstandard		Standard	t		Sig
	B	Std. Error	Beta
Const.	0.811	0.002		400.465	0
SSR	0.12	0.001	0.367	88.045	0
SD	−0.193	0.002	−0.453	−109.945	0
PDR	−0.089	0.001	−0.249	−60.499	0
SF	−0.314	0.002	−0.753	−189.957	0

Whereas, the ANOVA result in Table 4 shows zero significant value, zero residual mean square and high F-value. This shows that the regression estimates accurately. Table 5 shows coefficient of the regressed dataset with zero significant value and high t-value. The result also shows that the dependent variable is highly influenced by input independent parameters. Therefore, the output MOS can be determined by input KQI parameters.

From the above results, we can show that the average value of the standard deviation is less than 0.002 and this shows that the distribution of the data has less variation. Hence, the linear model from objective data can be formulated as

(10) $y=0.811+0.12x_1-0.193x_2-0.089x_3-0.314x_4$(10)

where y represents the output MOS value and {x₁, x₂, x₃, x₄} represents input KQI values. Independent weighting method (Adeolu & An Szucs, 2019; Bouraqia etal., 2020) is used to determine the weight of each coefficient of the KQI parameters. This helps to identify the impact of each KQI parameter on the user perception. This is done by applying independent weighting method on the magnitude of beta coefficients in Table 5 and from Equation (10)(10) $y=0.811+0.12x_1-0.193x_2-0.089x_3-0.314x_4$(10) . The result of the analysis is shown in Table 6.

Table 6. Weight coefficients of KQI parameters

Variables	SSR	SD	PDR	SF
Coefficient	0.23	0.25	0.19	0.33
Rank	3	2	4	1

Therefore, the value of the KQI parameters are calculated as SF is 33%, SD is 25%, SSR is 23% and PDR is 19%. These values also validate the accuracy of the results obtained during correlation.

4.2. Analysis of proposed data-driven-based fuzzy inference algorithm for QoE Modeling

We analyze the proposed data-driven-based fuzzy logic inference algorithm for QoE modeling that take the above KQI parameters as input and QoE value as an output. For the simulation, we use the MATLAB fuzzy logic toolbox (Umoh & Asuquo, 2018). figure 4 shows the four input parameters of the model such as SD, SSR, SF and PDR. We deploy Mamdani type inference system with 81 inference rules (Adeolu & An Szucs, 2019; Siris et al., 2014). The output of the model represents the QoE of the video streaming services.

Figure 4. Proposed data-driven-based algorithm for QoE modeling in LTE video streaming service.

The membership functions are obtained from the probability distribution function (pdf) of the input and output relationship. The output pdf can be triangular, trapezoidal, Gaussian or bell curves. In this work, based on the relationship between the KQI parameters and the output QoE values, the input and output parameters are selected to have a triangular pdf. The triangular curve represents the member function of input vector x and depends on three scalar parameters as (Adeolu & An Szucs, 2019; Lin & Yin, 2020; Matulin Štefica Mrvelj & Mrvelj, 2018; Umoh & Asuquo, 2018; Alreshoodi etal., 2015)

(11) $\left\{\begin{array}{c}0x\le a\\ \frac{x-a}{b-a}a\le x\le b\\ \frac{c-x}{c-b}b\le x\le c\\ 0x\ge c\end{array}\right.$(11)

The fuzzy logic inference rule maps the input fuzzy sets into an output fuzzy set. The input fuzzy sets are operated by a logical AND operation. The result of the logical operation determines the output fuzzy set. The weight of each KQI parameter coefficients from Table 6 has been considered to include their influence. Therefore, the output fuzzy set is determined by the sum of the product of each KQI parameter fuzzy score as

(12) $y=0.23{\mathrm{x}}_1+0.25{\mathrm{x}}_2+0.19{\mathrm{x}}_3+\hspace{0.17em}0.33{\mathrm{x}}_4$(12)

where based on Table 6, the value of the weight coefficients are SSR = 0.23, SD = 0.25, PDR = 0.19 and SF = 0.33. This shows that the MOS score for the condition of “IF SD is medium AND SF is medium AND PDR is medium AND SF is medium” is given by

(13) $y=0.23\mathrm{S}\mathrm{S}\mathrm{R}+0.25\mathrm{S}\mathrm{D}+0.19\mathrm{P}\mathrm{D}\mathrm{R}+0.33\mathrm{S}\mathrm{F}=3$(13)

Equation (13)(13) $y=0.23\mathrm{S}\mathrm{S}\mathrm{R}+0.25\mathrm{S}\mathrm{D}+0.19\mathrm{P}\mathrm{D}\mathrm{R}+0.33\mathrm{S}\mathrm{F}=3$(13) shows a quality level of Good in the output fuzzy set. The result of each inference rule is aggregated with the maximum rule and become ready for defuzzification. During model execution, all the rules determine the result with the minimum rule and aggregate the final output with the maximum rule, then a final value is obtained after defuzzification. The results can be displayed by using rule viewer or surface viewer.

The rule viewer in figure 5 shows the result in columns. It has a vertical red line on each input parameter and by moving this line to the left and right, it shows the impact of each parameter on the output QoE value. Except SSR, all inputs increase the QoE when the red line moves to the left. This shows that they are inversely correlated with the QoE. Whereas, the SSR increase as the red line moves to the right which means it is positively correlated with the QoE.

Figure 5. Partial QoE results with rule viewer.

For instance, the QoE is 3.87 when SD = 0.129, SSR = 0.838, SF = 0.189 and PDR = 0.1. The yellow shaded triangular pdf within each input parameter shows the value of the fuzzy score and their operation aggregates the final output. The QoE shall be obtained from the defuzzification of the aggregated fuzzy scores.

The surface viewer displays the results in three-dimensional view. It displays the variation of the two KQI parameters when the other two are being constant. For the simulation, we use KQI parameters of SSR = 0.95, SD = 0.1, SF = 0.1 and PDR = 0.1. As shown in figure 6, the yellow color shows the best QoE value, whereas the blue color shows the worst QoE value.

Figure 6. Surface viewer results of the QoE parameters.

figure 6a shows the impacts of SSR and SD when PDR and SF are being constant. The graph shows that the QoE increases with SSR and decreases with SD. Thus, the maximum QoE value (MOS = 4) is scored at SSR = 1 and SD = 0. The result also show that when the SSR is less than 0.6 and SD is above 0.5, the QoE is below 2.5, and this indicates poor in MOS scale value and unacceptable for LTE video streaming service. figure 6b shows the impacts of SD and SF when SSR and PDR being constant. The result shows that both parameters have inversely related with the QoE. Thus, the maximum QoE values is scored at SD = 0 and SF = 0. Similarly, figure 6c shows the impacts of SF and PDR on QoE when SD and SSR are being constant. The result shows that the best user perception is scored (MOS = 3.5) when SF and PDR is below 0.1. And, this shows a good service value on MOS scale. Finally, figure 6d shows the impacts of SSR and PDR when SD and SF being constant. The results shows that it gives the highest MOS value that is above 4.5. This is achieved at SSR = 1 and PDR = 0. Again, the QoE is below 1 on the MOS scale, when the SSR is below 0.5 and the PDR is above 0.5. This shows that the user perception is more sensitive with SSR and PDR. If the values of these parameters are highly degraded, the user perception would be considered as no service delivered.

4.3. Model validation with experimental customer survey data

The performance and accuracy of the proposed data-driven-based fuzzy inference algorithm for QoE modeling is validated by using experimental customer survey data. The accuracy of the model has been evaluated, and the root mean square (RMSE) results are shown in Tables 7, VIII and IX. The result in Table 7 confirms that the proposed model has a good accuracy. As explained in (The Mathwork Inc, 2019), an R value above 0.68 is accurate for linear regression. The RMSE value of 0.03175 also shows a good accuracy relative to existing models.

Table 7. Regression results of subjective and simulated QoE values

Model	R	R^2	Adj. R^2	RMSE
1	0.932	0.869	0.869	0.03175

Table 8 shows the results from ANOVA analysis of the subjective and simulated QoE value. The result shows a residual mean square error of 0. 001, which is acceptable and this confirms that the proposed model is highly consistent. Table 9 shows the linear regression coefficients of the subjective QoE and simulated QoE values.

Table 8. ANOVA results for subjective and simulated QoE values

Model	Sum of Sq.	df		Mean Sq.	F	Sig.
Regression	1.436	1	1.436		1424	0
Residual	0.216	214	0.001
Total	1.651	215

Table 9. Regression coefficient of subjective and simulated MOS values

	Unstand.		Stand.	t	Sig.
Model	B	Std. Error	Beta
Subjective MOS	0.192	0.016		11.832	0
Simulated MOS	0.772	0.02	0.932	37.742	0

The above results show that the relation can be approximated with linear curve fitting. The model for linear curve fitting is shown in figure 7. The linear curve fitting shows that the simulated and subjective QoE values have linear relationship with very small variation.

Figure 7. Curve fitting result of subjective and simulated MOS.

Finally, Table 10 shows the result of Pearson correlation of the subjective MOS and simulated MOS values. The result shows that the subjective and simulated MOS values have above 93.2% correlated. Hence, their relationship is linear with this percentage amount and which is also a good value (The Mathwork Inc, 2019).

Table 10. Correlation result between subjective and objective MOS values

	Subjective MOS	Simulated MOS
Subjective MOS	1	0.932
Simulated MOS	0.932	1

In general, the validation of the model results the best performance for all the three metrics. The proposed algorithm is highly accurate since RMSE = 0.03175, and it is highly consistent as the residual mean square is 0.001. The linearity of the model achieves 93.2% correlation and which is also best in linearity compared to existing models.

5. Conclusion

In this work, we have proposed data-driven-based quality of experience model using fuzzy logic inference algorithm for video streaming services in Addis Ababa city LTE network. The results show that the model can be used to replace conventional subjective measurement techniques that are costly and inefficient. The correlation, regression and ANOVA analysis results show that the KQI parameters have a great impact on QoE of LTE network video streaming. The coefficient influence analysis has shown that the SF value has high impact on video streaming, which is 33% and SD also plays critical role by 25%. This implies that the network performance on the capacity and delay parameters should be further investigated to improve the QoE. The proposed fuzzy inference algorithm for QoE modeling to LTE video streaming has good accuracy, consistency and linearity. The RMSE value of 0.03175 indicates better accuracy in prediction result. A residual mean square error of 0.001 also shows the consistency of the model. 93.2% Pearson correlation result shows the linearity of the model. Therefore, the model has better performance than current existing models. Hence, we highly recommend Ethio-Telecom to deploy the proposed fuzzy logic inference algorithm for QoE modeling to LTE video streaming. Analyzing the performances of different distributions of the membership functions and considering more datasets and subjects shall be the possible future works.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Amare Kassaw Yimer

Amare Yimer received the BSc degree in Electrical Engineering from Bahir Dar University in 2006 and the MSc degree in Electrical Engineering (Communication Systems Engineering) from Addis Ababa University in January 2011. He has got the PhD degree in Electrical Engineering in March 2021 from Addis Ababa University under the supervision of Prof. Dr.-Ing. Abelhak Zoubir in Signal Processing Group at Technical University of Darmstadt and Dr.-Ing. Dereje Hailemariam at Addis Ababa University. Currently, he is assistant professor at Bahir Dar University where he delivered lectures in Digital Communication, Wireless and Mobile Communication, Statistical Signal Processing and Machine Learning; and Fundamentals of Detection and Estimation Theory. Besides, he has participated on several researches and projects in the area of transportation systems, signal processing, ICT and wireless communication. His current research interests include massive MIMO systems, mmWave communication, resource allocation and optimization in wireless networks, quality of service and quality of experience modeling in mobile communication, machine learning and deep learning algorithms.