A content-based recommendation approach based on singular value decomposition

Abstract

In the Internet era, where information and communication technologies (ICT) allow data exchange, new tools able to select the correct data are needed. In this field, Recommender Systems have a prime location. The recommendation methods take many forms based on the information exploited in order to provide rating forecasts. However, all of them can choose the correct information to support users. Indeed, these methods allow for overcoming the information overload problem. This paper introduces the theoretical bases of a novel recommendation method defined Rating Singular Value Decomposition (RSVD). RSVD is a Content-Based method that exploits the Singular Value Decomposition properties in order to calculate rating forecasts. This method aims to elaborate the users and items profile to obtain matrices related to ones obtained in Collaborative Filtering methods that exploit Singular Value Decomposition. The accuracy of RSVD is compared with the accuracy of Collaborative Filtering methods, and a study on the sparsity problem is performed. The results obtained are promising.

Keywords:

• Recommender system
• content-based approach
• singular value decomposition
• rating forecast

1. Introduction

In the Internet era, the collection of available data arises the problem of Big Data (Castiglione et al., 2018), which is characterised by volume, velocity, and variety (Allam & Dhunny, 2019). Indeed, during the nineties, the first e-commerce sites had to face the Big Data problem through a tool that could filter and analyse data in order to provide support to users of the system. In this context, Recommender Systems (Quijano-Sánchez et al., 2020; Y. Zhang & Chen, 2020) were developed and, over the years, have owned great relevance. Over the years, Recommender Systems (RSs) have exploited much different information in order to improve the ability to provide suggestions to users. Based on the information exploited and the development, three main categories of RSs can be identified (Sinha & Dhanalakshmi, 2019a, 2019b). The First Generation RSs consist of the Content-Based, Collaborative Filtering, and Hybrid methods (Bobadilla et al., 2013; Ricci et al., 2015). The Second Generation RSs exploit the knowledge related to user preference (Eirinaki et al., 2018) and the context that allows defining Context-Aware Recommender Systems (CARSs) (Casillo, Colace, et al., 2021; Renjith et al., 2020). CARSs (Liu et al., 2021) were largely exploited in the tourism field because of their ability to exploit contextual information (Adomavicius & Tuzhilin, 2015) in order to improve forecasts related to users' preferences in a specific context (Casillo, Gupta, et al., 2022). The Third Generation RSs exploit Ontologies (Renjith et al., 2020; Sinha & Dhanalakshmi, 2019a, 2019b).

The central aim of RSs consists of providing proper support to users (Portugal et al., 2018). The support can be obtained by calculating forecasts that all recommender systems can get based on the exploited information. The problem associated with forecasts is the lack of sufficient information to support all users on all items. This paper will describe a recommendation approach, defined as Rating Singular Value Decomposition (RSVD), that exploits the properties of Singular Value Decomposition in Content-Based RS instead of Collaborative Filtering. In fact, SVD usually is exploited to reduce the problem dimensionality and can be exploited when known ratings are available. Instead, The proposed approach allows transforming users' and items' profiles according to the factorisation to calculate the ratings that are all unknown. The numerical results measure the accuracy of RSVD on the MovieLens100k dataset and testify to the ability of RSVD to provide forecasts when information is lacking.

This paper is organised as follows. Section 2 focuses on the description of Background and Related Works; Section 3 describes the proposed approach and its theoretical basis; Section 4 presents numerical experiments developed on the datasets MovieLens100k and MovieLens10M, and, finally, Section 5 contains conclusions and future works.

2. Background and related works

To filter information, Recommender Systems exploit three principal tools: the user of the system, the item that the system suggests, and the transaction that consists of an interaction between users and the system (Ricci et al., 2015). The most common form of transaction is the rating: an implicit or explicit evaluation of an item by a user. This evaluation can be expressed in various forms: Numerical Rating (the most used, the user chooses from a range of values based on his appreciation), Ordinary Rating (Strongly agree, Agree, Neutral, Disagree, Strongly disagree), Binary Rating (like, dislike) and Unary Rating (the information associated with the interaction between the user and an item is stored) (Ricci et al., 2015). The rating can be defined as a function, defined rating function or utility function and summarised in Equation (1(1) $r:(u,i)\in U\times I\mapsto r_{ui}\in \mathbb{R}$(1) ), that has as domain the Cartesian Product of the users set U and the items set I and as codomain, the real numbers set $\mathbb{R}$. (1) $r:(u,i)\in U\times I\mapsto r_{ui}\in \mathbb{R}$(1) Because of the lack of knowledge of all values $r_{ui}$ $\mathrm{\forall }(u,i)\in U\times I$, Recommender Systems have to provide a rating forecasts ${\hat{r}}_{ui}$ that estimates the unknown ratings and the efficacy of RSs is based on the ability to provide reasonable approximations. Based on how these forecasts are made and how information relating to users and items is stored and processed, it is possible to classify RSs.

The main typologies of Recommender Systems are Content-Based, Collaborative Filtering, and Hybrid.

Content-Based RSs generate rating forecasts through the feature vectors of the items and the user profile. The structure of an RS based on high-level content is composed of three phases. In the Content Analyzer phase, the features of the items are analysed to build a vector that describes them (Lops et al., 2011) and afterward, the Profile Learner phase involves the study of user preferences summarised through a numerical vector which constitutes the relative profile (Lops et al., 2011). At last, in the Filtering Component phase, the user-item affinity is generated through similarity measures between the feature vector of the item and the user profile vector.

D. Wang et al. (2018) develop a Content-Based RS that captures information about the abstract of a scientific article to suggest the most appropriate journals or conferences. After choosing the mode of feature acquisition, the Content-Based approach used general rating predictions through softmax regression, which generalises the logistic regression. Casillo, Conte, et al. (2021) propose a Content-Based RS that exploits users and items features in order to provide rating forecasts and uses contextual information to select the adequately calculated forecasts. Casillo, Conte, et al. (2022) exploit social information in order to create users' profiles and the experts in the cultural heritage field to create items profiles. These profiles allow calculating the user-item affinity. Pérez-Almaguer et al. (2021) provide a Content-Based group recommendation method. In particular, they calculate the items and users' profiles through two different approaches. The affinity between the entities is calculated through cosine similarity if item profiles consist of binary elements. If the item profiles have multivalued values they exploit an analysis on each feature. Then, the group recommendation takes advantage of the introduction of a taxonomy. Finally, some learning methods are exploited into the Content-Based field for various applications in healthcare and finance (Javed et al., 2021).

Collaborative Filtering RSs (Koren et al., 2009; Ricci et al., 2015) is based on the ratings that users give to the system about items they interact with and calculate predictions through the opinion of a community (Geuens et al., 2018), which allow recommending items which similar users have evaluate positively (Aberger, 2014; Anastasiu et al., 2016). Collaborative Filtering methods can be divided into two groups: Memory-Based and Model-Based (Casillo, Colace, et al., 2021). Memory-Based RSs exploit three main strategies. The first one consists in creating Neighbourhoods among users (User-Based) (F. Zhang et al., 2016). The similarity can be calculated through the Cosine Vector or the Pearson Correlation (Desrosiers & Karypis, 2011; Jain et al., 2020). The second strategy, instead, creates groups of items (Item-based) through the Pearson Correlation for the items or with the Adjusted Cosine formula (Desrosiers & Karypis, 2011; Jain et al., 2020). Finally, it is possible to combine the two techniques to obtain the Hybrid User-Item Based CF. Model-Based RSs aim to create a numerical model by analysing the rating matrix (Casillo, Colace, et al., 2021; Ricci et al., 2015). To analyse and elaborate the rating matrix, matrix factorisation methods are exploited. Among all of them, the most common is Principal Component Analysis (PCA) (Bokde et al., 2015; Vozalis & Margaritis, 2007), which aims to reduce the dimension of the problem through the analysis of the correlation matrix of data analysed (Kumaravel & Dutta, 2014), Non-Negative Matrix Factorization (NMF) (Bokde et al., 2015; Luo et al., 2014) that aims to factorise the rating matrix in the product of two matrices composed by non-negative elements (Hernando et al., 2016), Probabilistic Matrix Factorization (PMF) (Bokde et al., 2015; Mnih & Salakhutdinov, 2008), which exploits probabilistic theory to reduce the root mean squared error on the training data, and Singular Value Decomposition (SVD) (Golub & Van Loan, 2013; Gu et al., 2020; Symeonidis & Zioupos, 2016).

He et al. (2018) propose a Memory-Based CF of the Item-Based CF type built through neural networks. In particular, relevance is given to the user's history in the theoretical construction. Mnih and Salakhutdinov (2008) exploit probabilistic matrix factorisation, hence a Model-Based RS. The application of the PMF to the Gradient Descent method provides a stable Recommender System with few ratings from the users. Recommendation methods that fall in Model-Based approaches are Average Rating Filling (ARF) (Anastasiu et al., 2016; van der Vlugt, 2018), Stochastic Gradient Descent (SGD) (Anastasiu et al., 2016; R. Wang et al., 2019; Zhao et al., 2019) and Biased Stochastic Gradient Descent (BSGD) (R. Wang et al., 2019; Zhao et al., 2019). Gu et al. (2020) propose a weighted model based on SVD and gradient descent. In their paper, non-negative weights aim to solve the noise effect and reduce the unreliable rating forecasts. Luo et al. (2019) deeply studied eight models that exploit stochastic gradient descent and the effects of the different modification strategies on SGD. Yu et al. (2021) exploit SVD,SGD and Stacked AutoEncoder (G. Zhang et al., 2020) in order to overcome sparsity problem. Bhavana et al. (2019) propose an approach in which Singular Value Decomposition is developed through Graphics Processing Unit (GPU) and CUDA (Compute Unified Device Architecture) (Nickolls et al., 2008). This approach divides the rating matrix into blocks, and the factorisation development is processed in parallel. Nguyen and Zhu (2013) exploit matrix factorisation and item profiles. In particular, they introduce in the factorisation model items profiles that are composed of binary elements. Moreover, the analysis of common attributes for items and the introduction of the tag analysis concept translated to items improve the recommendation method's performance. Rendle (2010) exploits Support Vector Machine to provide rating forecasts and overcome the specialisation of the input data. The principal aim of the paper consists of improving the recommendation method to manage the sparsity problem. In addition, the provided model is linear.

Other typologies of RSs are Demographic Filtering, Knowledge-Based, and Community-Based. Demographic Filtering RS recommends an item to the user by using the information regarding the region to which he belongs, the language, the gender, and the age (Ricci et al., 2015). Knowledge-Base RS suggests an item to the user by considering how much the features of the suggested item meet the user's specific needs. To determine these needs, it is essential to build an appropriate user profile (Ricci et al., 2015). Community-Based RS recommends an item to the user by using the information acquired from social networks. The means of acquisition are the user tags or the preferences of the friends of the latter (Ricci et al., 2015). The social information can be exploited to create profiles (Gorripati & Vatsavayi, 2017) or in a Collaborative Filtering RS (Liang et al., 2019). Hence, the Community-Based RS main feature is the exploitation of social media information.

Table 1 summarises the advantages and disadvantages of RSs typologies described in this Section.

Table 1. Summary table of advantages and disadvantages of recommendation methods.

Method	Technique	Advantages	Disadvantages
Content Based		– Not a large number of users is need to forecasting	– Limited content analysis (Tarus et al., 2018)
		– Requires knowledge of item features only	– Over-specialisation
			– Cold start (new user) (F. Zhang et al., 2016)
Collaborative Filtering	Memory Based	– Ease of implementation and data addiction	– Dependence on known ratings
		– It adapts well to related items	– Cold Start (new user/new item/new community) (Karabadji et al., 2018)
		– Serendipity of results (D. Wang et al., 2018)	– Limited scalability for large dataset
			– Work poorly with sparse matrices
	Model Based	– Work well with sparse matrices	– High cost for the model construction
		– Adequately addresses the scalability problem	– Loss of information
Hybrid		– Exploits the advantages of different techniques	– High cost for the model construction
		– Interacts well with the issues of the exploited techniques	– Increase of complexity
Demographic Filtering		– No historical data of user transaction (Renjith et al., 2020)	– Difficulty in collecting all the demographic data (Sinha & Dhanalakshmi, 2019a)
Knowledge Based		– Deal well with cold start problem (Renjith et al., 2020)	– Highly computational intensive approach (Renjith et al., 2020)
Community Based		– Exploits social media information (Sinha & Dhanalakshmi, 2019b)	–High cost for the model construction

3. RSVD: a content-Based approach based on SVD

This section focuses on a novel Content-Based approach that exploits the properties of Singular Value Decomposition. SVD is usually exploited in Collaborative Filtering methods, reducing the requested memory and obtaining rating forecasts. Instead, the proposed approach, defined RSVD, aims to exploit SVD in a Content-Based recommendation method. To underline this difference, this section is divided into two subsections: the first describes how SVD is usually exploited, and the second presents the theoretical basis of RSVD.

3.1. SVD: basis and usage on collaborative filtering RSs

The Singular Value Decomposition allows to factorise a real rectangular matrix $R\in {\mathbb{R}}^{m\times n}$, where $m,n,\in \mathbb{N}$, into the product of three matrices according to Equation (2(2) $R=UD{V}^T$(2) ). (2) $R=UD{V}^T$(2) The matrices $U\in {\mathbb{R}}^{m\times m}$ and $V\in {\mathbb{R}}^{n\times n}$ are orthogonal and are defined matrices of left and right singular vectors respectively. The matrix $D=\text{diag}({\sigma }_1,\dots ,{\sigma }_p)\in {\mathbb{R}}^{m\times n}$ $p=min\{m,n\}$ is defined matrix of singular values and the elements ${\sigma }_1,\dots ,{\sigma }_p$ satisfy Equation (3(3) ${\sigma }_1\ge {\sigma }_2\ge \cdots \ge {\sigma }_k>{\sigma }_{k+1}=\cdots ={\sigma }_p=0$(3) ). (3) ${\sigma }_1\ge {\sigma }_2\ge \cdots \ge {\sigma }_k>{\sigma }_{k+1}=\cdots ={\sigma }_p=0$(3) In particular, matrix U columns represent the eigenvectors of the square matrix $R{R}^T\in {\mathbb{R}}^{m\times m}$ and, similarly, the columns of V represent the eigenvectors of the matrix $R^{T}R\in {\mathbb{R}}^{n\times n}$. The nonzero elements of the diagonal matrix D represent, then, the square roots of the eigenvalues of $R{R}^T$ and $R^{T}R$.

The great diffusion of Singular Value Decomposition in the application field is related to the ability to approximate the matrix R so that the error related to approximation is known. This fundamental property is established by the Eckart–Young Theorem 3.1 (Golub & Van Loan, 2013).

Theorem 3.1

of Eckart–Young

Let the matrix $R\in {\mathbb{R}}^{m\times n}$ and the matrices $U,V,D$ be expressed by relation (2(2) $R=UD{V}^T$(2) ) and let $r=r\left(R\right)$ the rank of matrix R, with $r\le p=min\{m,n\}$. The matrices $U_k,V_k$ and $D_k$ are defined from $U,V,D$ as follows:

• $U_k\in {\mathbb{R}}^{m\times k}$ contains the firsts k columns of the matrix U;

• $V_k\in {\mathbb{R}}^{n\times k}$ contains the firsts k columns of the matrix V;

• the matrix $D_k=\text{diag}({\sigma }_1,\dots ,{\sigma }_k)\in {\mathbb{R}}^{k\times k}$ is a square diagonal matrix that contains the first k raws and columns of the matrix D.

(4) $R_k=U_k{D}_k{V}_k^T$(4) Then the matrix $R_k$ defined in (4(4) $R_k=U_k{D}_k{V}_k^T$(4) ) is the matrix of rank k that best approximates the matrix R among all matrices of rank k: (5) $\begin{array}{rl}\underset{r\left(B\right)=k}{min}{\Vert R-B\Vert }_F^2& ={\Vert R-R_k\Vert }_F^2=\sum _{i=k+1}^p{\sigma }_i^2,\end{array}$(5) (6) $\begin{array}{rl}\underset{r\left(B\right)=k}{min}{\Vert R-B\Vert }_2& ={\Vert R-R_k\Vert }_2={\sigma }_{k+1},\end{array}$(6) where ${\Vert \cdot \Vert }_F$ and ${\Vert \cdot \Vert }_2$ are the Frobenius and the spectral norm, respectively.

The approximation error is estimated by Equations (5(5) $\begin{array}{rl}\underset{r\left(B\right)=k}{min}{\Vert R-B\Vert }_F^2& ={\Vert R-R_k\Vert }_F^2=\sum _{i=k+1}^p{\sigma }_i^2,\end{array}$(5) ) and (6(6) $\begin{array}{rl}\underset{r\left(B\right)=k}{min}{\Vert R-B\Vert }_2& ={\Vert R-R_k\Vert }_2={\sigma }_{k+1},\end{array}$(6) ). Theorem 3.1 allows for reducing the computational cost relating to SVD with an estimate of the acceptable error based on Equation (3(3) ${\sigma }_1\ge {\sigma }_2\ge \cdots \ge {\sigma }_k>{\sigma }_{k+1}=\cdots ={\sigma }_p=0$(3) ). The low-rank approximation of matrices is used in many applications such as control theory, signal processing, machine learning, image compression, information retrieval, quantum physics (Conte, 2020; Conte & Lubich, 2010; Koch & Lubich, 2007; Nonnenmacher & Lubich, 2008).

The Singular Value Decomposition is usually exploited in Collaborative Filtering methods, where a rating matrix is provided. This matrix $R\in {\mathbb{R}}^{m\times n}$ is related to a system composed by m users and n items. The SVD reduces the needed memory to allocate the matrix and provides rating forecasts through some State of the Art techniques such as Stochastic Gradient Descent or Average Rating Filling. The rating matrix can be factorised through Equation (2(2) $R=UD{V}^T$(2) ). Then, the Eckart–Young Theorem can be exploited to reduce the dimension of the matrices U, V and D. To obtain the reduction, the number of latent factors $k\in \mathbb{N}$ is selected, and the matrices $U_k,V_k,D_k$ can be obtained. Then, the matrices P and Q can be calculated according to Equations (7(7) $\begin{array}{rl}P& =U_k{\sqrt{D}}_k\in {\mathbb{R}}^{m\times k}\end{array}$(7) ) and (8(8) $\begin{array}{rl}Q& ={\sqrt{D}}_k{V}^T\in {\mathbb{R}}^{k\times n}\end{array}$(8) ), respectively. (7) $\begin{array}{rl}P& =U_k{\sqrt{D}}_k\in {\mathbb{R}}^{m\times k}\end{array}$(7) (8) $\begin{array}{rl}Q& ={\sqrt{D}}_k{V}^T\in {\mathbb{R}}^{k\times n}\end{array}$(8) The matrices P and Q contain the fake numerical features of the users and the items, respectively. For instance, in SGD and BSGD algorithms, they are defined randomly, and the elements of P and Q are corrected through the provided available ratings (van der Vlugt, 2018).

3.2. RSVD: the proposed approach

The paper aims to exploit the SVD properties in a Content-Based RS. In Collaborative Filtering RSs the known rating matrix R is factorised in the product of the matrices P and Q of relations (7(7) $\begin{array}{rl}P& =U_k{\sqrt{D}}_k\in {\mathbb{R}}^{m\times k}\end{array}$(7) ) and (8(8) $\begin{array}{rl}Q& ={\sqrt{D}}_k{V}^T\in {\mathbb{R}}^{k\times n}\end{array}$(8) ). The proposed approach defined RSVD aims to elaborate the matrices $\hat{P}$ and $\hat{Q}$, which contain the real profiles of users and items to obtain two matrices $\tilde{P}$ and $\tilde{Q}$. The latter matrices have the same properties as the matrices P and Q generated in the Collaborative Filtering case.

To understand the properties that must be obtained by elaborating users' and items' real profiles, some theoretical analysis on Singular Value Decomposition will be discussed.

Lemma 3.2

Let the matrix $R\in {\mathbb{R}}^{m\times n}$, $k\le r\left(R\right)$, $I_k$ the identity matrix of dimension k and the matrices $U_k,V_k,D_k$ related to Theorem 3.1, then the following properties are satisfied:

1. $U_k^T{U}_k=I_k$;

2. $V_k^T{V}_k=I_k$;

3. $R_k{R}_k^T=U_k{D}_k^2{U}_k^T\in {\mathbb{R}}^{m\times m}$;

4. $R_k^T{R}_k=V_k{D}_k^2{V}_k^T\in {\mathbb{R}}^{n\times n}$.

Proof.

1. From Theorem 3.1 the matrix $U_k$ is defined as the matrix containing the first k left singular vectors. (9) $U_k=(u_1,\dots ,u_k)\text{with }{u}_i\in {\mathbb{R}}^m,i=1,\dots ,k.$(9) The vectors $u_i\mathrm{\forall }i=1,\dots ,k$ are the columns of the orthogonal matrix U, therefore, by denoting with <, > the standard scalar product in ${\mathbb{R}}^m$, Equation (10(10) $〈u_i,u_j〉={\delta }_{i,j},i,j=1,\dots ,k,$(10) ) is verified, where δ is the Kronecker delta. (10) $〈u_i,u_j〉={\delta }_{i,j},i,j=1,\dots ,k,$(10) The matrix $B=U_k^T{U}_k$ has elements $B_{ij}=<u_i,u_j>$ $i,j=1,\dots ,k$, so, from (10(10) $〈u_i,u_j〉={\delta }_{i,j},i,j=1,\dots ,k,$(10) ), Equation (11(11) $U_k^T{U}_k=B=I_k$(11) ) is verified. (11) $U_k^T{U}_k=B=I_k$(11)

2. The proof is analogous to point 1.

3. From (4(4) $R_k=U_k{D}_k{V}_k^T$(4) ) Equation (12(12) $R_k{R}_k^T=\left(U_k{D}_k{V}_k^T\right){\left(U_k{D}_k{V}_k^T\right)}^T=\left(U_k{D}_k{V}_k^T\right)\left(V_k{D}_k{U}_k^T\right)=U_k{D}_k\left(V_k^T{V}_k\right)D_k{U}_k^T$(12) ) is satisfied. (12) $R_k{R}_k^T=\left(U_k{D}_k{V}_k^T\right){\left(U_k{D}_k{V}_k^T\right)}^T=\left(U_k{D}_k{V}_k^T\right)\left(V_k{D}_k{U}_k^T\right)=U_k{D}_k\left(V_k^T{V}_k\right)D_k{U}_k^T$(12) From property 2 of the Lemma the thesis follows.

4. The proof is analogous to point 3.

Through the properties analysed in the Lemma 3.2 some properties related to matrices P and Q of Equations (7(7) $\begin{array}{rl}P& =U_k{\sqrt{D}}_k\in {\mathbb{R}}^{m\times k}\end{array}$(7) ) and (8(8) $\begin{array}{rl}Q& ={\sqrt{D}}_k{V}^T\in {\mathbb{R}}^{k\times n}\end{array}$(8) ) are developed in the Lemma 3.3.

Lemma 3.3

Let the matrix $R\in {\mathbb{R}}^{m\times n}$ and the matrix $R_k=U_k{D}_k{V}_k^T$ obtained according to Theorem 3.1. Let P and Q the matrices defined in Equations (7(7) $\begin{array}{rl}P& =U_k{\sqrt{D}}_k\in {\mathbb{R}}^{m\times k}\end{array}$(7) ) and (8(8) $\begin{array}{rl}Q& ={\sqrt{D}}_k{V}^T\in {\mathbb{R}}^{k\times n}\end{array}$(8) ). Then the following properties apply:

1. $P{P}^T=U_k{D}_k{U}_k^T\in {\mathbb{R}}^{m\times m}$;

2. $P^{T}P=D_k\in {\mathbb{R}}^{k\times k}$;

3. $Q^{T}Q=V_k{D}_k{V}_k^T\in {\mathbb{R}}^{n\times n}$;

4. $Q{Q}^T=D_k\in {\mathbb{R}}^{k\times k}$

Proof.

1. From Equation (7(7) $\begin{array}{rl}P& =U_k{\sqrt{D}}_k\in {\mathbb{R}}^{m\times k}\end{array}$(7) ), the matrix $P=U_k\sqrt{D_k}$ satisfies Equation (13(13) $P{P}^T=\left(U_k\sqrt{D_k}\right){\left(U_k\sqrt{D_k}\right)}^T=\left(U_k\sqrt{D_k}\right)\left(\sqrt{D_k^T}{U}_k^T\right).$(13) ). (13) $P{P}^T=\left(U_k\sqrt{D_k}\right){\left(U_k\sqrt{D_k}\right)}^T=\left(U_k\sqrt{D_k}\right)\left(\sqrt{D_k^T}{U}_k^T\right).$(13) The matrix $D_k$ is square and diagonal, then $D_k=D_k^T$ and Equation (13(13) $P{P}^T=\left(U_k\sqrt{D_k}\right){\left(U_k\sqrt{D_k}\right)}^T=\left(U_k\sqrt{D_k}\right)\left(\sqrt{D_k^T}{U}_k^T\right).$(13) ) can be developed according to Equation (14(14) $P{P}^T=\left(U_k\sqrt{D_k}\right)\left(\sqrt{D_k}{U}_k^T\right)=U_k{D}_k{U}_k^T$(14) ). (14) $P{P}^T=\left(U_k\sqrt{D_k}\right)\left(\sqrt{D_k}{U}_k^T\right)=U_k{D}_k{U}_k^T$(14)

2. From $P=U_k\sqrt{D_k}$ we also obtain (15) $P^{T}P={\left(U_k\sqrt{D_k}\right)}^T\left(U_k\sqrt{D_k}\right)=\sqrt{D_k^T}\left(U_k^T{U}_k\right)\sqrt{D_k}.$(15) Property 1 of Lemma 3.2 allows to derive Equation (16(16) $P^{T}P=\sqrt{D_k^T}{I}_k\sqrt{D_k}=\sqrt{D_k^T}\sqrt{D_k}.$(16) ). (16) $P^{T}P=\sqrt{D_k^T}{I}_k\sqrt{D_k}=\sqrt{D_k^T}\sqrt{D_k}.$(16) The matrix $D_k$ is square and diagonal, then $D_k^T=D_k$ and the thesis immediately follows;

3. Proof similar to point 1;

4. Proof similar to point 2.

The Lemma 3.3 allows deducing that the columns of the matrix P constitute a system of orthogonal vectors as the rows of the Q matrix. These orthogonality properties are going to be emulated by the users and items profiles matrices $\hat{P}$ and $\hat{Q}$ through an elaboration that is going to be described in the Theorem 3.4.

Theorem 3.4

Let the input matrices $\hat{P}\in {\mathbb{R}}^{m\times k}$ and $\hat{Q}\in {\mathbb{R}}^{k\times n}$ have rank $r\left(\hat{P}\right)=r\left(\hat{Q}\right)=k$. Denote with ${\mu }_i$ the eigenvalues of matrix ${\hat{P}}^T\hat{P}$ and with ${\lambda }_i$ the eigenvalues of the matrix $\hat{Q}{\hat{Q}}^T$, $i=1,\dots ,k$. Then

1. There exist orthogonal matrices $X,Y\in {\mathbb{R}}^{k\times k}$ such that (17) $\begin{array}{rl}{Y}^T\left({\hat{P}}^T\hat{P}\right)Y& =D_{\hat{P}}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\mu }_1,\dots ,{\mu }_k)\end{array}$(17) (18) $\begin{array}{rl}{X}^T\left(\hat{Q}{\hat{Q}}^T\right)X& =D_{\hat{Q}}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\lambda }_1,\dots ,{\lambda }_k)\end{array}$(18) with $D_{\hat{P}}$, $D_{\hat{Q}}$ invertible matrices and ${\mu }_1\ge \cdots \ge {\mu }_k$ and ${\lambda }_1\ge \cdots \ge {\lambda }_k$.

   By further setting (19) $\begin{array}{rl}{\tilde{U}}_k& =\hat{P}Y\sqrt{D_{\hat{P}}^{-1}},{\tilde{V}}_k={\hat{Q}}^{T}X\sqrt{D_{\hat{Q}}^{-1}}{\tilde{D}}_k=\sqrt{D_{\hat{P}}}\sqrt{D_{\hat{Q}}}\end{array}$(19) (20) $\begin{array}{rl}\tilde{P}& ={\tilde{U}}_k\sqrt{{\tilde{D}}_k},\tilde{Q}=\sqrt{{\tilde{D}}_k}{\tilde{V}}_k^T,\end{array}$(20) (21) $\begin{array}{rl}\overline{R}& =\tilde{P}\tilde{Q},\end{array}$(21)

2. the matrices defined ${\tilde{U}}_k$ and ${\tilde{V}}_k$ in (19(19) $\begin{array}{rl}{\tilde{U}}_k& =\hat{P}Y\sqrt{D_{\hat{P}}^{-1}},{\tilde{V}}_k={\hat{Q}}^{T}X\sqrt{D_{\hat{Q}}^{-1}}{\tilde{D}}_k=\sqrt{D_{\hat{P}}}\sqrt{D_{\hat{Q}}}\end{array}$(19) ) satisfy properties 1 and 2 of Lemma 3.2, i.e. they have orthogonal columns;

3. the matrices in (19(19) $\begin{array}{rl}{\tilde{U}}_k& =\hat{P}Y\sqrt{D_{\hat{P}}^{-1}},{\tilde{V}}_k={\hat{Q}}^{T}X\sqrt{D_{\hat{Q}}^{-1}}{\tilde{D}}_k=\sqrt{D_{\hat{P}}}\sqrt{D_{\hat{Q}}}\end{array}$(19) ) and (20(20) $\begin{array}{rl}\tilde{P}& ={\tilde{U}}_k\sqrt{{\tilde{D}}_k},\tilde{Q}=\sqrt{{\tilde{D}}_k}{\tilde{V}}_k^T,\end{array}$(20) ) satisfy properties 1–4 of Lemma 3.3;

4. the matrix $\overline{R}$ in (21(21) $\begin{array}{rl}\overline{R}& =\tilde{P}\tilde{Q},\end{array}$(21) ) coincides with its reduced factorisation matrix related to Theorem 3.1, defined in (4(4) $R_k=U_k{D}_k{V}_k^T$(4) ), i.e. ${\overline{R}}_k=\overline{R}$.

5. the matrix $\overline{R}$ in (21(21) $\begin{array}{rl}\overline{R}& =\tilde{P}\tilde{Q},\end{array}$(21) ) has singular values ${\sigma }_1,\dots ,{\sigma }_k$ satisfying; (22) ${\sigma }_i=\sqrt{{\mu }_i}\sqrt{{\lambda }_i},i=1,\dots ,k,$(22)

Proof.

In order to prove (i), we observe that the matrix $\hat{Q}{\hat{Q}}^T\in {\mathbb{R}}^{k\times k}$ is a real symmetric matrix, then for the Spectral Theorema it is diagonalisable. In particular it is similar to the diagonal matrix $D_{\hat{Q}}\in {\mathbb{R}}^{k\times k}$ containing its eigenvalues on the diagonal, through the orthogonal matrix $X\in {\mathbb{R}}^{k\times k}$ containing the corresponding eigenvertors. Analogously the matrix ${\hat{P}}^T\hat{P}\in {\mathbb{R}}^{k\times k}$ is similar to the diagonal matrix $D_{\hat{P}}\in {\mathbb{R}}^{k\times k}$ containing its eigenvalues on the diagonal, through the orthogonal matrix $Y\in {\mathbb{R}}^{k\times k}$ containing the corresponding eigenvectors. Furthermore, from the hypothesis that matrices ${\hat{P}}^T\hat{P}$ e $\hat{Q}{\hat{Q}}^T$ have maximal rank k, we deduce that the respective eigenvalues are all non-zero (23) $\begin{array}{rl}{\lambda }_j& \ne 0\mathrm{\forall }j=1,\dots ,k\\ {\mu }_i& \ne 0\mathrm{\forall }i=1,\dots ,k\end{array}$(23) In particular, the matrices $D_{\hat{Q}}$ e $D_{\hat{P}}$ are invertible.

In order to prove (ii), by exploiting (17(17) $\begin{array}{rl}{Y}^T\left({\hat{P}}^T\hat{P}\right)Y& =D_{\hat{P}}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\mu }_1,\dots ,{\mu }_k)\end{array}$(17) ) and (18(18) $\begin{array}{rl}{X}^T\left(\hat{Q}{\hat{Q}}^T\right)X& =D_{\hat{Q}}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\lambda }_1,\dots ,{\lambda }_k)\end{array}$(18) ), we obtain (24) ${\tilde{U}}_k^T{\tilde{U}}_k=\sqrt{D_{\hat{P}}^{-1}}{Y}^T\hat{P}\hat{P}Y\sqrt{D_{\hat{P}}^{-1}}=I_k$(24) and (25) ${\tilde{U}}_k^T{\tilde{U}}_k=\sqrt{D_{\hat{P}}^{-1}}{Y}^T\hat{P}\hat{P}Y\sqrt{D_{\hat{P}}^{-1}}=I_k,$(25) where $I_k$ is the identity matrix of dimension k.

As regards (iii), the properties 1–4 of Lemma 3.3 immediately follow from (20(20) $\begin{array}{rl}\tilde{P}& ={\tilde{U}}_k\sqrt{{\tilde{D}}_k},\tilde{Q}=\sqrt{{\tilde{D}}_k}{\tilde{V}}_k^T,\end{array}$(20) ), by using (ii).

We finally prove properties (iv) and (v). By substituting (20(20) $\begin{array}{rl}\tilde{P}& ={\tilde{U}}_k\sqrt{{\tilde{D}}_k},\tilde{Q}=\sqrt{{\tilde{D}}_k}{\tilde{V}}_k^T,\end{array}$(20) ) in Equation (21(21) $\begin{array}{rl}\overline{R}& =\tilde{P}\tilde{Q},\end{array}$(21) ) we obtain: (26) $\overline{R}=\tilde{P}\tilde{Q}={\tilde{U}}_k{\tilde{D}}_k{\tilde{V}}_k^T$(26) Then, by construction, the obtained matrix $\overline{R}$ coincides with matrix ${\overline{R}}_k$ associated to relation (4(4) $R_k=U_k{D}_k{V}_k^T$(4) ).

Furthermore, again by construction, the singular values ${\sigma }_1,\dots ,{\sigma }_k$ of $\overline{R}$ coincide with the product of the square roots of the eigenvalues of the matrices ${\hat{P}}^T\hat{P}$ and $\hat{Q}{\hat{Q}}^T$: (27) ${\sigma }_i={\left({\hat{D}}_k\right)}_{ii}=\sqrt{{\mu }_i}\sqrt{{\lambda }_i}\mathrm{\forall }i=1,\dots ,k$(27)

The Theorem 3.4 constitutes the theoretical basis of the RSVD method and allows generating the matrices $\tilde{P}$ and $\tilde{Q}$ that represent the fake numerical profiles matrices related to users and items. Moreover, they are obtained from the input matrices $\hat{P}$ and $\hat{Q}$ that contain the user's and items profiles. The RSVD method has been applied in designing an application to support an individual visiting a new location (Casillo et al., 2019; Casillo, Conte, et al., 2022). The application operates in three successive phases:

• the Fetch Phase acquires data about the user's position and provides an initial description of the site of interest. It also searches for all resources, services, and events through external sites and requests access to the user's social data for the creation of a profile that describes his interests;

• the Decode Phase calculates user/resource affinities and introduces the context variables through the Context Dimension Tree;

• the Execute Phase presents the information processed in the previous phases through a contiguous narrative. In addition, the system provides a list of recommended activities to users.

In the design of the described application, the Recommender Systems are inserted in the Decode Phase with the calculation of user/resource affinities. The available data for creating the rating forecasts (in the specific case coinciding with the affinities) are user profiles and information acquired on resources through external sites. According to Casillo, Conte, et al. (2022), the construction of users' profiles takes advantage of the social network information and calculates the friendship degree among users and the level of interest in the selected categories of interest. The items' profiles are built through binary vectors in which each feature describes if the item belongs to the specific category of interest. However, the creation of profiles is strictly related to each study case. Applying the recommendation method in a specific field can involve different methods to create profiles. For instance, the aim to enhance cultural heritage sites can need to exploit experts to profile items (Casillo, De Santo, et al., 2021); instead, the application in the learning paths field can involve the necessity to use a questionnaire to profile users (Carbone et al., 2021). The different techniques exploited in various fields allow determining the input matrices $\hat{P}$ and $\hat{Q}$.

The elaboration of input matrices $\hat{P}$ and $\hat{Q}$ is described by the Algorithm 1 and summarised in Figure 1. The input matrices are elaborated through the Users Elaboration Submodule and the Items Elaboration Submodule. The first Submodule processes points 2 and 4, which are related to the Algorithm 1. In particular, the Users Elaboration Submodule calculates the eigenvalues and the eigenvectors of the matrix ${\hat{P}}^T\hat{P}$ and memorises the first in the diagonal elements of the matrix $D_{\hat{P}}$ and the seconds in the matrix Y. Then the matrix $\tilde{P}=\hat{P}Y$ is obtained. Instead, the second Submodule processes points 3 and 5 related to the Algorithm 1. In particular, the Items Elaboration Submodule calculates the eigenvalues and the eigenvectors of the matrix $\hat{Q}{\hat{Q}}^T$ and memorises the firsts in the diagonal elements of the matrix $D_{\hat{Q}}$ and the seconds in the matrix X. Then the matrix $\tilde{Q}=X^T\hat{Q}$ is obtained.

Figure 1. Architecture related to RSVD method.

The obtained matrices $\tilde{P}$ and $\tilde{Q}$ satisfy the orthogonality properties as proved in the Theorem 3.4. Finally, the Rating Forecast Submodule calculates the rating forecasts through the matrix's product according to Equation (21(21) $\begin{array}{rl}\overline{R}& =\tilde{P}\tilde{Q},\end{array}$(21) ). Finally, the rating forecasts matrix $\overline{R}$ is regularised through the procedure described by Barragáns-Martínez et al. (2010) for Collaborative Filtering methods.

4. Numerical results

This Section provides the numerical experiments related to the accuracy measure of the RSVD method. The proposed approach is compared to Average Rating Filling (ARF) (Anastasiu et al., 2016; van der Vlugt, 2018), Stochastic Gradient Descent (SGD) (Anastasiu et al., 2016; R. Wang et al., 2019; Zhao et al., 2019), and Biased Stochastic Gradient Descent (BSGD) (R. Wang et al., 2019; Zhao et al., 2019) methods. Another method compared with RSVD is the process that obtains rating forecasts through the product of the input matrices $\hat{P}$ and $\hat{Q}$. This method is defined as PQ and aims to give consistency to RSVD and apply the Theorem 3.4.

To compare the accuracy related to the methods, the datasets MovieLens100k and MovieLens10M are exploited, and the accuracy measures are the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). Let N be the set of pairs $(i,j)$ for which the numerical test is carried out and ${\hat{r}}_{ij}$ the prediction made for the pair $(i,j)$. Mean Absolute Error is defined in Equation (28(28) $\begin{array}{rl}\text{MAE}& =\sum _{(i,j)\in N}\frac{|{\hat{r}}_{ij}-r_{ij}|}{|N|}\end{array}$(28) ) and the Root Mean Square Error is defined in Equation (29(29) $\begin{array}{rl}\text{RMSE}& =\sqrt{\sum _{(i,j)\in N}\frac{{({\hat{r}}_{ij}-r_{ij})}^2}{|N|}}\end{array}$(29) ). (28) $\begin{array}{rl}\text{MAE}& =\sum _{(i,j)\in N}\frac{|{\hat{r}}_{ij}-r_{ij}|}{|N|}\end{array}$(28) (29) $\begin{array}{rl}\text{RMSE}& =\sqrt{\sum _{(i,j)\in N}\frac{{({\hat{r}}_{ij}-r_{ij})}^2}{|N|}}\end{array}$(29) The exploited datasets are MovieLens100k, and MovieLens10M made available by the GroupLens Research Project of the University of Minnesota (Harper & Konstan, 2015). The item profile matrix $\hat{Q}$ was obtained through the 'u.item' file for MovieLens100k and from the “movies.csv” file for the MovieLens10M database. The item matrix element $(c,j)$ is 1 if the movie jth is part of the category cth otherwise it is zero. Movie categories are available in the 'u.genre' file provided by the database. About the user profiles matrix, in the case of MovieLens100k, the matrix $\hat{P}$ was formed through the means of the known ratings by category associated with each user. On the other hand, in the case of the MovieLens 10M database, tag information was exploited for user profiling. Since the available tags are 100,000, users without information were provided an average profile $p=\frac{(1,\dots ,1)}{\Vert (1,\dots ,1)\Vert }$ that does not characterise the user but that allows the system to provide the best items available. This choice allows overcoming the problem of the initial cold start of the system that later will have the opportunity to adjust the specific user's preferences. The chosen value k is selected by the number of movie categories provided equally to 18.

The employment of datasets that contain known ratings and tags penalises the proposed approach. However, this application provides an estimation of the proposed approach's accuracy.

The parameters needed for SGD and BSGD to provide reliable rating forecasts are selected experimentally, and the results provided are summarised in Figures 2 and 3. Figure 2 describes the variation of the RMSE related to the parameter α. Figure 3 describes the variation of the RMSE related to the variation of the parameters α and λ

Figure 2. Graphical representation of RMSE error of method SDG when α changes.

Figure 3. Graphical representation of RMSE error of method BSDG when α and λ change.

After the experimental evaluation of the parameters, the values exploited to compare these methods to RSVD are:

• $\alpha =0.01$ for SGD;

• $\alpha =0.04$ and $\lambda =0.005$ for BSGD.

The number of iterations needed for the training phase of SGD and BSGD is 10, 20, and 40. Moreover, due to oscillating results on the value of MAE for the methods SGD and BSGD, the reported results are the median of the value obtained in 11 experiments.

Furthermore, the rating forecasts provided by RSVD, and PQ methods on MovieLens100k and MovieLens10M are regularised on the interval $[1,5]$.

4.1. Numerical results on movieLens100k

The results provided by the methods on the dataset MovieLens100k are summarised in Table 2. The RSVD method returns a worse accuracy than the comparison methods ARF, SGD, and BSGD, except for the case of the BSGD method applied with 10 iterations.

Table 2. Results of the methods ARF, SGD, BSGD, PQ, and RSVD on MovieLens100k dataset.

Method	k	iter (if any)	MAE	RMSE
ARF	18		0.6882	0.8741
SGD	18	10	0.7227	0.7813
	18	20	0.7210	0.6973
	18	40	0.6506	0.6586
BSGD	18	10	0.4829	1.0721
	18	20	0.4821	0.9198
	18	40	0.5824	0.8043
PQ	18		1.6458	1.8986
RSVD	18		0.7664	0.9512

These results applied in the field that benefits the comparison methods testify that despite the adaption of the proposed approach on a classical Collaborative Filtering dataset, the provided accuracy is promising. Indeed, the provided results provide evidence to generate reliable rating forecasts despite the need to adapt the Content-Based procedure of RSVD to the presence of available ratings. In addition, the results provided allow evaluating the idea of using available ratings for profile creation in RSVD in case of the impossibility of acquiring other typologies of information. The use of a matrix factorisation technique improves the ability of RSVD to deal with the scalability problem compared to classical Content-Based methods.

The value k, in the RSVD method, represents the number of categories of interest to the user. Therefore it is possible to intuitively build user and item profiles needed for RSVD, without adding too much computational cost to the method.

The next focus is on the differences between the RSVD and PQ procedures. The poor results found for the second method justify the increase in complexity associated with the first, which, as mentioned, is comparable with previously tested methods.

Finally, RSVD is compared with the recommendation method exploited by Casillo et al. (2019). This methodology, defined as PREC, represents a Content-Aware RS in which the affinity user-item calculation can be extrapolated to compare the accuracy provided with the RSVD method. Table 3 shows the accuracy results related to MAE and RMSE provided by RSVD and PREC in MovieLens100k dataset.

Table 3. Accuracy related to RSVD and PREC methods on MovieLens100k dataset.

Method	MAE	RMSE
RSVD	0.7664	0.9512
PREC	0.8757	1.1618

The accuracy provided by the RSVD method is better than the recommendation method exploited in Casillo et al. (2019). The Mean Absolute Error and the Root Mean Square Error are lower than the errors related to PREC.

4.2. Numerical tests on movieLens10M

This subsection compares the RSVD method with SGD and BSGD on MovieLens10M. This database contains 10 million ratings and 100,000 tags. The $\hat{P}$ is built only with tags. If there are not enough ratings in order to determine a profile of user a standard one is used. Defined $p=(1,\dots ,1)\in {\mathbb{R}}^k$, the standard user profile is provided by Equation (30(30) $p_u=(\frac{1}{\Vert p\Vert }\dots \frac{1}{\Vert p\Vert })\in {\mathbb{R}}^k$(30) ). (30) $p_u=(\frac{1}{\Vert p\Vert }\dots \frac{1}{\Vert p\Vert })\in {\mathbb{R}}^k$(30) Instead, the SGD and BSGD methods are tested with training sets selected in various cardinalities to prove that RSVD can give better rating forecasts with less information. The cardinalities chosen for training sets are:

• 100,000 training ratings;

• 165,000 training ratings;

• 500,000 training ratings;

• 1,000,000 training ratings;

The methods exploit all possible cardinality of the training sets to the training phase of SGD and BSGD. Instead, RSVD uses all ratings of the dataset in the test phase because this method does not need available ratings to predict forecasts.

Table 4 provides the numerical results of the methods with the specific cardinality related to the training phase. The comparison between SGD and RSVD underlines that the first method needs at least 1 million known ratings to provide better MAE and RMSE than RSVD. The numerical results testify that SGD provides better forecasts than RSVD through an information quantity greater than 100 times that of RSVD. The accuracy difference provided by numerical results can be related to the different information exploited by Collaborative Filtering methods (SGD and BSGD) and the proposed approach RSVD. The RSVD method does not exploit users' preferences but only the items category and the standard user profile (30(30) $p_u=(\frac{1}{\Vert p\Vert }\dots \frac{1}{\Vert p\Vert })\in {\mathbb{R}}^k$(30) ). Moreover, RSVD does not need a training phase and builds predictions without using available ratings, and it gives good values of MAE and RMSE. In addition, using an average profile without information allows for overcoming the problem of a cold start in practice.

Table 4. Results of the methods SGD, BSGD and RSVD.

Method	Training cardinality	MAE	RMSE
SGD	100,000 ratings	2.4420	2.6583
	165,000 ratings	1.9197	2.2261
	500,000 ratings	0.9354	1.2274
	1 milion ratings	0.7800	1.0194
BSGD	100,000 ratings	0.8612	1.0975
	165,000 ratings	0.8870	1.1322
	500,000 ratings	0.9274	1.1878
	1 milion ratings	0.8978	1.1547
RSVD	100,000 tags	0.9588	1.1877

Comment on Numerical Results.

Section 4 focuses on the numerical results of the proposed approach. The provided results return an accuracy measure worse than the ARF, SGD, and BSGD methods on MovieLens100k. Instead, the comparison with the PQ method provides evidence of the consistency of the Theorem 3.4 and the need to modify the users and items profiles to improve the rating forecasts' reliability. Moreover, the comparison with PREC testifies that the proposed approach improves the recommendation method exploited by Casillo et al. (2019) that represents the field in which the RSVD method is born. On MovieLens10M, the ability of the proposed approach to work in the information lack field is tested. The comparison methods exploited the users' preferences through a few known ratings, and the training cardinality varies in the four possibilities described before. Instead, RSVD works with the user interaction with the movies or creates a standard profile that can not exploit the user preference. Despite this disadvantage, the accuracy is promising because the test phase is based on known rating datasets. These datasets are focused on the Collaborative Filtering methods of testing. Indeed, the proposed approach can work in the classical Content-Based field, with a process to create users and items profiles, with the ability to deal with the scalability problem.

The experiments testify that the proposed approach provides reliable rating forecasts. It does not overcome the Collaborative Filtering comparison methods but obtains satisfactory accuracy results, despite the proposed approach being Content-Based. Moreover, on the MovieLens10M dataset, the recommendation methodology provides an MAE of 0.9588 and an RMSE of 1.1877, working with tags for user profile creation. Moreover, the Content-Based recommendation methods have application fields different from Collaborative methods. Then, to validate the proposed approach, efficacy experiments in various application domains must be organised.

5. Conclusions

This paper aims to propose a novel methodology that provides rating forecasts without known ratings. The experimental results are challenging compared to the methods presented in Section 3. After the theoretical formalisation of the RSVD method, it may be concluded that the possibility of making forecasts, without the need for known ratings through the information associated with the characteristics of users and items, allows providing an adequate starting point for a system that can not take advantage of standard procedures. Due to the encouraging results regarding the RSVD method, future works include introducing the context variables to complete and finalise the Decode Phase within the presented case study.

Disclosure statement

No potential conflict of interest was reported by the author(s).