Personalized tourist route recommendation model with a trajectory understanding via neural networks

ABSTRACT

Travel recommendations form a major part of tourism service. Traditional collaborative filtering and Markov model are not appropriate for expressing the trajectory features, for travel preferences of tourists are dynamic and affected by previous behaviors. Inspired by the success of deep learning in sequence learning, a personalized recurrent neural network (P-RecN) is proposed for tourist route recommendation. It is data-driven and adaptively learns the unknown mapping of historical trajectory input to recommended route output. Specifically, a trajectory encoding module is designed to mine the semantic information of trajectory data, and LSTM neural networks are used to capture the sequence travel patterns of tourists. In particular, a temporal attention mechanism is integrated to emphasize the main behavioral intention of tourists. We retrieve a geotagged photo dataset in Shanghai, and evaluate our model in terms of accuracy and ranking ability. Experimental results illustrated that P-RecN outperforms other baseline approaches and can effectively understand the travel patterns of tourists.

KEYWORDS:

• Recommendation system
• travel trajectory
• recurrent neural networks
• Flickr geotagged photos

1. Introduction

With the rapid development of tourism industry, major changes have taken place in tourists’ travel behavior (Uriely 2005; Rodríguez et al. 2012). Pre-organized itineraries or packages recommended by travel agencies are no longer the first choices of tourists. Personalized travel arrangements are favored by more and more tourists (Uriely 2005; Hyde and Lawson 2003; Mou et al. 2020b). Online travel platforms provide tourists with convenient information services, but tourists often need to spend much energy and time reading and filtering information to aid itinerary planning (Majid et al. 2013; Kolahkaj et al. 2020). Therefore, how to intelligently recommend personalized tourist routes through aggregating information is an urgent problem in tourism research (Leask 2010; Buhalis and Law 2008; Santos et al. 2019).

The popularity of smartphones and the rise of social media have allowed tourists to share their geographic location information anytime and anywhere (Li et al. 2019), leading to an accumulation of large amounts of geotagged data on social media (Cai et al. 2018b; Koswatte, McDougall, and Liu 2018; Yan et al. 2020). As a typical representative of tourists’ collective wisdom, these volunteered geographic information data metaphor the travel trajectory, expose travel preferences of tourists, and provide another way for personalized tourist route recommendations research (Xu et al. 2021; Huang et al. 2021a; Wu et al. 2019). Currently, research on using geotagged social media data for travel recommendations has been carried out and with some results (McKercher et al. 2014; Hannam, Butler, and Paris 2014). However, most of the current research only uses the traditional recommendation model, or make limited improvement, while the research on fine-grained movement pattern analysis and personalized travel recommendation is still in its early stages (Hannam, Butler, and Paris 2014; Liao and Zheng 2018; Yuan 2018).

Firstly, most studies stress travel preferences of the majority, ignoring the specific needs of individual tourists (Kim et al. 2013; Zheng, Huang, and Li 2017; Xiaoyu Sun, Huang, and Peng 2018). Secondly, the existing studies rarely consider the order of visitation of tourist attractions (Xia, Zeephongsekul, and Packer 2011; Yang, Fik, and Zhang 2013), which is usually thoughtless, for the next visit of the tourist is closely linked to historical choice and its order (Song et al. 2010; Zheng, Huang, and Li 2017). Thirdly, current research rarely discusses the dynamic change of tourists’ interest, since uncertain information such as traffic, weather, or emergencies is not only random but also changing over time (McKercher et al. 2014; Verbeeck, Vansteenwegen, and Aghezzaf 2016; Kolahkaj et al. 2020), resulting in changes in interest over time. Finally and most importantly, tourists may conduct different travel behaviors at different times, leading to different importance of tourist attractions at different times (Wang et al. 2021; Brum-Bastos, Long, and Demšar 2018; Xu et al. 2022). For example, tourists make their way to airports, hotels, restaurants, and attractions. Airports and hotels have a greater impact on the check-in of attractions than restaurants, although their check-in order is not next to each other. Therefore, how to integrate various complex information to precisely model the behavior of tourists and recommend personalized travel routes still faces many challenges (McKercher et al. 2014; Hannam, Butler, and Paris 2014).

The latest development of neural networks gives us a new perspective on this issue. In this study, we introduce a personalized recurrent neural network recommendation model, called P-RecN, which can address the temporal relationship of historical visits and capture important travel behaviors. More specifically, a trajectory encoding module is designed to extract semantic features from tourists’ trajectory data and travel background as input. Then, the sequential travel behavior of tourists is simulated using two independent Long Short Term Memory networks (LSTM), one encoding the travel history of the previous stage and the other recommending potential tourist attractions for the next stage with beam search technique. Furthermore, our model introduces a temporal attention mechanism to capture tourists’ movement patterns in temporally correlated environments.

Travel trajectories are reconstructed from geotagged photos and comparative experiments are designed using several state-of-the-art methods. Results indicate that the P-RecN model is achievable, with higher accuracy than other baselines. We further explore the short-term and long-term dependence of various attractions in travel trajectories and explain the travel patterns of tourists. This research may contribute to the expansion of the mobility analysis of tourists in the direction of active integration of artificial intelligence technology into smart tourism.

The rest of this paper is organized as follows: section 2 reviews the related work; section 3 introduces the P-RecN model; section 4 compares the performance of different models; section 5 discusses the advantages and limitations of the model, as well as its application in practice; section 6 concludes this paper and suggests future work.

2. Related work

Numerous work has solved the task of attractions recommendation and route planning in recent years, and we emphasize the three tasks most related to this research and enumerate some representative literature.

2.1 Conventional attraction recommendation

Collaborative filtering (CF), the most representative, achieves recommendation by measuring the similarity of tourists or attractions (Cui, Luo, and Wang 2018; Bao et al. 2015). Xu, Chen, and Chen (2015) used CF to provide recommendations for tourists with similar topic preferences. Majid et al. (2013) constructed a tourist-attraction adjacency matrix based on geotagged photos and predict the preference value of tourists to attractions. To alleviate data sparsity, different contextual information has been taken into consideration, including geographic influence, social relationships, time, etc. For example, Huang (2016) has designed three context-aware CF that can recommend attractions matching tourists’ preferences and context. Content-based methods are another popular recommendation model that builds preference models by extracting the features of tourists and attractions, which can resist the cold-start problem (Bao et al. 2015; Bobadilla et al. 2013). But its recommendation accuracy depends on the manual design of feature engineering and the fine construction of preference models. Neural networks have no strict requirements on data, can fit the complex relationship of tourism data and accurately model the implicit features of the interaction between tourists and attractions without complex feature extraction engineering.

The transfer relationship between attractions has been included in travel recommendation and Markov Chain (MC) is a common state transfer model. Xia, Zeephongsekul, and Packer (2011) simulated the temporal and spatial movement of tourists using a Semi-Markov model. Kurashima et al. (2013) presented a Topic-Markov method to analyze tourists’ preferences and achieve continuous attraction prediction. Liu and Wang (2018) used high-order MC to capture sensitive visits in travel history. However, the MC-based method relies on the assumption of strong independence between different positions in the sequence and has a drawback in capturing long-term dependencies. Existing study also shows that high-order MC has complex calculation problems, and discarding part of the data cannot describe the complete information of the entire travel trajectory (Song et al. 2010; Zheng, Huang, and Li 2017).

Machine Learning has become an increasingly powerful tool in tourism recommendation research (Deng and Li 2018; Sun et al. 2019). Wan et al. (2018) trained two classifiers, K-Nearest Neighbor and Bayes, to quantify season, temperature, weather and location history of tourists to predict the next attraction. Xiaoyu Sun et al. (2018) used Support Vector Machine (SVM) and Gradient Decision Boosting Tree to improve the sorting effect of the ranking list of attraction recommendations. Ding and Chen (2018) suggested a personalized POI recommendation model, which embeds the categorical, geographical and co-visiting influences into the deep neural network. These studies have made significant contributions to the research on personalized travel recommendation. They concentrate on successful query processing from a given dataset, whereas our model is a semi-supervised learning model that focuses on capturing the intrinsic movement patterns of trajectories from tourists’ travel histories.

2.2 Travel route recommendation

Route recommendation aims to generate a continuous sequence of attractions that meet specific constraints and maximize the obtained utility. Most studies model it as Orienteering Problem (OP) or its variants (Gavalas et al. 2014; Gunawan, Lau, and Vansteenwegen 2016; Sarkar et al. 2020). Kotiloglu et al. (2017) proposed a framework of ‘filter-first, tour-second’ and discussed the multi-constraint and multi-period problems of route planning. Lim et al. (2018) considered the time budget and personalized visit time characteristics to maximize the global benefits of attraction popularity and interest preferences. Different constraints and utility gains will bring different itinerary generation solutions. Zheng, Liao, and Qin (2017) considered the variable sightseeing value of attractions and the aesthetic fatigue of tourists, and proposed a four-step heuristic algorithm for one-day tour routes recommendation. Later, Liao and Zheng (2018) further discussed this issue in a time-related constraint to improve the reliability of route design. These researches have tried to simulate the complex reality of tourists and attractions (such as attraction opening hours, consumption budget, etc.) and excellent recommendation performance has been proven. However, tourism activities are a changeable process, and certain constraints may not support the dynamic use of the model. In contrast, P-RecN is a data-driven algorithm, which relies on the powerful self-learning and self-organization capabilities of neural networks to capture the inherent movement patterns presented by trajectory data.

2.3 RNN-based trajectory learning

Recurrent Neural Network (RNN) has emerged as the most promising model for solving location sequence learning problems, and contextual data can be successfully incorporated into the RNN structure. May Petry et al. (2020) suggested a trajectory classification method based on attribute embedding and RNNs to capture trajectory sequence patterns. Liu et al. (2019) used a Gated Recurrent Unit to model spatio-temporal behavior of tourists. Yi et al. (2020) used Large-scale Information Network Embedding to obtain location vector representation, and applied BiLSTM-CNN for locations recommendation. However, most RNN models rely on the last hidden layer vector when calculating the output, which makes it difficult to understand the specific part of the input sequence. It is worth noting that these works are orthogonal to ours. The difference lies in the fact that we use an encoder-decoder framework to model the location sequence learning problem, propose a semi-supervised learning method to capture the main behavior intention and sequential patterns implicit in the trajectory, and generate the recommended route in an end-to-end manner.

3. Methodology

3.1 Definition and framework

The personalized tourist route recommendation is modeled as a sequence learning problem. Given a tourist’s continuous visit history $X=\{x_1,x_2,\dots ,x_T\}$ in a period, the purpose of the P-RecN model is to recommend a tourist route $Y=\{y_{1^{\mathrm{\prime }}},y_{2^{\mathrm{\prime }}},\dots ,y_{T^{\mathrm{\prime }}}\}$ that the tourist is most likely to visit in the next period, where $x_i(1\le i\le T)$ represents the i-th visited attraction of the tourist, and $y_{i^{\mathrm{\prime }}}(1^{\mathrm{\prime }}\le i^{\mathrm{\prime }}\le T^{\mathrm{\prime }})$ represents the $i^{\mathrm{\prime }}-th$ attraction to be visited next.

Given the above definition, the workflow of the P-RecN model is shown in Figure 1, which includes three parts: a trajectory encoding module that integrates tourist trajectory information and context information, an LSTM encoder-decoder neural network module combined with a temporal attention mechanism, and an output module. Detailed information and functions of each part of the model will be introduced in the following sections.

Figure 1. Research workflow of P-RecN.

3.2 Trajectory encoding module

Data in neural networks is transmitted in the form of vectors. The trajectory encoding module of P-RecN can map the trajectory information and context information that affect the selection of attractions into feature vectors, making the training process more efficient. Considering the sparsity and heterogeneity of different information, we use an embedding layer to uniformly encode them. In neural networks, the embedding layer can map enumeration values representing similar patterns into adjacent latent spaces to obtain low-dimensional dense vectors. Figure 2 shows an example of encoding a single tourist trajectory. These different types of context information are mapped into d-dimensional vectors by one-hot encoding. One-hot encoding mainly uses an N-bit state register to encode N states, and only the position of the corresponding attribute value is 1. In our example, there are 9 types of attractions, so the one-hot encoded attraction category vector has 9 dimensions, and Nanjingxilu in Figure 2 corresponds to the third element in the encoded vector. The encoded vectors are multiplied by their respective embedding matrices to convert discrete vectors into efficient but meaningful embedding representations for deep learning algorithms that reduce latent space dimensionality while better measuring their similarity.

Figure 2. An example of tourist travel trajectory embedding.

In this paper, we randomly assign each attraction a unique ID to mark the attraction’s unique identity, and tourists’ travel trajectories are represented by continuous attractions ID strings. But ID strings cannot represent the co-occurrence relationship between attractions and cannot be directly used as neural network input. Therefore, the study adopts word embedding technology to learn the representation of attractions. The method converts attraction IDs into a dense vector of continuous values by learning an embedding matrix W^D×M, where M is the number of attractions and D is the embedding dimension of the attractions. The word embedding acts on the trajectory data, it is natural to treat each attraction as a word, the attraction set as the total vocabulary, and each attraction is identified by a unique attraction ID in the same way as the word ID. The embedding matrices can be randomly initialized and trained with other components of P-RecN or initialized with pretrained embeddings.

Different information must be aggregated to be fed into the recurrent component of P-RecN for tourist sequence behavior learning. The study employs a connection function to fuse different embedding vectors in a concatenated manner, so that at time t, all trajectory features and contextual information are contained in the input vector x_t, as shown in Figure 2.

3.3 LSTM encoder–decoder neural network

The interaction of attractions occurs in a certain order, and LSTM is the mainstream technology of sequence learning in neural networks, where it is used to capture the high-order dependencies of the previous attraction on the next. LSTM is a variant of RNN (Hochreiter and Schmidhuber 1997). It introduces a new hidden state, called cell state, which stores the summary of past interaction. The calculation process is shown in Figure 3. f_t, i_t, and o_t respectively represent the forget gate, input gate and output gate, which control the discarding, updating and outputting of the amount of information in the cell memory.

Figure 3. Structure of LSTM neural network unit repeating module.

Specifically, given the previous attraction embedding sequence$X=\{x_1,x_2,\dots ,x_t\}$, LSTM takes each attraction interaction as input, and outputs the corresponding hidden state$h=\{h_1,h_2,\dots ,h_t\}$after three gate mechanisms. Each hidden state h_t contains information about the current moment and the previous moment. At the same time, h_t also captures the sequential pattern of the entire sequence until time t. Where σ indicates a sigmoid function, defined as$\sigma =1/(1+e\widehat{(}-x))$. Tanh, a hyperbolic tangent function, is used to adjust the value flowing through the neural network. ⊙ is element-wise multiplication.

To establish the mapping relationship between historical visits and recommendation results, we stack the LSTM neural units into an encoder-decoder structure. Specifically, the encoder is constructed to analyze the sequence features hidden in the travel trajectory, and the decoder is used to generate the output sequence. Unlike the LSTM structure, the input and output trajectory lengths of LSTM-based encoder-decoder are variable, and its training process is an end-to-end process, which has great flexibility for sequence-to-sequence modeling tasks.

As shown in Figure 4, the encoder inputs the travel trajectory $x_1,x_2,\dots ,x_T$ of length T recursively, and gradually updates the hidden state h_t and cell state c_t. After T time steps, the encoder compresses the hidden state of each time step into a fixed semantic vector S through a function $\lambda$. The decoder can be regarded as the inverse operation of the encoder, using a virtual identifier y₀ as the first input, and then decoding the corresponding hidden state according to the semantic vector S and the output $y_{t-1^{\mathrm{\prime }}}$ of the previous step. (3.1) $h_t^e=LST{M}_e(x_t,h_{t-1})$(3.1)

(3.2) $S=\lambda (h_1,h_2,\dots ,h_t)$(3.2) (3.3) $h_{t^{\mathrm{\prime }}}^d=LST{M}_d(y_{t-1^{\mathrm{\prime }}},h_{t-1^{\mathrm{\prime }}},S)$(3.3)

Figure 4. LSTM encoder-decoder module.

where $h_t^e$ and $h_t^d$represent the hidden states of the encoding and decoding stages. During the decoding stage, the decoder generally chooses the candidate with the maximum conditional probability as the current optimum result. (3.4) $p\left(y_{t^{\mathrm{\prime }}}\right)=\prod _{t=1^{\mathrm{\prime }}}^{T^{\mathrm{\prime }}}p\left(y_{t^{\mathrm{\prime }}}\right|\{y_{t-1^{\mathrm{\prime }}},y_{t-2^{\mathrm{\prime }}},\dots ,y_{1^{\mathrm{\prime }}}\},s)=\tau (h_{t^{\mathrm{\prime }}}^d,y_{t-1^{\mathrm{\prime }}},s)$(3.4) To reduce the impact of error propagation, beam search technique is used to find the global optimal solution (Sutskever, Vinyals, and Le 2014). It can select the K alternatives with the maximum conditional probability for the input of each time step of the decoders. After T iterations, K candidate sequences are generated as the most probable K recommendation results. The K highest probabilities here do not only refer to the highest probability at the current moment but the sum of the cumulative probabilities on the entire path up to now. Like other studies, the K is set to 3, because a larger value does not always help improve model performance (Huang et al. 2021b; Li et al. 2020). The decoder combined with beam search technology outputs the global optimal solution and inputs it to the output module. The output module acts as a sorting function, selecting the best trip from the candidate paths generated by the decoder as the target route.

Despite using beam search technology, a potential problem of the encoder-decoder is that the encoder compresses all travel history into a fixed-length vector, which will inevitably lose important information. The decoder relies on the final semantic vector and cannot accurately understand the importance of each attraction. Especially, the more tourist attractions tourists visit, the more obvious the importance of tourist attractions at a different time. To fill this gap, a temporal attention mechanism is introduced to force the decoder to select input vectors closely related to the current output.

3.4 Temporal attention mechanism

The attention mechanism can make the neural network pay more attention to the important information in the input and reduce the interference of irrelevant information. It has been widely used in the fields of machine translation and computer vision (Chaudhari et al. 2021; Vaswani et al. 2017). Here, a temporal attention is designed to allow the decoder to dynamically select important parts of the past behavior sequence to focus on the main intention of the tourist. Its essence is the alignment model between the target attraction and each attraction in the historical trajectory. By learning the influence weights of all hidden states and retrieving their weighted sum, the dynamic time correlation between the future and historical time steps is captured. The weighted sum of the hidden states is interpreted as the main intent feature of the tourist. Figure 5 shows the structure of the model.

Figure 5. The network structure of temporal attention.

Given the previous hidden state$h_{t-1^{\mathrm{\prime }}}^d$and cell state$c_{t-1^{\mathrm{\prime }}}^d$in the decoder, the dependency correlation $e_{t,t^{\mathrm{\prime }}}$between the hidden state h_t and the output of future time step $t^{\mathrm{\prime }}$ is measured by a multilayer perceptron. (3.5) $e_{t,t^{\mathrm{\prime }}}={\vartheta }_d^{\mathrm{T}}\tanh \left(W_d\right[h_{t-1^{\mathrm{\prime }}}^d;c_{t-1^{\mathrm{\prime }}}^d]+U_d{h}_t+b_d^{})$(3.5) where${\vartheta }_d^{\mathrm{T}}$, $W_d$, $U_d$ and $b_d$represent the parameters that need to be learned. The temporal attention weights of different hidden states are normalized into [0, 1] using a softmax function. Then the temporal attention semantic vector $s_{t^{\mathrm{\prime }}}$ is calculated by the weighted sum: (3.6) ${\alpha }_{t,t^{\mathrm{\prime }}}={\displaystyle \frac{\exp \left(e_{t,t^{\mathrm{\prime }}}\right)}{{\sum }_{k=1}^T\exp (e_{t^{\mathrm{\prime }},k})}}$(3.6) (3.7) $s_{t^{\mathrm{\prime }}}={\sum }_{t=1}^T{\alpha }_{t,t^{\mathrm{\prime }}}{h}_t$(3.7) Among them,$s_{t^{\mathrm{\prime }}}$represents the semantic vector corresponding to time t, and${\alpha }_{t,t^{\mathrm{\prime }}}$represents the normalized attention weights. The higher the value is, the more attention will be paid to the input.

3.5 Network learning

Given the historical trajectory record $V_s$ of a tourist$u\in U$, the current query time$Q_T$, and the query attractions set$Q_A$, our goal is to train an optimal parameter loss to match the $V_s\to V_r$ mapping relationship. The objective function is formulated by a log-likelihood function as follows: (3.8) $loss=\arg \underset{\mathrm{\Theta }}{max}{\displaystyle \frac{1}{\left|\mathrm{\Phi }\right|}\sum _{Q\in \mathrm{\Phi }}\log \left(P_{V_r}\right(a\in A|V_s,u,Q_T,Q_A))}$(3.8) where Θ is the model parameters set. Φ is the set of $\{V_s,u,Q_T,Q_A\}$and$Q\in \mathrm{\Phi }$. The complexity of the model comes mainly from the space complexity and the amount of computation (time complexity). The space complexity O(m) is determined by the number of parameters m of the model. The amount of computation directly determines the forward time of the model and the training/prediction time. The study uses Stochastic Gradient Descent (SGD) technique and Back Propagation Through Time (BPTT) algorithm to optimize the network parameters with a time complexity O(n), where n is the number of multiply-accumulate operations. We also use dropout technology to reduce the risk of overfitting, enabling each neuron to have some probability of not participating in forward and backward propagation.

4. Experiments

4.1 Datasets

In our work, 768,817 geotagged photos are collected from January 2004 to December 2018 in Shanghai, China through the public API of Flickr. Flickr, a popular social media website, accumulates hundreds of millions of geotagged photos taken by photographers and is a widely used data source in travel recommendations studies (Su et al. 2016; Miah et al. 2017; Mou et al. 2020aa). We also extract the descriptive information of each photo, including: ‘Flickr ID’, ‘user ID’, ‘latitude’, ‘longitude’, ‘datetaken’, ‘tags’, ‘title’, and so on. In order to obtain high-precision data, the original data needs to be preprocessed: (1) Eliminating the records whose geographic locations are outside Shanghai; (2) Deleting duplicate records, that is, there are multiple similar photos in the same location at the same time, and the tag information is used to judge the similarity; (3) Deleting non-tourism data based on whether the tag information is related to Shanghai tourism (Sun et al. 2015); (4) Remove local residents. Compared with tourists, residents can take photos at any time of the year, so a heuristic approach based on stay periods was used to distinguish tourists and residents, and the maximum duration is set to 21 days by checking the time span between the first and last photos (De Choudhury et al. 2010; Girardin et al. 2008; Derdouri and Osaragi 2021). Finally, 79,201 photos are retained, and the location distribution is shown in Figure 6. It can be seen that the spatial distribution of tourists’ photo locations is uneven. Most of the shooting locations are concentrated in the city center, while relatively few are located in the suburban areas.

Figure 6. Spatial distribution of Flickr photo data in Shanghai.

Here, an attraction is defined as a special region that is popular in a city and attracts tourists to visit and take photos. It refers not only to a traditional resort, but also to a park, a museum, and even a popular area on the street. Because of this, the behavior of tourists uploading photos is viewed as a positive vote that the location is worth a visit, and extracting tourist attractions from a Flickr photo collection can be regarded as a clustering problem for identifying dense photographic locations (Chen, Arribas-Bel, and Singleton 2019; Chen, Arribas-Bel, and Singleton 2020). Therefore, DBSCAN algorithm is used to cluster the geographical location of photos, and each cluster is identified with a uniquely determined ID used to mark the identity of attractions (Sun et al. 2015). Based on the time when the photos were taken, connecting tourist attractions allows further building of travel trajectories. If the time difference between visits to consecutive attractions is more than 8 h, we will further divide this travel history into separate travel sequence. Afterward, the tourists with less than three attractions and the attractions visited by less than five tourists are eliminated to address data sparseness issues. In the end, a total of 129 tourist attractions and 11,307 valid trajectory records of 4854 tourist are obtained. Figure 7 plots the distribution of these trajectories in Shanghai.

Figure 7. Spatial distribution of tourist attractions and tourist trajectory.

To better portray tourists’ travel background, other contextual information is added to the original data. Table 1 shows the specific examples of attraction context information. Weather information is collected in Weather Underground (wundgerground.com) and added to every historical visit of tourists. We define attraction popularity based on the number of attraction visits, and divide attraction categories into nine different types based on Foursquare.com, the nature of local tourist attractions and previous research (Kotiloglu et al. 2017; Deng et al. 2019): Arts & Entertainment, Great Outdoors, Shop & Service, College & University, Food, Travel & Transport, Residence, Nightlife Spots, Professional & Other Places.

Table 1. Attributes description of tourist attractions in Shanghai.

Attribute	Range/example	N
Category	{Arts & Entertainment, College & University, … }	9
Pop	0.714∈[0,1]	–
Spatial intervals	0.75km	–
Visit time	1∈[0,24]	24
Weather	{Clear, Rain/Snow, Clouds, Heavy rain/snow}	4
Temperature	{Cold, Cool, Warm, Hot}	4
Season	{Spring, Summer, Autumn, Winter}	4
Weekday	{Monday, Tuesday, Wednesday, … }	7

4.2 Experiment setup

4.2.1 Baseline methods

In the experiments, the P-RecN model is compared with several baselines. For each baseline method, the best parameter settings are used in the paper, and the source code and python package provided by the original paper are used for implementation. Several variants of our methods are also implemented for comparison purposes.

POP: A simple approach that selects popular attractions from all attractions to build recommended sequence.

LSTM: LSTM has shown its superiority in human trajectory learning.

LSTM-seq2seq: One LSTM is used to encode the input sequence, and the other LSTM is used to iteratively predict future sequences.

POIRank: POIRank uses RankSVM to calculate the POI score, taking into account contextual information such as category and popularity, and then connects them according to the score to recommend trajectories.

ST-RNN: An RNN-based method that combines temporal and spatial features to predict the next location.

DeepMove: A recurrent neural network with attention mechanism that jointly integrates several factors to learn trajectory sequential patterns.

PersTour (Lim et al. 2018): An OP solution to construct travel itinerary based on tourists’ interest and the variability of the duration of visits to attractions, where the time budget includes both personalized visit duration and traveling time.

PWP (Sarkar and Majumder 2021): It provides multiple itineraries considering the interest of visitors, popularity of itineraries and the travel cost.

P-RecN-nc: A variant model of P-RecN without any contextual information.

P-RecN-na: The model uses the same framework of P-RecN, but without attention structure.

4.2.2 Evaluation metrics and parameter setting

This paper uses Recall, Precision and F1-score as the evaluation metrics. Among them, Recall refers to the percentage of attractions that appear in the recommended result as well as in the actual travel route, and Precision refers to the percentage of attractions that appear in the actual travel route as well as in the recommended result. F1-score represents the harmonic mean of Recall and Precision. We also use Mean Absolute Error (MAE) as the model error evaluation metrics to evaluate the reliability and accuracy of recommended itineraries.

The visiting order is a significant part of the recommended route and Pairs-F1 (Chen, Ong, and Xie 2016) is adopted to evaluate the quality of sequential recommendations. It considers both visiting order and correctness of each pair of attractions by calculating the F1-score of attraction pairs, regardless of whether they are adjacent. (3.9) $pairs-F1={\displaystyle \frac{2\times r\ast \times p\ast }{r\ast +p\ast }}$(3.9)

The value of Pairs-F1 is between 0 and 1, r* and p* are the Recall and Precision of ordered attraction pairs.

According to experimental requirements, each experiment is performed five times, and the average value is recorded as the final result. For the model setting, both the encoder and the decoder are set to have a two-layer LSTM structure. By comparing different values {64,128,256,512}, the final hidden state is set to 256. The model learning rate is set to 0.01, and the batch size is set to 64. To reduce the risk of overfitting, the dropout size is set to 0.5. All experiments are tested with Python 3.6 and TensorFlow 1.8. The implementation process is carried out on a 16G memory platform with Intel (R) Core (TM) i7-8700 @ 3.2 GHz and a single GPU acceleration.

4.3 Performance evaluation

Table 2 presents the recommendation performance of various models. We can find that POP only gives suggestions through pure popularity selection, with the worst performance. Due to the positive influence of the encoder-decoder framework, the performance of LSTM-seq2seq is better than that of LSTM, but still does not achieve good performance. This indicates that a good neural network architecture is not enough to obtain good accuracy in the task of attraction recommendation. Combining the performance of ST-RNN and DeepMove, it can be seen that the RNN-based network structure combined with context information can better capture the sequence patterns and behavior preferences of tourists. From the performance of the two variants, we can see that the design of the trajectory encoding module and the temporal attention mechanism is indeed conducive to improve performance of route recommendation.

Table 2. Evaluation results among different models.

	Recall	Precision	F1-score	Pairs-F1	MAE
POP	0.343	0.447	0.406	0.212	1.231
LSTM	0.584	0.513	0.556	0.319	0.797
LSTM-seq2seq	0.643	0.579	0.616	0.347	0.680
POIRank	0.687	0.642	0.667	0.436	0.436
ST-RNN	0.727	0.682	0.688	0.405	0.560
DeepMove	0.751	0.692	0.706	0.442	0.403
Perstour	0.746	0.708	0.727	0.468	0.330
PWP	0.764	0.733	0.757	0.547	0.242
P-RecN-nc	0.758	0.726	0.752	0.513	0.293
P-RecN-na	0.771	0.732	0.760	0.546	0.267
P-RecN	0.794	0.762	0.786	0.578	0.172

It can be seen from Pairs-F1 and MAE that the traditional recommendation model lacks the self-adjustment ability and the sequential recommendation performance is not ideal due to the accumulation of errors. In contrast, PWP and PersTour model route planning as an orienteering problem and adds additional constraints to control the selection of candidate attractions, and their performance is no less than that of the proposed model. It is observed that the performance of PWP is better than that of PersTour, which may be due to the fact that PWP places more emphasis on the popularity of itineraries and the interest of tourists. It is worth noting that P-RecN does not explicitly model features such as the popularity and category of attractions like PersTour and POIRank, but P-RecN still shows significant superiority. We attribute it to the result of natural selection during the training phase. In fact, P-RecN learns features such as popularity and category of attractions in an implicit and unexplainable way, and adaptively balances the relationship between the transition probability captured by LSTM and the context feature during the recommendation process. In short, as the motivation of our work, P-RecN is superior to previous work in learning the implicit information of trajectory data, especially the performance of sequence recommendation proves that the network architecture based on LSTM is effective in maintaining the order of attractions.

4.4 Sensitive analysis

4.4.1 Influence of embedding dimension size

The embedding vector represents the location description of the real geographic entity in the latent space. The semantic relationship between attractions is learned from their co-occurrence (neighborhood relationship) in geographic space. Generally speaking, the larger the embedding dimension is, the more useful information the model captures. In Figure 8, the performance of P-RecN increases as the embedding dimension increases, indicating that P-RecN cannot learn interactions of features with too small dimensions. It may be that the embedding dimension is too small and the hidden patterns are lost, and the embedding vector cannot satisfactorily express the implicit relationship between the attractions. On the other hand, the implicit interaction between attractions may be too complicated. Too high embedding dimension will weaken the relationship between attractions and lead to overfitting, i.e. P-RecN performs well on the training set but not well on the test. Of course, the larger the dimension, the greater the demand for computing resources (processing speed, memory consumption).

Figure 8. Influence of embedding dimension size.

4.4.2 Influence of different length input and output trajectories

Figure 9 shows the performance of the model under different length input and output sequences. We can find that the predictability of tourist preferences varies with the length of the input trajectory. On the one hand, with the increase of travel route length, the model can describe more travel information and its performance is improved. Good performance only on medium-length input trajectories may be due to the limitation of the Flickr data in Shanghai and that most tourists travel in a hurry and their trajectory is short. These problems force the model to use collective wisdom to generate recommendations. The recommendation bias may be caused by the inconsistency of some tourists’ travel intentions with the mainstream society. On the other hand, the performance of the model does not decrease significantly as the length of the travel trajectory increases, which means that the preference of tourists is not merely controlled by the length of the route. Contextual information, the dependence between different attractions, and the degree of dependence will also affect tourists’ travel decision-making.

Figure 9. Performance of different length input and output trajectories. (a) The performance of different length input trajectories; (b) the performance of different length output trajectories.

Similarly, as the recommended length changes, P-RecN also follows the trend of first increasing and then decreasing. The reason may be that when the recommended length is short, there are more candidate attractions, and the uncertainty of the recommended choice is greater. As the recommended length gradually increases, the model gradually accumulates experience and the recommendation performance improves. When the recommended route is longer, P-RecN still exhibits good performance. This may be attributed to two aspects of the model design: (1) temporal attention mechanism can efficiently calculate the correlation between recommended attractions and historical visits, and control the output of candidate attractions through parallel operations; (2) the beam search technology adaptively selects the best result when decoding the output to avoid the reduction of accuracy caused by the accumulation of errors. The experimental results further prove the superiority of the P-RecN model in the task of route recommendation.

4.5 Empirical analysis

Taking the last attraction of tourists as the recommended target, the influence weights captured by the temporal attention mechanism in the historical trajectory is visualized. As is shown in Figure 10, the different numbers represent different attractions. The color depth of the blue grid indicates how dependent the target attraction is on the historical visit. When recommending the target attraction, different orders of the same attraction in historical visits will impose different influence weights, which reveals that tourists’ travel itineraries and visiting orders of attractions are indispensable components in their travel history structures. Similarly, the order dependence of the target attractions on any attractions visited in the travel history does not monotonously change with the relative position of the attractions in the trajectory, revealing the randomness and uncertainty of individual behavior. Attractions with greater influence on target attractions may be visited early or recently, indicating that short-term and long-term dependencies are prevalent in tourists’ travel trajectories. Through the temporal attention mechanism, we can more intuitively comprehend the significance of different attractions and explore tourists’ potential travel preferences.

Figure 10. Visualization of temporal attention weights heat map of different attractions.

Figure 11(a) shows a tourist’s travel trajectory in the physical space related to target attractions. The depth of blue color around the attractions indicates the dependence of the target attractions on historical visits. It can be found that attraction 121 is more dependent on attraction 129 and attraction 123, possibly because they have similar categories of attraction and shorter transfer distances. From Figure 11(b), we can find that the tourist may prefer to choose attractions such as attraction 124, 127, 120 and 121 after visiting attraction 123. It is not easy to precisely locate tourist’s interests by digging out the popular preferences of tourist flow, this may require further consideration of the sequential movement patterns of tourists. We may notice that the frequent patterns mining results of FP-growth are also appeared in the representation results of attention mechanism (Figure 11(a)), indicating that the model is effective in capturing the partial and overall travel patterns of tourists and achieving accurate recommendations. Overall, exploring the travel sequence and the primary behavioral intent of tourists will help us better understand the complicated travel rules in the real world.

Figure 11. Individual tourist travel trajectory and frequent pattern mining results associated with the target attraction: (a) individual tourist trajectory and corresponding temporal attention weights, (b) results of frequent patterns mining using FP-growth.

Figure 12 shows several typical cases recommended by the P-RecN model. It can be found that false recommendations also appear in the results, as shown in Figure 12(a) and (b). There are possible reasons for this situation: (1) Similar to product and video recommendation, there is also a long-tail distribution phenomenon in travel recommendations, that is, most tourists focus on popular attractions, while tail attractions receive less attention; (2) The travel preferences of some tourists are quite different. This is a typical user cold start problem. The recommendation results cannot meet the special needs of these tourists, which is also in line with the reality. Figure 12(a) and (b) also show two correct recommendation trajectories, which demonstrate the good performance of the proposed recommendation model.

Figure 12. Four representative recommended results of the P-RecN. (a) and (b) are incorrect recommendation results, (c) and (d) are completely correct recommendations. The black line represents the historical trajectory of tourists, the blue and red lines represent the actual route and the recommended result, respectively.

5. Discussion

Our model shows flaws while showing some superiority. First, P-RecN is essentially a data-driven method that automatically captures travel preferences from historical data without labor-intensive feature engineering, which greatly pushes the recommendation system to the intelligent direction. But the model needs to be continuously adjusted through a large number of parameters and nonlinear transformations to fit the complex travel mechanism in the real world. Compared with traditional methods, the neural network presents relatively less internal tourism information, which is not as rich as explanatory recommendation methods (Suresh Kumar et al. 2020; Montavon, Samek, and Müller 2018; Li et al. 2020). Second, the attraction category information input by the model may only be applicable to Shanghai. Due to differences in tourism resources, the input of attraction category features in different cities needs to be redefined. In addition, the contextual information involved in the model is only for general travel events, and the outbreak of specific events may not be conducive to model expansion, such as COVID-19 (Chinazzi et al. 2020). These fresh samples may hinder the generalization performance of the model, and the adaptability of the model to fresh samples still needs further improvement.

Furthermore, this study uses Flickr social media data to reconstruct the travel trajectory of tourists by establishing intelligent algorithms and rules, which provides a new idea for the study of tourism big data (Cenamor et al. 2017; Cai, Lee, and Lee 2018a). However, like other geotagged social media data, Flickr data is also subject to bias. In our case, the data used was not representative of every tourist who visited the city. For example, for some tourists who visited an attraction but did not take or upload a photo, we are not sure if the attraction is to the taste of the tourist. In addition, there are differences among contributors to Flickr data, active visitors may upload more photos, and their data may dominate the model learning process. Differences in the contribution of domestic and foreign tourists will also appear similar. These problems lead to the fact that the collective wisdom learned by the model through the tourist group may be concentrated in a specific group, and the differences in tourist behavior caused by tourist heterogeneity cannot be fully exploited. Nevertheless, the application of Flickr data in travel recommendation research is still interesting because it reflects tourists’ perception of temporal and spatial in real physical environments. The integration of more data sources will be our research focus, such as mobile phone signaling data, travel data, etc., which can make up for the challenges brought by the heterogeneity of tourists.

The proposed neural network model can provide personalized service in an environment that affects tourists’ travel preferences, which caters to the demand-oriented concept of smart tourism (Zheng and Liao 2019; Yang, Fik, and Zhang 2013). From personalized travel recommendation service to destination management, and then to crowd control, the proposed model offers broad application perspectives (Kang and Gretzel 2012; Mckercher and Lau 2008). Its most straightforward application is to optimize tourists’ travel experience. Personalized suggestions will be made available to specific tourists along the way, highlighting the optional attractions recommended by the model in the next tourist area. It can also predict the geographic distribution of future tourists. Tourism communities can make targeted adjustments to infrastructure and services to improve the adhesion and loyalty of tourists, which is conducive to the transformation of tourism communities from the construction and management of a single attraction to the development of a comprehensive destination. Moreover, it can also reveal the potentially crowded areas, and provide early warning information for those recommended to these areas. This is in line with the operating laws of tourism market and is very important for the sustainable development of tourism.

6. Conclusions and future work

Understanding tourists’ travel behavior and recommending attractions or itineraries have become a hotspot for research in recent years (Yang, Fik, and Zhang 2013; Wong and McKercher 2012; Yeh and Cheng 2015). In this article, we present a neural network model to understand the potential movement patterns from tourists’ travel trajectories and recommend personalized travel routes. LSTMs are used to learn sequential behaviors from trajectory-encoded data, combined with a temporal attention mechanism to capture tourists’ complex behavioural preferences. An offline experiment is designed from geotagged photos to assess the model performance. Results show that our model has higher recommendation performance than other baselines. Besides, we further explore the influencing factors of the model and analyze the tourists’ travel patterns presented by the model. In short, this tourist recommendation method based on neural network may provide valuable insights for smart tourism research.

In the future, there is still much to explore. First, more Flickr datasets will be applied to validate model performance. And other travel information will be reviewed from the geotagged photo, such as social relations and demographic characteristics of tourists (Al-Shamri 2016; Xiang and Gretzel 2010). This information has been proven to be a significant element in improving the quality of travel recommendations. Second, statistical tests (such as Friedman testing, Nemenyi test) also need to be formed and further analyzed, which is crucial for model performance exploration. Finally, studying the interpretability of neural networks in the travel recommendation process is also a focus of future work, which is important for the promotion of neural network models in the tourism industry.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are openly available in figshare at https://figshare.com/s/eb75e27d6baa568962df.

Additional information

Funding

This research was supported in part by the National Natural Science Foundation of China (42171460), and the Open Fund of Henan Key Laboratory for Synergistic Prevention of Water and Soil Environmental Pollution, Xinyang Normal University (KLSPWSEP-A09).