A reinforcement learning based dynamic room pricing model for hotel industry

Figures & data

Figure 1. Basic components of RL: environment, state, action, agent, and reward.

Table 1. Key notations for the Q-Learning algorithm.

$\delta$	State space
$A(s)$	Action space
$Q(s_t,a_t)$	Current Q value
$Q^{\mathit{\text{new}}}(s_t,a_t)$	New Q value
$\alpha$	Learning rate
$r_t$	Reward
$\gamma$	Discount rate
$\mathrm{\text{max}}Q(s_{t+1},a)$	Maximum future Q value

Figure 2. Visualization of the Q-Table.

Table 2. Feature categories and the number of features in each category.

	Feature categories	# of features
1	Activities	81
2	Spa and health	49
3	Facility service and features	43
4	Pool and beach	35
5	Food and beverage	32

Figure 3. Total room based demand of the all hotels by year & month.

Figure 4. Reward rate distribution.

Figure 5. Profit rate distribution.

Table 3. Hyperparameter space for the proposed pricing model and their candidate values.

	Hyperparameter	Values
1	LEARNING RATE	{0.01, 0.05, 0.1, 0.2}
2	EPSILON	{0.1, 0.2, 0.3, …, 0.8, 0.9}
3	EPSILON DECAY	{0.9, 0.99, 0.998}
4	STATE SPACE SIZE	{18, 30, 36, 50, 60, 100}
5	INITIAL_PROFIT	{rand(0,0.20), rand(0,0.15), rand(0,0.10)}
6	LOW_PENALTY	{−10, −5, −1}
7	HIGH_PENALTY	{−400, −300, −200, −100}
8	REWARD	{10, 25, 50}

Figure 6. Offline evaluation for proposed reward function $r_{i,t}$ with different state space size.

Figure 7. (a, c, e) Offline evaluation for reward function $r_t\mathrm{(}\text{profit rate}\mathrm{)}$ with different state space size. (b, d, f) Offline evaluation for reward function $r_{i,t}\mathrm{(}\text{rct difference}\mathrm{)}$ with different state space size.

Table 4. MAPE Results of test sets representing different time periods.

Test data period	MAPE
Last record	15.15
July	12.68
August	13.95
Septembler	18.12

Table 5. Results comparison with the most dense region: ‘‘Side/antalya/Turkey’.

Test data period	MAPE
All Hotels	15.15
Only ‘‘Side’	14.81
‘‘Side’ in All	13.43