Treatment experience, burden and unmet needs (TRIBUNE) in MS study: results from Turkey

Abstract

Objective:

To estimate the economic burden of multiple sclerosis (MS) in Turkey, including the relapses and disease severity, and to evaluate the quality-of-life of MS patients.

Methods:

The Treatment Experience, Burden and Unmet Needs (TRIBUNE) study was a multi-national, cross-sectional, retrospective, burden-of-illness survey. Total costs were calculated using unit costs derived from price lists or published literature, where relevant, and inflated to 2011 TL prices.

Results:

A total of 295 MS patients (74% females) were included in the analysis. The population had a mean age of 36 years; 73% had the relapsing–remitting form. Mean Expanded Disability Status Scale (EDSS) score was 2.2. Twenty-two per cent of the MS patients required hospitalization in the past year and spent an average of 29.2 days/year in hospital. These values were 43% and 5.6 days for the outpatients, respectively. Total cost per patient/year was 18,700 TL (Turkish Lira). Total costs for patients with mild, moderate, and severe disability were 15,418 TL, 26,002 TL, and 44,208 TL per patient/year, respectively. The mean EuroQol 5D scores in the same groups were 0.73, 0.52, and 0.05, respectively.

Conclusions:

Multiple sclerosis imposes a significant economic burden on patients and society in Turkey.

Keywords::

• Direct cost
• Indirect cost
• Economic burden
• Quality-of-life
• Multiple sclerosis
• Turkey

Introduction

Multiple sclerosis (MS) is a chronic debilitating disease characterized by inflammatory demyelination within the central nervous system1. Loss of myelin disrupts the nerves’ ability to conduct electrical impulses to and from the brain, triggering a variety of symptoms, including fatigue, visual disturbances secondary to optic neuritis, and sensory disturbances such as paresthesias or hypoesthesia1. Spasticity of the limbs, urinary incontinence, sexual problems, fatigue, depression, and mild emotional or intellectual changes may also develop over time1.

For many years, the diagnosis was based exclusively on the clinical history and the physical examination. Progressively, what have been termed para-clinical and laboratory tests have been increasingly used, culminating in magnetic resonance imaging (MRI). Diagnostic criteria for MS include clinical and para-clinical laboratory assessments emphasizing the need to demonstrate dissemination of lesions in space and time and to exclude alternative diagnoses. Although the diagnosis can be made on clinical grounds alone, MRI of the central nervous system can support the clinical criteria, as most recently emphasized by the so-called McDonald Criteria of the International Panel on Diagnosis of MS2^,3. The McDonald Criteria resulted in earlier diagnosis of the disease with a high degree of specificity and sensitivity.

Multiple sclerosis is a typical CD4 T cell-mediated autoimmune disease of the central nervous system that leads to inflammation, demyelination, axonal damage, glial scarring, and a broad range of neurological deficits. The prevalence of MS follows a latitudinal gradient in an incomplete distribution model, and varies from less than 5/100,000 in low-risk areas (e.g., most of Africa and Eastern Asia), to more than 100/100,000 in high-risk areas (e.g., northern and central Europe, North America, and Southeastern Australia)4. The majority of MS cases (approximately two thirds) occur in young adults between the ages of 20–40, with incidence peaking between the ages of 30–35.3. Females are 2–3-times more likely to develop MS than males4. MS is a chronic and disabiling disease. However, despite their disability, most people with MS have a normal life span5.

Although the clinical course of the disease is highly variable, four disease types can be recognized6: Relapsing–remitting MS (RRMS) is characterized by clearly defined attacks of new or recurrent neurologic symptoms and signs with full or partial recovery and lack of disease progression between disease relapses. This type accounts for ∼80–85% of initial diagnoses of MS. Primary–progressive MS (PPMS) is characterized by disease progression from onset with occasional plateaus and temporary minor improvements. Approximately 10–15% of MS patients have PPMS. Secondary–progressive MS (SPMS) has a typical initial RR disease course followed by progression with or without occasional relapses, minor remissions and plateaus. Finally, progressive–relapsing MS (PRMS) is characterized by progressive disease from onset, with clear acute relapses, with or without full recovery.

MS imposes considerable cost on individuals, families, the healthcare system, and society. A study by Prescott et al.7 reported the total annual cost in 2004 for MS-related medical care as $12,879 per patient/episode, with 64.8% of the total charges attributable to prescription drugs. With respect to the clinical factors that affect MS costs, research has shown that indirect as well as direct costs increase exponentially as disability progresses. On the standard measure of progressive disability known as the Expanded Disability Status Scale (EDSS), moderate limitations increased costs by factors of 1.15–3.43 and losing the ability to walk without aid or a wheelchair increased costs by 3.86–7.46-times the cost of patients with mild disabilities7. Exacerbations of MS symptoms, referred to as relapses, also substantially increase the cost of medical care.

To date, measurement of health utility in MS studies are limited. Fisk et al.8 compared three generic measures of health utility (EQ-5D, HUI Mark III, SF-6D) for the evaluation of MS treatments and found that HUI Mark III could be the most appropriate measure. In a study evaluating the quality-of-life along with costs in nine European countries, utility was found to be similar across countries at ∼0.70 for a patient with an EDSS of 2.0 and ∼0.45 for a patient with an EDSS of 6.59.

Türk Börü et al.10 investigated the prevalence of MS in the district of Maltepe, Istanbul, Turkey. In this cross-sectional field research study using door-to-door sampling, 33 of 32,531 people were diagnosed with MS. The prevalence of MS was 101.4/100,000 within the total population (118.7/100,000 in the female and 76.0/100,000 in the male populations). A survey in three rural areas of coastal Black Sea regions of Turkey reported a prevalence of 51/100,000 with average ages 30.5 years at onset and 39.1 years at diagnosis11. In a recent report, the incidence of the disease was stated as 1.74/100,000 and female/male ratio as 2.312. The mean age reported was 33 for both females and males13. Clinical features and course were typical of European MS indicating that Turkey is a high-risk MS area, similar to most regions of Mediterranean Europe.

The objective of the Treatment Experience, Burden and Unmet Needs (TRIBUNE) study was to estimate the economic burden of MS in Turkey including relapses and disease severity and to evaluate the health state utilities of the MS patients.

Patients and methods

The TRIBUNE study was a multi-national, cross-sectional, retrospective, burden-of-illness survey. Data were collected from MS patients from eight countries (Canada, France, Germany, Italy, Netherlands, Spain, Turkey, and the UK). This article presents the results for Turkey.

Study design

Patients identified from treatment centers were invited to participate in the study if they were at least 18 years old and had clinically definite MS (International Classification of Diseases, Tenth Revision [ICD-10] G35)14. Patients were excluded if they had any physical or mental illness leading to their inadequate participation in the study or were currently enrolled in a clinical trial. All participants gave informed consent and the study was approved by Ege University Faculty of Medicine Ethics Committee.

Patient evaluation

Patients self-completed a questionnaire which is comprised of items regarding demographics, disease information, treatment information, resource consumption, utilities, and validated questionnaires on health-related quality-of-life and fatigue. Disease information was obtained via the questions on the year of diagnosis, year of first symptoms, and type of MS.

Disability was evaluated using the EDSS. The version of the scale was patient-rated and ranged from 0–9 in 0.5 unit increments, with higher scores representing severe disability15^,16. Co-morbidities including arthritis, osteoporosis, cognitive impairment, depression/anxiety, sleep abnormalities, hypertension, and urinary tract infections/incontinence were also evaluated. Data on the number of relapses experienced during the previous 12 months were reported. A relapse (or exacerbation) of MS was defined as the development of new symptoms or worsening of old symptoms that last longer than 48 hours. In a relapse, MS symptoms generally worsen over a period of a few days to several weeks. Symptoms then disappear partially or completely for several weeks, months, or even years. During a relapse, several different symptoms can get worse at the same time.

Patient health state utilities were evaluated via the EuroQol 5D (EQ-5D). The EQ-5D is a generic preference-based measure of HRQL that consists of two parts. The first part includes five separate domains; mobility, capacity for self-care, conduct of usual activities, pain/discomfort, and anxiety/depression. Each domain has three levels: no problems, some problems, extreme problems. The second part consists of a Visual Analog Scale (EQVAS) to measure self-perceived health on a vertical scale from 0–100, where 0 is the worst imaginable health state and 10 is the best imaginable health state. The EQ-5D defines 243 health states, and has a range from −0.6–1.017.

Cost calculation for Turkey

The total costs of MS for Turkey were calculated using unit costs derived from price lists or published literature, where relevant, and inflated to 2011 TL prices (10.45% as reported by the Turkish Statistical Institute). The cost per day for inpatient care in hospitals and for medical procedures were taken from the Turkey Public Expenditure Review (report no: 36764-TR)18 and Social Security Institution. The cost per outpatient visit in hospitals, rehabilitation centers, and nursing homes were collected through World Health Organization (WHO) estimates of unit costs for patient services for Turkey19. Cost per visit for physicians, specialists, and other healthcare professionals were obtained from WHO estimates of unit costs for patient services for Turkey and the Turkey Public Expenditure Review (report no: 36764-TR)18. Both for MS and other prescribed medications, the costs were obtained from the Ministry of Health. Patients’ estimations were used for costs for OTC medications. Productivity losses of the patients and informal caregivers were calculated from work due to MS with the average gross salary per hour in Turkey20.

Analysis

Patients’ demographics and disease characteristics, as well as resource utilization and costs, were analyzed using descriptive statistics (percentage, mean, and standard deviation). Although the data are skewed, mean values are presented as the mean values have been bootstrapped. The median values give information about the sample in a study, but bootstrapping makes mean values represent better the true mean value in the population21.

Confidence intervals (95%) were estimated by non-parametric bootstrapping. Independent samples were analyzed using Mann-Whitney non-parametric test to determine whether differences between comparisons were statistically significant (p value <0.05). All calculations were done using STATA version.9 software.

Results

Patient demographic and disease information

A total of 295 MS patients (74% females) were included in the analysis. The population had a mean age of 36 years (a mean age of 28 years at first symptoms and 30 years at diagnosis). Of these, 43% of the patients were employed, 2% retired due to MS, and 29% used an informal caregiver. Nearly three out of four patients (73%) had the relapsing–remitting form of the disease (RRMS). The percentages of patients with secondary-progressive disease (SPMS) and primary-progressive disease (PPMS) were 12% and 4%, respectively. The mean EDSS score of the patient population was 2.2. The majority (74%) had mild disability (EDSS ≤3), 23% had moderate (EDSS score 4–6.5), and 3% had severe (EDSS score 7–9) disability due to MS (Table 1).

Table 1. Demographic and disease characteristics of the study population.

Patient characteristics	n	%	Mean (SD)
Number of patients	295	100%
Female	217	73.6%
Age			36.0 (9.1)
Employed	123	41.7%
Retired due to MS	7	2.4%
Have informal caregiver	86	29.2%
MS type
RRMS	214	72.5%
SPMS and PPMS	47	15.9%
Don’t know	34	11.5%
Onset of MS
Age at diagnosis; mean (SD)			30.0 (8.5)
Age at first symptoms; mean (SD)			27.7 (8.5)
Severity of MS
EDSS score; mean (SD)			2.2 (2.1)
Mild (EDSS score 0–3)	219	74.2%
Moderate (EDSS score 4–6.5)	67	22.7%
Severe (EDSS score 7–9)	9	3.1%
Relapses in previous 12 months
No	157	53.2%
Yes	120	40.7%
Treated with steroids	105	35.6%
Hospitalized	60	20.3%

MS, Multiple sclerosis; RRMS, relapsing–remitting multiple sclerosis; SD, standard deviation; SPMS, secondary-progressive disease; PPMS, primary-progressive multiple sclerosis; EDSS, Expanded Disability Status Scale.

Resource utilization

Inpatient care data revealed that 22% of patients required hospitalization in the past year and spent an average of 29.2 days per year in hospital, leading to a cost of 1577.35 TL per year. These values were 43% and 5.6 days for the outpatients, respectively. In total, 48% of patients reported visiting a neurologist (4.5 visits per year) in the past year, and 62% received a MRI scan. Consultations with neurologists led to a cost of 166.56 TL per year. The majority of the patients (83%) were receiving disease-modifying MS therapy (e.g., interferon beta-1a im, interferon beta-1a sc, interferon beta-1b, glatiramer acetate, and natalizumab). The proportion of patients using over-the-counter (OTC) medications was 9.5%. Most frequently prescribed co-medications were 29% for pain, 29.5% for depression, and 8.5% for fatigue. Twenty-nine per cent of the patients used informal care (350.7 h per year), with a cost of 1220.11 TL per year (Table 2).

Table 2. Resource utilization of the study population.

	Number (%) of patients	Mean value		Mean cost (TL/year)
Inpatient care	admitted within 12 months			1641.41
Hospital	64 (21.7)	29.21	day/year	1577.35
Rehabilitation center	14 (4.8)	5.78	day/year	63.54
Nursing home	2 (0.7)	0.05	day/year	0.52
Outpatient care	admitted within 3 months			263.89
Hospital	126 (42.7)	5.56	visit/year	205.69
Rehabilitation center	11 (3.7)	1.51	visit/year	55.69
Nursing home	1 (0.3)	0.07	visit/year	2.51
Consultations	consulted within 3 months			393.32
General practitioner	5 (1.7)	0.11	visit/year	4.01
Neurologist	142 (48.1)	4.50	visit/year	166.56
Other specialist	24 (8.1)	1.25	visit/year	46.16
Nurse	5 (1.7)	0.81	visit/year	30.10
Physiotherapist	25 (8.5)	2.74	visit/year	101.34
Psychologist	10 (3.4)	0.22	visit/year	8.03
Occupational therapist	5 (1.7)	0.18	visit/year	6.52
Optician	24 (8.1)	0.83	visit/year	30.60
Investigations	tested within 12 months			109.61
MRI	183 (62.0)	0.62	test/year	102.72
CT	14 (4.8)	0.05	test/year	5.37
Spinal tap	7 (2.4)	0.02	test/year	1.52
MS treatments	treated within 3 months
	244 (82.7)	–		–
Prescribed co-medications	treated within 1 month			293.71
Antidepressant	86 (29.2)	100.03	day treated/year	62.02
Pain medication	87 (29.5)	52.11	day treated/year	71.39
Fatigue medication	25 (8.5)	26.52	day treated/year	51.98
OTC medications	28 (9.5)	108.38	day treated/year	108.32
Utensils and modifications	with investments within 12 months			792.64
Home modifications	11 (3.7)	–		422.54
Car modifications	4 (1.4)	–		143.76
Walking stick	24 (8.1)	–		18.79
Wheelchair	12 (4.1)	–		207.55
Assistance	with assistance within 3 months			2407.78
Nurse	5 (1.7)	1.06	hour/year	15.91
Home help	16 (5.4)	65.05	hour/year	613.38
Professional help	5 (1.7)	59.21	hour/year	558.38
Informal care	86 (29.2)	350.68	hour/year	1220.11
Indirect costs	with indirect costs within 3 months			846.03
Sick leave	37 (12.5)	6.18	day/year	445.18
Early retirement due to MS	7 (2.4)	44.54	hour/year	400.85

MRI, Magnetic resonance imaging; CT, computerized tomography; OTC, over-the-counter; MS, multiple sclerosis.

Costs

Total cost per patient/year was 18,700 TL. As shown in Table 3, the cost for MS treatments was the highest among the direct medical costs (11,294 TL per patient/year), which was followed by the cost for inpatient care (1641 TL per patient/year). Costs per patient/year increased with disease severity. Total costs for patients with mild, moderate, and severe disability were 15,418 TL, 26,002 TL, and 44,208 TL per patient/year, respectively. Although costs for most of the direct medical care items were markedly higher in the severely disabled patients compared to those with mild and moderate disability, the cost for MS treatments was highest in those with moderate disability (Tables 3 and 4). The costs of relapses for those requiring hospitalization and not requiring hospitalization were 9,095 TL and 2,764 TL, respectively (data not shown). p-values were not calculated, since the purpose of the analysis is to describe the cost of disease in general and in sub-groups, not just to make inferences on the ‘statistically significant’ difference. Readers are expected to make their own comments on the ‘clinical significance’.

Table 3. Direct medical, direct non-medical, and indirect costs of the patients with mild, moderate, and severe disability (TL per year; 2011 year costs).

	Mild (n = 219)	Moderate (n = 67)	Severe (n = 9)	Total (n = 295)
Direct medical
Inpatient care	1359	2117	4960	1641
Outpatient care	218	376	543	264
Consultations	398	349	608	393
Investigations	111	111	63	110
MS treatments	10,869	13,572	4683	11,294
Other treatments	671	1551	3324	952
Direct non-medical
Utensils and modifications	412	1208	6950	793
Assistance	872	5076	19,919	2408
Indirect
Sick leave	276	885	1280	445
Retirement due to MS	231	756	1877	400
Total	15,418	26,002	44,208	18,700

MS, Multiple sclerosis.

Table 4. Direct medical, direct non-medical, and indirect costs of the patients with each EDSS score category (TL per year; 2011 year costs).

	EDSS 0 (n = 64)		EDSS 1 (n = 74)		EDSS 2 (n = 56)		EDSS 3 (n = 25)		EDSS 4 (n = 29)		EDSS 5 (n = 18)		EDSS 6 (n = 16)		EDSS 6.5 (n = 4)		EDSS 7 (n = 1)		EDSS 8 (n = 6)		EDSS 9 (n = 2)
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Medical care
Inpatient care	883	3297	1281	3951	1258	3695	3032	6656	1586	4285	645	1599	4613	6673	2619	4353	0		7441	8220	0	0
Outpatient care	215	600	122	233	238	496	468	851	418	627	271	490	453	1095	222	352	148		493	1208	888	1256
Consultations	229	602	552	3620	341	632	503	1014	347	462	181	165	592	1500	148	121	148		740	1813	444	628
Investigations	82	88	129	147	121	181	109	104	118	104	98	92	111	94	117	165	0		59	98	108	153
MS treatments	9391	6673	11,283	6113	11,940	5819	11,025	6869	13,955	15,098	17,961	18,067	9428	6960	7623	9090	0		5082	8067	5826	8239
Other treatments	582	1876	490	1349	1018	2570	660	1561	1238	2234	1744	2596	2258	3413	111	202	8554		3044	3835	1549	1106
Non-medical care
Utensils and modifications	520	2364	350	1881	4	31	1236	3454	407	2102	2538	7178	1424	3758	173	116	3266		7410	13,017	7415	5867
Assistance	385	1252	292	1996	688	2859	4244	11,209	3009	8167	4520	6903	9797	28,565	3688	7376	16,893		27,064	32,939	0	0
Indirect costs	104	408	428	1552	698	2430	1351	4677	1400	3649	1979	4481	1056	4223	4223	8447	0		1920	4703	8447	11,945
Total costs	12,391	9361	14,928	9359	16,307	6664	22,629	19,129	22,478	19,751	29,937	21,306	29,733	30,524	18,924	23,865	29,009		53,252	54,274	24,676	10,503

In particular, the costs for utensils/modifications and professional care/informal care items showed dramatic fold increases in patients with severe disability (17-fold and 22-fold increases compared to patients with mild disability) (Table 5).

Table 5. Fold increases in costs for the study population with moderate and severe disability in comparison to those with mild disability.

	Mild (n = 219)	Moderate (n = 67)	Severe (n = 9)
Inpatient care	Ref	2×	4×
Consultations	Ref	1×	2×
Utensils/modifications	Ref	3×	17×
Professional care/informal care	Ref	6×	22×
Sick leave/retirement due to MS	Ref	3×	6×

Ref, reference; MS, Multiple sclerosis.

Health state utilities

The mean EQ-5D score of the study population was 0.66. In patients with mild disability, the mean EQ-5D score was 0.73, and in those with moderate and severe disability these values were 0.52 and 0.05, respectively. Patients with severe symptoms had decreased utility score. The mean VAS score of the study population was 68.1. The mean VAS score of 74.1 for patients with mild disability decreased to 53.9 and 44.7 in those with moderate and severe disability, respectively.

Discussion

The results of the TRIBUNE study demonstrate that MS imposes a significant economic burden on patients and society in Turkey. Total cost was estimated at 18,700 TL per patient/year. Disease severity had a negative impact on costs per patient/year and health state utilities of the patients.

The prevalence rate of MS in the Turkish population living in a district of Istanbul was found to be 101.4/100,000 within the total population, being much higher than neighboring countries10. In addition to environmental factors (e.g., geographical variation), longer survival, advanced diagnostic techniques, and increased access to healthcare services may have contributed to the increase in prevalence22. A recent study by Çelik et al.23 reported the rough prevalence of MS as 33.9 per 100,000 and annual incidence rate as 3.48 per 100,000 in the metropolitan area of Edirne city in Turkey. In the same study, the new MS patients in the Edirne city centers were re-screened and the incidence and prevalence rates were found to be 2.60 per 100,000 and 36.5 per 100,000, respectively. In a very recent report presenting MS results for 2012, the incidence of the disease was stated as 1.74/100,000 and female/male ratio as 2.3 in a total of 785 patients. According to this report, the average age of the patient population was 33 years. Black Sea region and Aegean region were the leading regions with highest disease incidence, as 3.04 and 2.98 per 100,000 cases, respectively12. Seventy-five per cent of these patients were diagnosed MS at the time of second attack or later. Among these patients, the rate of optic symptoms were higher in those at age ≤40 and cerebellar symptoms were higher in patients at age >40. Interesting findings of this report were that female patients had a significantly higher rate of sensory symptoms than males (p = 0.022)13.

The cost data of the TRIBUNE study provide important information on the economic burden of MS on the country. The study results showed that direct and indirect costs increase with disease severity. The costs for utensils/modifications increased by 17-fold and professional care/informal care increased by 22-fold in severely disabled for patients when compared to those with mild disability. According to the study data, the direct medical cost per patient/year was highest in patients with moderate disability. MS treatments make up the majority of direct medical costs and are highest in patients with mild and moderate forms of the disease. On the other hand, patients with severe disease had lower costs for MS treatments compared to mild and moderate groups, but had the highest inpatient costs. The direct non-medical costs increased in the same line with disease severity.

Costs actually vary among disease types. Costs are higher overall for patients with SPMS than for those with RRMS. However, when controlling for EDSS this difference is less clear; costs appear to be driven by the level of EDSS rather than by the type of MS. Costs increase with increasing severity of the disease, the level of EDSS being significantly correlated with costs. The mean cost per patient with severe disease (EDSS 7.0 and above) is 4–5 times higher than the cost of a patients with mild disease (up to EDSS 3.5)24. Therefore, patients with all disease types were included without any protocol-driven exclusion criteria, in order to make the study sample more representative of a real-life population.

A study conducted in nine European countries demonstrated that healthcare utilization varies across countries, reflecting differences in the organization of healthcare systems9. Comparison of the mean costs per patient with EDSS score 2.0 and 6.5 showed marked increases in direct costs of patients in the latter group in all countries. In patients with EDSS score of 6.5, the direct costs were highest in Italy (37,355 €) due to considerably high informal care costs (22,215 €). Except for UK and Belgium, in other countries inpatient care costs were higher than outpatient care costs, as observed in Turkish MS patients. In our study, informal care cost was found to be high with a level reaching that of the inpatient care costs. Informal care is generally a function of the extent of services offered by the healthcare systems. This may also explain the reason for the relatively high use of informal care in Turkish MS patients. Our study data also revealed that the direct costs exceed the indirect costs of MS. In our country, the rate of women workers is lower in comparison to that of socioeconomically developed countries; thus, this may lead to a lower rate of retirement due to disability and sick leave.

Another important finding of the study was that disease severity had a negative impact on the life quality of the study population. This finding was in the same parallel with the study by Kobelt et al.9, showing the strong correlation between disability and quality-of-life in European countries. In our study, mean EQ-5D score of patients with mild disability decreased by ∼16-fold in severely disabled patients (0.73 vs 0.05).

In conclusion, in line with the data of other TRIBUNE countries25, Turkey data estimated that economic burden of MS increases with disease-related disability which can be explained by a higher need for medical and non-medical care associated with progressing disability and that there is a decrease in mean utility scores as disease progresses, all of which suggest that slowing disease progression can reduce the patients burden of MS.

Transparency

Declaration of funding

The study has been sponsored by Novartis Pharmaceuticals.

Declaration of financial/other interests

Z. Caliskan is an employee of Novartis, Turkey. JME peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgments

The authors thank Professor Aksel Siva, Professor Celya Irkec, Professor Husnu Efendi, Associate Professor Muhtesem Gedizoglu, Professor Nur Yuceya and Associate Professor Nihal Isik (other TRIBUNE study members) for their kind contribution to the manuscript.