1,207
Views
0
CrossRef citations to date
0
Altmetric
Emerging Topics in Nutrition

Biomarkers of Zinc and Copper Status and Associated Factors in Outpatients with Ischemic and Non-Ischemic Heart Failure

ORCID Icon, ORCID Icon, ORCID Icon, ORCID Icon, ORCID Icon, ORCID Icon, ORCID Icon & ORCID Icon show all
Pages 231-239 | Received 18 Oct 2020, Accepted 14 Jan 2021, Published online: 11 Feb 2021

Abstract

Background: Few studies have explored the impact of ischemic and non-ischemic etiologies of heart failure and other factors associated with heart failure on zinc and copper status. This study examined zinc and copper status in 80 outpatients with ischemic (n = 36) and non-ischemic (n = 44) heart failure and associations with biodemographic, clinical, biochemical, and nutritional parameters.

Materials: Biomarkers of plasma zinc and copper, copper-zinc ratio, 24-h urinary zinc excretion, ceruloplasmin, and dietary intake of zinc and copper were assessed. Plasma zinc and copper and urinary zinc were measured by inductively coupled plasma mass spectrometry (ICP-MS).

Results: Patients with ischemic heart failure showed lower dietary zinc intake and higher dietary copper intake (both p = 0.02). Zinc and copper in plasma, copper-zinc ratio, ceruloplasmin, and 24-h urinary zinc excretion showed no statistical differences between the groups (all p ≥ 0.05). An inverse association was found between age (β =−0.001; p = 0.005) and the use of diuretics (β = -0.047; p = 0.013) and plasma zinc. Copper levels in plasma (β = 0.001; p < 0.001), and albumin (β = 0.090; p<0.001) were directly associated with plasma zinc. A positive association was found between ceruloplasmin (β = 0.011; p < 0.001), gamma-glutamyl transferase (β = 0.001; p < 0.001), albumin (β = 0.077; p = 0.001), and high-sensitivity c-reactive protein (β = 0.001; p = 0.024) and plasma copper.

Conclusion: Zinc and copper biomarkers in clinically stable patients with heart failure did not seem to be responsive to the differences in zinc and copper intake observed in this study, regardless of heart failure etiology. The predictors of plasma zinc and copper levels related to oxidative stress and inflammation should be monitored in heart failure clinical practice.

Introduction

Heart failure (HF) is a syndrome characterized by structural and/or functional cardiac abnormality, resulting in a reduced cardiac output and/or elevated intracardiac pressures at rest or during stress (Citation1,Citation2). Evidence supports the role of micronutrients in the development and progression of HF, notably zinc and copper (Citation3,Citation4). Zinc has antioxidant and anti-inflammatory properties (Citation5), while copper plays a role in cardiovascular physiology and acts as an antioxidant agent (Citation6,Citation7), meaning that the assessment of these elements is particularly important in the context of HF (Citation8). The leading causes of ischemic HF are coronary artery disease (CAD) and acute myocardial infarction (AMI), while the etiology of non-ischemic HF includes arterial hypertension, diabetes mellitus, Chagas disease, dilated cardiomyopathy, valvular heart disease, and congenital heart defects (Citation1,Citation9,Citation10).

Studies show that ischemia due to CAD and/or AMI contributes to a greater extent to the state of chronic inflammation and oxidative stress, leading to increased production of proinflammatory cytokines and free radicals, meaning that the homeostatic imbalance of zinc and copper may be higher in patients with ischemic HF than in those with non-ischemic HF (Citation11,Citation12). However, there are gaps in the literature on the impact of ischemic and non-ischemic etiologies on zinc and copper status in outpatients with HF, given that most studies focus on the link between these elements and advanced HF (Citation13). In view of the above, we hypothesized that outpatients with ischemic HF show lower zinc levels in plasma and higher copper levels in plasma, and consequently a higher copper-zinc ratio, due to the ischemic etiology of HF. In addition, we explored other factors associated with HF that could have an impact on the metabolism of these elements (Citation14–16). Thus, the objective of this study was to examine zinc and copper status in outpatients with ischemic and non-ischemic HF and associations with biodemographic, clinical, biochemical, and nutritional parameters.

Material and methods

Study population and design

A cross-sectional study was conducted at the Heart Failure Clinic at the Onofre Lopes University Hospital of the Federal University of Rio Grande do Norte. The study sample included patients with ages ranged between 21 to 83 years old with a diagnosis of HF, according to the Boston point system and the Framingham criteria (Citation17), confirmed by the Doppler echocardiogram. Adolescents, pregnant women, patients with chronic kidney disease undergoing hemodialysis, liver disease and/or thyroid disorders, cognitive impairment, and who had undergone bariatric surgery were not included. Patients who used vitamin-mineral supplements and/or oral contraceptives in the last 3 months were also excluded.

Participants were selected using non-probability sampling. All 123 patients who received treatment in the period April 2017 to November 2018 were considered eligible. Eighty-eight of these patients met the inclusion criteria and were invited to participate in the study, 4 of whom refused to take part. Of the 84 patients who agreed to participate in the study, 1 dropped out, 1 had HF of undefined etiology, and 2 died before blood collection, resulting in losses of 9%. Data was collected from a final sample of 80 patients stratified into two groups: “ischemic HF” (n = 36) and “non-ischemic HF” (n = 44). HF etiology was determined from the coronary angiography report. Patients with obstructive cardiac lesions and/or history of AMI were categorized as having ischemic HF and patients who did not show any signs of CAD and/or AMI were categorized into non-ischemic HF group.

The study was conducted in 3 stages. The first stage consisted of the selection and recruitment of participants. Participants were then asked to provide sociodemographic information and complete a first 24-hour dietary recall. In the second stage a biochemical assessment was performed and weight and height measurements were obtained for calculation of body mass index (BMI). Participants also provided information on medication use and completed a second 24-hour dietary recall. In the third stage, the participants received the results of the biochemical tests and completed a third 24-hour dietary recall.

Information on HF etiology, functional classification (New York Heart Association (NHYA)), heart rate, and associated comorbidities was taken from the patients’ electronic health records, while left ventricular ejection fraction (LVEF) and type of HF (<40% – heart failure with reduced ejection fraction (HFrEF); 40–49% – heart failure with mid-range ejection fraction (HFmrEF); ≥50% – heart failure with preserved ejection fraction (HFpEF) were determined from the Doppler echocardiography report.

The study was approved by the Onofre Lopes University the Hospital’s Research Ethics Committee (CAAE 59827516.2.0000.5292). All participants signed an informed consent form.

Assessment of dietary intake

Data on habitual food intake was obtained using three 24-hour dietary recalls performed at an interval of 30–45 days, in accordance to the recommendations published by Thompson and Byers (Citation18). The dietary interviews were conducted by registered nutritionists and graduate nutrition students trained at the Department of Nutrition of Federal University of Rio Grande do Norte. Photographs of utensils and containers were used to identify food-serving items and quantify serving sizes, the latter of which were classified as small, medium, or large. Dietary records were checked by trained nutritionists and details regarding recipes and portion sizes were noted and clarified with each participant. The quantity of each food item and drink was converted into grams or milliliters using a measurement chart for food consumed in Brazil. The foods were converted into energy and nutrients using the software Virtual Nutri Plus® 2.0 (São Paulo, Brazil). New preparations and foods were added to the software database as necessary, along with their nutrient composition taken from the Brazilian food composition table and the U.S. Department of Agriculture (USDA) database, as appropriate. Nutritional information taken from industrial food labels was also added to the software database.

The Multiple Source Method (MSM – https://msm.dife.de/) was used to measure usual nutrient intake and correct for within-individual variance (Citation19). Nutrient intakes were then adjusted for total energy intake using the method proposed by Willet and Stampfer (Citation20). The prevalence of inadequate zinc and copper intake was estimated using the estimated average requirements (EAR) cut-point method (Citation21,Citation22).

Biochemical analysis

Blood samples were collected in the morning after an overnight fast (12 hours). Fasting blood glucose, total cholesterol, triglyceride levels, aspartate aminotransferase (ATT), alanine aminotransferase (ALT), and gamma glutamyl transferase (GGT) were determined using enzymatic analysis. Levels of high-density lipoprotein (HDL-c), albumin, transferrin, and transferrin saturation were measured using colorimetric methods. Low-density lipoprotein (LDL-c) values were calculated using the Friedewald formula [LDL-c = total cholesterol – HDL-c + (triglycerides/5)], while non-high-density lipoprotein (non-HDL-c) was determined considering the difference between total cholesterol and HDL-c, as recommended by the Brazilian Society of Cardiology (Citation23). Glycated hemoglobin (HbA1c) and high-sensitivity c-reactive protein (hs-CRP) were analyzed by immunoturbidimetry. Urea and creatinine were measured using the UV kinetic and kinetic methods, respectively. The estimated glomerular filtration rate (eGFR) was calculated using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation, according to the definitions proposed by the International Society of Nephrology (Citation24), where low eGFR was considered to be <60 mL/min/1.73m2. All analyses were performed using the CMD 800i X1 Chemistry Analyzer (Diamond Diagnostics®, Holliston, Massachusetts, USA) with Wiener lab® kits (Wiener® Lab Group, Argentina). Ferritin was determined by electrochemiluminescence using the Cobas® 6000 e601 module (Roche® Professional Diagnostics, Risch-Rotkreuz, Switzerland).

The hemoglobin and hematocrit tests were performed using the cyanide-free colorimetric method and microtechnique, respectively, with the Advia® 2120i Hematology System hematology analyzer (Siemens Healthcare®, Erlangen, Germany). Ceruloplasmin levels were quantified in a private laboratory (Natal/Rio Grande do Norte) by turbidimetry using the Beckman Coulter® AU5800 clinical chemistry analyzer and test kits (Brea, California, USA). The cutoff values for ceruloplasmin were 18–45 mg/dL.

Analysis of the biomarkers of zinc and copper

All materials used were cleaned by soaking in 20% v/v nitric acid solution for at least 12 hours, and rinsing six times with Milli-Q water (Millipore Direct-Q® 3 UV Water Purification System, Bedford, MA, USA). Blood samples were collected in tubes containing 30% sodium citrate anticoagulant solution. Plasma samples were separated by centrifugation (900 g, 10 minutes) and storage at −20 °C in 2 mL polypropylene conical tubes (Eppendorf™, Germany), until analysis. Urinary zinc excretion was measured using 24 hours urine samples. A fraction of the collected urine samples was then storage at −20 °C in a polypropylene tube until analysis.

Zinc and copper levels in plasma and urinary zinc were measured by inductively coupled plasma mass spectrometry (ICP-MS) by using the NexION® 2000 ICP-MS (PerkinElmer Instruments®), following the methodology described by Batista et al. (Citation25) and Batista et al. (Citation26), respectively. The intra-assay coefficients of variation of zinc and copper in plasma were 1.6% and 1.3%, respectively, while for urinary zinc the intra-assay coefficient was 2.7%. In order to verify the accuracy and precision of the proposed method, Reference Materials of human blood from the L’Institut National de Santé Publique du Québec (Canada), and a Standard Reference Material of human urine (SRM 2670a) from the National Institute of Standards and Technology (NIST) (USA) were analyzed. All found values are in good agreement with target values (t test, 95%). The following reference values were adopted: plasma zinc (70–110 μg/dL) (Citation27); 24-h urinary zinc excretion (300–600 μg/24 h) (Citation28); and plasma copper (70–140 μg/dL) (Citation28).

Statistical analysis

The sample calculation was performed using the G Power software, version 3.1.9.2 (available at: http://www.gpower.hhu.de/). The objective was to measure the post-hoc power of Student's t-test for two independent samples, considering the values of plasma zinc for the calculation. Thus, our study had a statistical power of 14.88% to detect a plasma zinc level difference of 4 μg/dL (or more) between the groups.

The continuous variables were expressed as means (standard deviation) for data with a normal distribution and medians (1st quartile - 3rd quartile) for skewed data. The categorical variables were expressed as absolute frequencies (percentage frequency). Data distribution was evaluated using the Shapiro-Wilk test. Missing data was observed for the following variables: 24-h urinary zinc excretion, ceruloplasmin, creatinine, eGFR, albumin, LDL-c, triglycerides, ferritin, GGT, fasting blood glucose, HbA1c, hs-CRP, and transferrin saturation. The missing data for these variables were imputed by single imputation, which replaces missing values with plausible values to complete the data set. The variables 24-h urinary zinc excretion and ceruloplasmin were not imputed because the percentage of missing values exceeded the limit set by the method. The variables were compared between the patient groups (patients with ischemic HF and patients with non-ischemic HF) using Student’s t- test, the Mann–Whitney test, chi-squared test, or Fisher’s exact test, accordingly.

Two models were used to determine associations between plasma zinc and copper and the biodemographic, clinical, biochemical, and nutritional variables, using plasma zinc and copper, respectively, as the dependent variables. These variables were tested for data normality and log-transformation was performed reduce skewness. In the final models, the independent variables were selected using the stepwise method. Residual scatter plots were built to determine whether the data satisfied the Gauss-Markov conditions, which measures the variance and behavior of the residuals. The Kolmogorov-Smirnov test was performed to test the normality of the residuals. Multiple linear regression models were run with the respective β coefficients, standard errors, 95% confidence intervals (CI), and p-values, adopting a significance level of 5% (p < 0.05). The analyses were performed using the statistical software packages R version 3.6.0 and SPSS version 25.0 (Statistical Package for the Social Sciences, Chicago, USA).

Results

Both patient groups were predominantly male. The average age of the ischemic group was greater than that of the non-ischemic group (p < 0.01). The most frequent type of HF was HFrEF (53.8%) and most patients were in NYHA class I/II (92.5%), without any significant difference between the groups (both p ≥ 0.05). Frequency of arterial hypertension, diabetes mellitus, and use of medication (hypoglycemic, hypolipidemic, and antiplatelet agents) was greater in patients with ischemic HF than in non-ischemic patients (all p < 0.05). Significant differences between the two groups were found for proteins and total fat intake (both p < 0.05) ().

Table 1. Biodemographic, Clinical and Dietary Characteristics of the Patients with Ischemic and Non-Ischemic Heart Failure Groups.

Differences in LDL-c levels were found between the groups (75 (54–97) mg/dL in patients with ischemic HF and 99 (76–121) mg/dL in patients with non-ischemic HF) (p < 0.01); however, there were no statistically significant differences between the groups for the other biochemical variables (all p ≥ 0.05) ().

Table 2. Biochemical Parameters in Patients with Ischemic and Non-Ischemic Heart Failure Groups.

Patients with ischemic HF showed a statistically significant lower dietary intake of zinc (6.76 (6.02–7.91) mg/d), and higher copper intake (0.96 (0.77–1.17) mg/d) (both p = 0.02) than patients with non-ischemic HF (zinc intake 8.44 (6.76–9.63) mg/d; copper intake 0.83 (0.68–0.96) mg/d) (). Prevalence of inadequate zinc intake was higher in the ischemic group (71% in women and 82% in men), while prevalence of inadequate copper intake was higher in the non-ischemic group (54% in women and 17% in men).

Table 3. Biomarkers of zinc and copper status in patients with ischemic and non-ischemic heart failure groups.

There were no significant differences between the groups in zinc and copper levels in plasma, copper-zinc ratio, 24-h urinary zinc excretion, and ceruloplasmin (all p ≥ 0.05). The median values of plasma zinc and copper, 24-h urinary zinc excretion, and ceruloplasmin were in line with the reference values in both groups (). Plasma zinc values of less than 70 µg/dL were found in 17.5% of participants, while plasma copper values of over 140 µg/dL and 24-h urinary zinc excretion values of over 600 µg/24 h were found in 20% of patients.

The results of the regression analysis indicated inverse associations between age (β = −0.001; p = 0.005) and the use of diuretics (β = −0.047; p = 0.013) and plasma zinc. Conversely, transferrin saturation (β = 0.002; p = 0.014), plasma copper (β = 0.001; p < 0.001), albumin (β = 0.090; p < 0.001), and ischemic etiology of HF (β = 0.038; p = 0.012) were directly associated with plasma zinc (). The results also confirmed a direct association between ceruloplasmin (β = 0.011; p < 0.001), GGT (β = 0.001; p < 0.001), albumin (β = 0.077; p = 0.001), hs-CRP (β = 0.001; p = 0.024), and dietary calcium (β = 0.00015; p = 0.015) and plasma copper. In contrast, the independent variables ex-drinker (β = −0.070; p < 0.001) and dietary fiber (β = −0.016; p = 0,008) were inversely associated with plasma copper ().

Table 4. Multiple linear regression models with dependent variables plasma zinc and copper in patients with heart failure.

Discussion

To the best of our knowledge, this is the first study to evaluate zinc and copper status in outpatients with ischemic HF and non-ischemic HF considering a set of biomarkers. Although we observed differences in dietary zinc and copper intake between the groups, the median values for the biomarkers of zinc and copper status were in line with the reference values, with no statistically significant differences between the groups. Another important finding of this study were the associations between biodemographic, clinical, biochemical, and dietary variables and zinc and copper levels, suggesting that they may be potential predictors of the status of these elements in patients with HF.

The higher prevalence of inadequate zinc intake in the ischemic group may be explained by stricter dietary guidance to restrict foods sources of this micronutrient, particularly those of animal origin. One of the key recommendations for the treatment of CAD is limiting saturated fats (Citation23), which come mainly from animal sources, consequently reducing the consumption of important food sources of zinc. The prevalence of inadequate zinc intake observed in the present study was higher than that reported by Lourenço et al. (Citation29) (42.1% in women and 38.2% in men) in a study with patients with HF. Evidence of the consequences of inadequate and excess intake of copper in patients with HF is scarce and conflicting (Citation30,Citation31). The fact that copper intake was higher in patients with ischemic HF may be attributed to higher consumption of wholegrain cereals observed in the 24-hour recalls of the present study (data not shown).

We failed to confirm the hypothesis that patients with ischemic HF show lower zinc levels in plasma and higher copper levels in plasma, and consequently a higher copper-zinc ratio, due to the ischemic etiology of HF. Although the patients with ischemic HF were older than the non-ischemic group and the prevalence of diabetes mellitus, arterial hypertension, and use of medication was greater in this group, the findings suggest that pathophysiologic changes associated with HF and clinical stability may be determinants of zinc and copper status, regardless of etiology and other clinical conditions.

Unlike our findings, a number of studies assessing zinc and copper status in patients with HF point to a reduction in zinc levels in plasma and increased copper levels in plasma (Citation13,Citation32,Citation33). Conversely, a study assessing plasma zinc, iron, and selenium in Brazilian patients with HF reported that zinc levels were higher in most patients (Citation34). A meta-analysis of zinc status in patients with ischemic and idiopathic HF (Citation35) showed lower zinc levels in plasma in patients with idiopathic HF compared to controls. Also, no difference in zinc status was observed between the ischemic group and control group.

A number of factors may explain the values found for zinc and copper levels in plasma in patients with ischemic and non-ischemic HF: (Citation1) the sample comprised clinically stable outpatients, most of whom were in NYHA class I/II. Patients with HF have reduced intake and lower status of a number of micronutrients, with the exception of copper status, which tends to be elevated, mainly in patients who were in NYHA class III/IV. Generally, these patients have decreased sensation of hunger, diet restrictions, fatigue, shortness of breath, nausea, anxiety, and sadness, decreased absorption as a consequence of gut edema, and increased urinary losses owing to drug therapy. These factors can contribute to the reduction of nutritional intake in patients with HF and compromise the status of micronutrients (Citation36); (Citation2) the small number of older persons in the study sample (Citation37); (Citation3) the low prevalence of abnormal albumin and hs-CRP levels (Citation37); and (Citation4) the rigid homeostatic control of these elements, especially zinc (Citation16,Citation38).

International expert committees have indicated 24-h urinary zinc excretion as a potential zinc biomarker, even though pathophysiologic conditions associated with increased protein catabolism are linked to increased zinc excretion (Citation39). Unfortunately, this biomarker has been little used in studies with patients with HF, limiting the comparability of our findings. Our findings are therefore particularly interesting because they show that both groups of patients had normal values of urinary zinc excretion, indicating good homeostatic control and, consequently, maintenance of zinc concentrations in plasma within the normal range, regardless of HF etiology (Citation16).

The difference in copper intake observed between the groups did not influence copper levels in plasma. This is probably because the homeostatic mechanisms involved in copper homeostasis are similar to those involved in zinc homeostasis (Citation40). As in our study, Xu et al. (Citation41) also failed to find differences in ceruloplasmin levels between groups of hospitalized patients with ischemic HF and non-ischemic HF. Nevertheless, previous studies indicate that ceruloplasmin is related to reduced functional capacity and increased mortality in patients with HF (Citation42,Citation43), supporting its use as a copper biomarker.

We confirmed that there was an inverse relationship between plasma zinc and age and the use of diuretics, corroborating the findings of other studies (Citation16,Citation36,Citation44,Citation45). Daily energy intake declines with advancing age, compromising albumin synthesis and leading to a reduction in the intake of food sources of zinc (Citation44).

Although the proportion of older persons in our sample was low, this finding suggests an increased risk of zinc deficiency with advancing age in patients with HF due to increases in life expectancy as a result of advances in HF therapy, particularly the use of diuretics that can potentiate urinary zinc excretion (Citation45). The positive association between ischemic HF and zinc levels in plasma was unexpected, despite comorbidities such as diabetes and arterial hypertension in this group of patients that result in poor zinc status.

Another important finding of this study is the positive association between albumin and zinc and copper levels in plasma, bearing in mind that hypoalbuminemia is a common condition in the context of HF (Citation46). Hypoalbuminemia has a direct impact on the transport of plasma zinc and copper, particularly zinc, compromising biological functions (Citation16,Citation47). These findings should be considered in the clinical monitoring of hypoalbuminemia and its impact on the risk of copper and zinc deficiency.

Our findings show that albumin, inflammatory markers (ceruloplasmin and hs-CRP), and oxidative stress (GGT) markers are predictors of copper levels in plasma (Citation48,Citation49). GGT in particular is considered an important marker in the context of HF. Abnormal levels of this transferase are related to an increased risk of mortality and an indication for cardiac transplantation (Citation50). The exact mechanism involved in this relation is not clear. However, it has been reported that GGT and copper share common biological functions in the body’s response to oxidative stress, whereby increases in the expression of GGT leads to an increase in copper levels in plasma (Citation14).

Likewise, the underlying mechanisms that cause an increase in hs-CRP, which in turn lead to increased copper levels in plasma, remain unclear. Authors suggest that in response to chronic inflammation, as occurs in HF, copper levels increase, triggering physiological responses to combat oxidative stress, which in turn activate inflammatory responses, leading to increased levels of inflammatory markers such as hs-CRP. In line with these findings, our linear regression model indicated that albumin, hs-CRP, ceruloplasmin, and GGT have a synergistic effect increasing plasma copper, exacerbating the risk of a more unfavorable prognosis in patients with HF.

Previous studies evaluating other nutrients in HF patients, such as vitamin D, amino acids, B vitamins and microelements supplementation, have demonstrated potential effects on cardiac function, improvement of symptoms, functional capacity and LVEF, especially in ischemic HF. A case-control study performed by Buleu et al. (Citation51) suggests the role of vitamin D in flux-mediated vasodilation with endothelial function and vascular tone optimization. In a review evaluating potential benefits of multiple micronutrient supplementation in HF, particularly in patients with low LVEF (≤40%), the authors concluded that micronutrient supplementation improves health outcomes in HF patients by increasing LVEF, exercise tolerance and functional capacity, in addition to beneficial effects on the contractile function of the heart, incidence of cardiac events and mortality (Citation52). These findings highlight the importance of evaluating the role of micronutrients in HF patients, in order to suggest nutritional guidelines and improve quality of life for this population.

The present study has the following limitations: (Citation1) the assessment of biomarkers of zinc and copper status was performed using a cross-sectional study design and lacked a healthy control group, thus limiting comparability; (Citation2) the study sample was recruited in a single health facility; however, this clinic is a state referral center for HF treatment; (Citation3) the number of participants in each patient group was small, which may have affected statistical power in the differences of the measures between groups.

In conclusion, plasma zinc and copper, urinary zinc, and ceruloplasmin levels in clinically stable patients with HF did not seem to be responsive to the differences in zinc and copper intake observed in this study, regardless of HF etiology. It is important to emphasize that predictors of plasma zinc and copper levels related to oxidative stress and inflammation should be used with caution for monitoring HF in clinical practice. These results may help design effective clinical and nutritional intervention protocols.

Authors' contributions

FLAF, RVZD and KCMSE conceived and designed the study. FLAF, RCSDK, NRDL and RVZD performed the collection of data FLAF, LFCP, FBJ, SCVCL and KCMSE assembled, analyses and interpretation the data. FLAF, RCSDK, NRDL, RVZD, SCVCL, FBJ, LFCP and KCMSE drafting the article and revision it critically for intellectual content. All authors read and approved the final version of the article to be submitted.

Disclosure statement

The authors declare no conflicts of interest.

Additional information

Funding

This work was supported by the State of Rio Grande do Norte Research Foundation (Fundação de Apoio à Pesquisa do Estado do Rio Grande do Norte – FAPERN)/Coordination of Improvement of Higher Education Personnel (Coordenação de Aperfeiçoamento de Pessoal do Nível Superior – CAPES) (Grant number 006/2014), and State of São Paulo Research Foundation (Fundação de Amparo à Pesquisa do Estado de São Paulo – FAPESP) (Grant number 2018/24069-3). This study was funded in part by the Coordination of Improvement of Higher Education Personnel (Coordenação de Aperfeiçoamento de Pessoal do Nível Superior – CAPES) (Grant number 001).

References

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.