150
Views
2
CrossRef citations to date
0
Altmetric
Original Research

Calcium Hydroxide Increases Human Umbilical Cord Mesenchymal Stem Cells Expressions of Apoptotic Protease-Activating Factor-1, Caspase-3 and Caspase-9

, , , , , , , & ORCID Icon show all
Pages 59-65 | Published online: 08 Mar 2021

Abstract

Purpose

Calcium hydroxide is a gold standard dental material generally used for pulpal and periapical therapy including regenerative endodontic procedures because of its positive properties. However, evaluation about this material on stem cells is limited. Human umbilical cord mesenchymal stem cells (HUCMSCs) are potential to be used in regenerative therapy. Regenerative therapy needs a sustainable cell supply to maintain its regenerative capacity. The aim of this study was to ascertain the apoptosis result of calcium hydroxide on HUCMSCs through the expression of apoptotic protease-activating factor-1 (APAF-1), caspase-3, and caspase-9.

Materials and Methods

This study used a thawed frozen stock of passage 5 HUCMSCs, grown in minimum essential medium (MEM) alpha containing calcium hydroxide at concentration of 0.1 microgram/mL for 1, 3 and 7 days. Polyclonal antibody with fluorescence isothiocyanate (FITC) label was used to evaluate the expressions. APAF-1, caspase-3, and caspase-9 expressions were recorded and compared on every observation day using fluorescence microscope. Analysis of variance was performed to analyze the significance among the results of treatment groups. The results were concluded significant if p<0.05.

Results

The addition of calcium hydroxide in MEM alpha medium increases HUCMSCs expression of APAF-1, caspase-3 and caspase-9 significantly, compared to the control group without calcium hydroxide (p<0.05) in all the times. Day 1 showed the lowest increase followed by higher expressions on day 3 and day 7.

Conclusion

HUCMSCs express increased APAF-1, caspase-3 and caspase-9 after in-vitro calcium hydroxide exposure. This should be considered when using calcium hydroxide on HUCMSCs for regenerative procedures with regard to other positive properties.

Introduction

Mesenchymal stem cells (MSC) are the most commonly used for regenerative therapy. These cells can easily be isolated, transplanted and It has the capacity of immune regulatory, self-renewal and differentiation into many cell types, such as osteocyte, neurons, adipocyte and chondrocyte.Citation1 Human umbilical cord mesenchymal stem cells (HUCMSCs) are MSC derived from Wharton’s jelly. HUCMSCs may be an appealing contender for application in periapical, pulpal, and alveolar bone regeneration.Citation2 With regard to other adult stem cell sources, HUCMSCs are primordial, involving non-invasive collection method, higher differentiation capacity, immunosuppressive, immune privileged, and rich in stemness characteristics.Citation3,Citation4

Calcium hydroxide has been used in a number of treatment protocols in endodontics, including inter-appointment dressing, pulp capping agents, and root canal sealers.Citation5 Calcium hydroxide is a vital alkaline substance administered in a favorable protocol of regenerative therapy.Citation6 Calcium hydroxide may supply calcium ions that are necessary in the outset of intrinsic apoptotic pathway.

Stem cell availability and sustainability is important for the regenerative process in regenerative procedures, including regenerative endodontic procedures. The availability and persistence of HUCMSCs can be affected by programmed cell death or apoptosis. Previous study showed that stem cells reactions may vary based on time variation of contact towards the presence of calcium hydroxide.Citation7 Apoptosis is crucial for tissue development and regeneration.Citation8 Apoptotic protease-activating factor-1 (APAF-1), caspase-3, and caspase-9 are among prominent markers for apoptosis. In order to understand the effect of calcium hydroxide on HUCMSCs’ survival, we observed the expression of these apoptotic biomarkers on HUCMSCs. Therefore, the aim of this study was to explore the expression of apoptotic protease-activating factor-1 (APAF-1), caspase-3, and caspase-9 on HUCMSCs after continuous contact with calcium hydroxide for 1 day, 3 days, and 7 days period in vitro. The hypothesis of this study is calcium hydroxide would increase apoptotic protease-activating factor-1 (APAF-1), caspase-3, and caspase-9 expressions on HUCMSCs after continuous contact for 1 day, 3 days, and 7 days period in vitro.

Materials and Methods

Preparation of HUCMSCs and Calcium Hydroxide

Ethical clearance for this study was given by the Commission of Ethical Clearance for Health Research from Faculty of Dental Medicine Universitas Airlangga, Surabaya, Indonesia (No. 059/HRECC.FODM/II/2020). This study was conducted following ethical standards of experiments. The cell donors provided informed consent, in accordance with the Declaration of Helsinki. Frozen HUCMSCs stock from passage 5 was supplied by the Stem Cell Research and Development Center Universitas Airlangga, Surabaya, Indonesia. This frozen stock was previously isolated and characterized.

HUCMSCs was confirmed previously by conducting flow cytometric check utilizing FACS Calibur (BD Biosciences, USA) for specific antibodies purchased from Becton-Dickinson (BD Biosciences, USA) for positive CD73, CD90, CD105, and negative CD45 and CD34. This fluorescence activated cell sorter (FACS) instrument was utilized to count the fluorescence intensity expressed by the cells.Citation9,Citation10 Frozen stock of passage 5 HUCMSCs was taken from minus 80°C cold storage and thawed by water bathing. After thawing, HUCMSCs were transported to other container with 10 mL minimum essential medium (MEM) alpha (Gibco, UK) at 37°C and centrifuged until they formed a cell pellet at 1600 rpm for 5 minutes. The cell pellet was resuspended in 12 mm culture plate containing MEM alpha medium and stored in incubator for several hours at 37°C. The thawed HUCMSCs were expanded and ready to be used for further study.Citation11

Calcium hydroxide was obtained by combining minimum essential medium alpha (Gibco, UK) with the powder (EMSURE Merck, Germany). Calcium hydroxide concentration of 0.1 microgram/mL was used in this experiment.

Evaluation of Apoptosis

HUCMSCs were assigned into 9 groups of control and 9 groups of calcium hydroxide treatment, with allocation for 1, 3, and 7 days of observations. Each group consisted of six wells of M24 plate (Iwaki Asahi, Japan). Every M24 plate well was seeded with 250.000 HUCMSCs in 1 mL media. HUCMSCs in the treatment groups were grown in minimum essential medium (MEM) alpha containing 0.1 microgram/mL calcium hydroxide, and the control groups were grown in MEM alpha medium only without the addition of calcium hydroxide. The groups were cultured in incubator at 37°C and 5% CO2 and observed for 1, 3, and 7 days.

Apoptosis reaction of HUCMSCs was investigated through the expressions of apoptotic protease-activating factor-1 (Bioss Antibodies, USA), Caspase-3 (Bioss Antibodies, USA), and Caspase-9 (Bioss Antibodies, USA). The investigation was done following the manufacturer’s instructions Other previous study used flow cytometric analysis for assessments.Citation12 In this research APAF-1, Caspase-3, and Caspase-9 expressions were assessed for apoptosis using polyclonal antibody with fluorescence isothiocyanate (FITC) label (Bioss Antibodies, USA). Observation of the results was done with fluorescence microscope (Olympus, Japan) with imaging system at 100x magnification and processed in ImageJ software for fluorescence quantification (National Institute of Health, USA).Citation13

Statistical Analysis

The evaluation was conducted in triplicates. Data were expressed as mean ± standard deviation from the experiment. The data were checked for normal distribution. T-test was carried out for comparisons between treatment and control groups. One-way analysis of variance (ANOVA) was used for comparisons of three groups among day 1, 3, and 7 on each expression. Difference among groups was appraised significant if P<0.05.

Results

Mean and standard deviation (SD) of APAF-1, Caspase-3, and Caspase-9 expressions from HUCMSCs in control groups and calcium hydroxide groups is available on . Calcium hydroxide increase APAF-1 expression, corresponds to the days observed (). APAF-1 expression was low on day 1 of both control and calcium hydroxide groups. Both control and calcium hydroxide groups showed significant increase (p<0.05) from day 1 to day 3, and then the calcium hydroxide group increase significantly on day 7 compared to the control group. There was a significant increase of APAF-1 expression on both the control and calcium hydroxide treatment groups from day 1 to day 3, and from day 3 to day 7 (p<0.05). Significance (p value) among exposure day in the control groups and calcium hydroxide groups on APAF-1 expression is available in .

Table 1 Mean and Standard Deviation (SD) of APAF-1, Caspase-3, and Caspase-9 Expressions from HUCMSCs in Control Groups and Calcium Hydroxide Groups

Table 2 Significance (P-value) Among Exposure Day in the Control Groups and Calcium Hydroxide Groups on APAF-1 Expression

Figure 1 HUCMSCs culture seen under light microscope (L) and fluorescence microscope (F) showing the expressions of APAF-1, Caspase-3, and Caspase-9 observed on day 1, day 3, and day 7 between control groups and calcium hydroxide (CH) groups.

Figure 1 HUCMSCs culture seen under light microscope (L) and fluorescence microscope (F) showing the expressions of APAF-1, Caspase-3, and Caspase-9 observed on day 1, day 3, and day 7 between control groups and calcium hydroxide (CH) groups.

Calcium hydroxide increase caspase-3 expression, corresponds to the days observed (). Caspase-3 expression of the control group was low on day 1, day 3, and day 7. Caspase-3 expression of calcium hydroxide group increase significantly on day 3 and then slightly decrease on day 7 (there was no significant decrease (p>0.05) between day 3 and day 7). There was no significant difference of Caspase-3 expression on day 1. There was significant difference between the control groups and calcium hydroxide groups on day 3 and day 7 (p<0.05). Significance (p value) among exposure day in the control groups and calcium hydroxide groups on caspase-3 expression is available in .

Table 3 Significance (P-value) Among Exposure Day in the Control Groups and Calcium Hydroxide Groups on Caspase-3 Expression

Calcium hydroxide increase Caspase-9 expression, corresponds to the days observed (). Caspase-9 expression of both the control groups and calcium hydroxide groups were low on day 1. The control group showed a gradual increase from day 1 to day 7. The calcium hydroxide groups revealed a significant increase of Caspase-9 on day 3 and then quite steady on day 7 without significance (p>0.05). Significance (p value) among exposure day in the control groups and calcium hydroxide groups on caspase-9 expression is available in .

Table 4 Significance (P-value) Among Exposure Day in the Control Groups and Calcium Hydroxide Groups on Caspase-9 Expression

Discussion

There are many sources of mesenchymal stem cells. Among other stem cells, HUCMSCs is prominent because the isolation process is non-invasive and they can be expanded in large quantities.Citation14 HUCMSCs and its application in regenerative dental procedures are limited. In this study, the HUCMSCs are according to standardization of MSCs cluster of differentiation surface markers and morphology.Citation15 The HUCMSCs were grown in minimum essential medium alpha with and without the addition of calcium hydroxide, under normal culture condition. MTT assay was used in this study to measure cell viability under the effect of calcium hydroxide, and concentration of 0.1 microgram/mL was chosen because it has more than 60% viable cells. In this study we found that 0.1 microgram/mL of calcium hydroxide promotes the expressions of APAF-1, caspase-3 and caspase-9.

Apoptosis is a usual form of programmed cell death process that participates in homeostatic control of cell population, without inflammation take place.Citation16 Apoptosis is sophisticatedly managed and balanced mechanism which helps in the elimination of unfavorable cells throughout every organism’s life cycle.Citation17 Apoptosis is initiated by an intrinsic (classical or mitochondrial) pathway and an extrinsic (death receptor or cytoplasmic) pathway.Citation18 Both pathways converge in the final apoptotic execution phase, which is marked by nuclear DNA breakdown, protein cleavage, and apoptotic cell identification by phagocytic cells.Citation19

Any condition or stimuli to a cell’s environment is capable to induce apoptotic signaling.Citation20 Cells experiencing apoptosis release extracellular vesicles, such as apoptotic bodies, micro-vesicles, and apoptotic exosomes, which roles in immune responses and inflammation, as an active communication from dying cells to surrounding living cells.Citation17 Defected or dysregulation in apoptotic pathways may lead towards various malignancies and diseases, including AIDS, diabetes, and neurodegenerative diseases involving the perturbation of genes.Citation21

Apoptosis is influenced by many signals interdependently. In intrinsic apoptotic pathway there are other influencing factors, such as the effector proteins BAK and BAX, antiapoptotic proteins (A1, Mcl-1, Bcl-xL, Bcl-w, and Bcl-2) and proapoptotic BH3-only proteins (Puma, Noxa, Bid, Bad, Bik, Bmf, Bim, and Hrk).Citation22 Active BAX and BAK induce mitochondrial outer-membrane permeabilization (MOMP), and initiates cytochrome c efflux, which form a complex with APAF-1 and activate caspase-9 to activate caspase-3.Citation23 In extrinsic apoptotic pathway there are influencing factors, such as TNF receptor superfamily (TNF alpha, FAS and TRAIL receptors), cytosolic death domains (DD) receptors to bind TNFR-associated death domain (TRADD) or Fas-associated death domain (FADD), and finally form a death-inducing signaling complex (DISC), which would activate caspase-3, caspase-6 and caspase-7.Citation17

APAF-1 (Apoptotic protease activating factor 1) is normally present in cytoplasm in inactive form, and can be activated by cytochrome c and controlled by pro and anti-apoptotic molecules.Citation19 APAF-1 is responsible for caspase-9 activation. Aside from its central role in the initiation of cell death, APAF-1 have non-apoptotic functions. The non-apoptotic roles of APAF-1 are modulatory effect on cell cycle during DNA damage induced by genotoxic stress and participation in the cytoskeleton arrangement and centrosomic microtubule nucleation process.Citation24

Caspases are a collection of proteases recognized for their important function in programmed cell death, abbreviated from cysteine-aspartic protease activity.Citation19 Caspases have been grouped into apoptotic caspases and inflammatory caspases. According to the action mechanism, apoptotic caspases are classified into initiator caspases and executioner caspases. Caspase-3 and caspase-9 belong to the apoptotic caspase group; caspase-3 is an executioner and caspase-9 is an initiator.Citation17

Caspase-3, a pivotal role in apoptosis, is a downstream effector and is affected by intrinsic and extrinsic pathways.Citation25,Citation26 Various stimuli can affect the apoptosis. Calcium ion plays a crucial role in the intrinsic apoptotic pathway: high regulation of calcium into mitochondria lead to the deliverance of cytochrome c.Citation16 In the cytoplasm, cytochrome c ties to a cytoplasmic protein known as APAF-1 and shapes the apoptosome that initiates procaspase-9 into caspase-9 and then triggers other caspases that will eventually execute cell disintegration.Citation27

Increased expression of APAF-1 would mean increased levels of apoptosis. Caspase 9 would turn on pro-caspase-3 into caspase-3 as a caspase effector which conveys apoptosis.Citation28 The quantity of APAF-1 and procaspase-9 affect the proportion of caspase-9 homodimers and heterodimers shaped in the apoptosome.Citation29

Previous reports demonstrated that mitochondrial caspases in intrinsic apoptosis have several non-apoptotic roles, including cellular reprogramming, differentiation, immunogenic, and proliferation.Citation23 Most caspases have roles in cell proliferation, survival, inflammation, or differentiation.Citation25 Other than lethal function in apoptosis, caspase-3 and caspase-9 have non-lethal function in cell differentiation, with caspase-3 having more functions in cell maturation and activation.Citation23 Caspase-3 has a vital role in tissue regeneration, differentiation, and neural development differently and not involving apoptotic activity.Citation30

In this study, the addition of calcium hydroxide was correlated with high expression of APAF-1, caspase-3, and caspase-9, but these intense expressions are also happened to the control groups. HUCMSCs experience apoptosis on its own timing, but the addition of calcium hydroxide accelerated HUCMSCs to apoptosis. Calcium hydroxide trigger the mitochondrial apoptosis. This trigger might not only cause apoptosis but might also have a function in immune defense mechanism.Citation17 The increased apoptosis in this study may imply clinical practice during regenerative endodontic procedures. In this case, it may have roles in stem cell proliferation, survival, or differentiation. Another study involving calcium hydroxide provided that it upregulates HUCMSCs interleukin-10 expression and osteogenic differentiation.Citation31 As this research is an in vitro study, there are limitations. Therefore, we should also explore at broader potential aspects or biomarkers other than apoptosis.

However, apoptosis is needed for homeostasis and pathological processes.Citation22 Previous reports showed that, during apoptosis, cells sustain the integrity of the plasma membrane to prevent inflammation.Citation8 This study indicates evidence regarding HUCMSCs apoptosis induced by calcium hydroxide. Appropriate apoptosis of HUCMSCs is advantageous for tissue regeneration.

Conclusion

In conclusion, although HUCMSCs has a high proliferation capacity and anti-inflammation capacity, this study provides evidence that calcium hydroxide promotes HUCMSCs to apoptosis, even if it was administered in non-toxic dose. Further studies are needed to know more about HUCMSCs use in regenerative dental procedures.

Acknowledgments

The authors would like to thank Lembaga Pengelola Dana Pendidikan Kementerian Keuangan Republik Indonesia, Faculty of Medicine, and Faculty of Dental Medicine Universitas Airlangga for the given technical supports.

Disclosure

The authors report no conflicts of interest in this work.

References

  • Rajabzadeh N, Fathi E, Farahzadi R. Stem cell-based regenerative medicine. Stem Cell Investig. 2019;6:19. doi:10.21037/sci.2019.06.04
  • Meguid EA, Ke Y, Ji J, El-Hashash AHK. Stem cells applications in bone and tooth repair and regeneration: new insights, tools, and hopes. J Cell Physiol. 2017;9999:1–11. doi:10.1002/jcp.25940
  • Li T, Xia M, Gao Y, Chen Y, Xu Y. Human umbilical cord mesenchymal stem cells: an overview of their potential in cell-based therapy. Expert Opin Biol Ther. 2015;15:1293–1306. doi:10.1517/14712598.2015.1051528
  • Subramanian A, Fong CY, Biswas A, Bongso A. Comparative characterization of cells from the various compartments of the human umbilical cord shows that the Wharton’s jelly compartment provides the best source of clinically utilizable mesenchymal stem cells. PLoS One. 2015;10(6):e0127992. doi:10.1371/journal.pone.0127992
  • Ba-Hattab R, Al-Jamie M, Aldreib H, Alessa L, Alonazi M. Calcium hydroxide in endodontics: an overview. Open J Stomatol. 2016;6:274–289. doi:10.4236/ojst.2016.612033
  • Kahler B, Chugal N, Lin LM. Alkaline materials and regenerative endodontics: a review. Materials. 2017;10:1389. doi:10.3390/ma10121389
  • Prasetyo EP, Widjiastuti I, Cahyani F, et al. Cytotoxicity of calcium hydroxide on human umbilical cord mesenchymal stem cells. Pesqui Bras Odontopediatria Clin Integr. 2020;20:e0044. doi:10.1590/pboci.2020.149
  • Wu T, Li L, Du R, Jiang L, Zhu Y. Hydrogen peroxide induces apoptosis in human dental pulp cells via caspase-9 dependent pathway. J Endod. 2013;39(9):1151–1155. doi:10.1016/j.joen.2013.06.006
  • Fathi E, Valipour B, Sanaat Z, Charoudeh HN, Farahzadi R. Interleukin-6, −8, and TGF-b secreted from mesenchymal stem cells show functional role in reduction of telomerase activity of leukemia cell via Wnt5a/b-catenin and P53 pathways. Adv Pharm Bull. 2020;10(2):307–314. doi:10.34172/apb.2020.037
  • Mobarak H, Fathi E, Farahzadi R, Zarghami N, Javanmardi S. L-carnitine significantly decreased aging of rat adipose tissue-derived mesenchymal stem cells. Vet Res Commun. 2017;41:41–47. doi:10.1007/s11259-016-9670-9
  • Kuntjoro M, Prasetyo EP, Cahyani F, et al. Lipopolysaccharide’s cytotoxicity on human umbilical cord mesenchymal stem cells. Pesqui Bras Odontopediatria Clin Integr. 2020;20:e0048. doi:10.1590/pboci.2020.153
  • Farahzadi R, Fathi E, Vietor I. Mesenchymal stem cells could be considered as a candidate for further studies in cell-based therapy of Alzheimer’s disease via targeting the signaling pathways. ACS Chem Neurosci. 2020;11:1424–1435. doi:10.1021/acschemneuro.0c00052
  • Jensen EC. Quantitative analysis of histological staining and fluorescence using ImageJ. Anatomical Record. 2013;296:378–381. doi:10.1002/ar.22641
  • Deng Y, Zhang Y, Ye L, et al. Umbilical cord-derived mesenchymal stem cells instruct monocytes towards an IL10-producing phenotype by secreting IL6 and HGF. Sci Rep. 2016;6:37566. doi:10.1038/srep37566
  • Nugraha AP, Prasetyo EP, Kuntjoro M, et al. The effect of cobalt (II) chloride in the viability percentage and the induced hypoxia inducible factor-1a of human adipose mesenchymal stem cells (HAMSCs): an in vitro study. Sys Rev Pharm. 2020;11(6):308–314. doi:10.31838/srp.2020.6.49
  • Kontogiannis TG, Tosios KI, Kerezoudis NP. Effect of calcium hydroxide as intracanal medicament on the expression of caspase-9 located within the radicular cyst epithelium. Aust Endod J. 2019;1–5. doi:10.1111/aej.12325
  • Kakarla R, Hur J, Kim YJ, Kim J, Chwae Y. Apoptotic cell-derived exosomes: messages from dying cells. Exp Mol Med. 2020;52:1–6. doi:10.1038/s12276-019-0362-8
  • Rogers C, Fernandes-Alnemri T, Mayes L, Alnemri D, Cingolani G, Alnemri ES. Cleavage of DFNA5 by caspase-3 during apoptosis mediates progression to secondary necrotic/pyroptotic cell death. Nat Commun. 2017;8:14128. doi:10.1038/ncomms14128
  • Cavalcante GC, Schaan AP, Cabral GF, et al. A cell’s fate: an overview of the molecular biology and genetics of apoptosis. Int J Mol Sci. 2019;20:4133. doi:10.3390/ijms20174133
  • Brokatzky D, Dorflinger B, Haimovici A, et al. A non-death function of the mitochondrial apoptosis apparatus in immunity. EMBO J. 2019;38:e100907. doi:10.15252/embj.2018100907
  • Arif M, Syed A, Mahmood A, Khan S, Rizwan M, Munir A. Modelling of apoptosis through gene interaction network and analysis of gene expression pattern. Meta Gene. 2020;25:100730. doi:10.1016/j.mgene.2020.100730
  • Lamb HM. Double agents of cell death: novel emerging functions of apoptotic regulators. FEBS J. 2020;287:2647–2663. doi:10.1111/febs.15308
  • McArthur K, Kile BT. Apoptotic caspases: multiple or mistaken identities. Trends Cell Biol. 2018;28(6):475–493. doi:10.1016/j.tcb.2018.02.003
  • Shakeri R, Kheirollahi A, Davoodi J. Apaf-1: regulation and function in cell death. Biochimie. 2017;135:111–125. doi:10.1016/j.biochi.2017.02.001
  • Aydogan A, Kocer G, Ozmen O, Kocer M, Onal L, Koskan O. Immunohistochemical expression of caspase-3, caspase-5, caspase-7 and apoptotic protease-activating factor-1 (APAF-1) in the liver and kidney of rats exposed to zoledronic acid (ZOL) and basic fibroblast growth factor (bFGF). Vet Q. 2014;34(3):137–142. doi:10.1080/01652176.2014.928759
  • Pradeep AR, Suke DK, Prasad MVR, et al. Expression of key executioner of apoptosis caspase-3 in periodontal health and disease. J Investig Clin Dent. 2016;7:174–179. doi:10.1111/jicd.12134
  • Taylor RC, Cullen SP, Martin SJ. Apoptosis: controlled demolition at the cellular level. Nat Rev Mol Cell Biol. 2008;9:231–241. doi:10.1038/nrm2312
  • Nandarani RE, Widjiastuti I, Mooduto L. Pulp fibroblast cell apoptosis after application of hema dentine bonding material with ethanol and water solvent. Braz Dent J. 2019;30(3):208–212. doi:10.1590/0103-6440201902524
  • Wu C, Lee S, Malladi S, et al. The apaf-1 apoptosome induces formation of caspase-9 homo- and heterodimers with distinct activities. Nat Commun. 2016;7:13565. doi:10.1038/ncomms13565
  • Shalini S, Dorstyn L, Dawar S, Kumar S. Old, new and emerging functions of caspases. Cell Death Differ. 2015;22:526–539. doi:10.1038/cdd.2014.216
  • Prasetyo EP, Kuntjoro M, Cahyani F, et al. Calcium hydroxide upregulates interleukin-10 expression in time dependent exposure and induces osteogenic differentiation of human umbilical cord mesenchymal stem cells. Int J Pharm Res. 2021;13(1):140–145. doi:10.31838/ijpr/2021.13.01.023