341
Views
6
CrossRef citations to date
0
Altmetric
Articles

Modelling immune response to BK virus infection and donor kidney in renal transplant recipients

, , , , &
Pages 127-152 | Received 20 Jun 2014, Accepted 25 Jan 2015, Published online: 13 Mar 2015

Abstract

In this paper, we develop and validate with bootstrapping techniques a mechanistic mathematical model of immune response to both BK virus infection and a donor kidney based on known and hypothesized mechanisms in the literature. The model presented does not capture all the details of the immune response but possesses key features that describe a very complex immunological process. We then estimate model parameters using a least squares approach with a typical set of available clinical data. Sensitivity analysis combined with asymptotic theory is used to determine the number of parameters that can be reliably estimated given the limited number of observations.

AMS Subject Classifications:

1. Introduction

According to the OPTN/SRTR 2011 Annual Report, [Citation1] 17,604 kidney transplants were performed in the United States between 2010 and 2011. Overall, there were 54,599 active candidates on the waiting list for kidney transplants, roughly threefold more than those that underwent transplant. These numbers reflect trends consistent with previous years, in which the number of patients waiting for transplants greatly exceeds the availability of organs. Given these facts, and the fact that as of 30 June 2011, 164,200 adults in the US were surviving with a functioning kidney graft, about twice as many as a decade earlier, optimal care of renal transplant patients is of great importance.

To reduce risk of allograft rejection, the standard of care for renal transplant recipients involves life-long pharmacological immunosuppression, making patients susceptible to opportunistic infections. Specifically, this therapy can render the recipients susceptible to an array of viral pathogens and may also reactivate latent viruses. For some time, human polyomavirus type 1, named ‘BK virus’ (BKV), has been a common pathogen found in kidney transplant patients. BKV is one of the two human polyomaviruses and was first discovered in 1970 in the urine of a kidney transplant patient with the initials BK.[Citation2] This double-stranded non-enveloped DNA virus with icosahedral capsids asymptomatically infects more than 90% of the adult population worldwide and establishes a state of non-replicative infection,[Citation3Citation5] or latent state. The infection is established in the kidneys and peripheral blood, specifically the renal tubular epithelial and urothelial cells, where replication-permissive cells express the viral capsid proteins followed by virion assembly in the nucleus.[Citation6] This process eventually causes host cell lysis and the release of infectious progeny, deeming BKV replication as cytopathic, leading to a new round of active infection and latency. A high-level of BKV replication in the kidney results in a complication known as polyomavirus associated nephropathy (PVAN).[Citation7Citation10] It is characterized by viral cytopathic changes of renal tubular epithelial cells, with enlarged nuclei, cell rounding, detachment and denudation of basal membranes.[Citation11] Increasing prevalence rates of PVAN (1–10%) have been reported, with allograft dysfunction and loss in greater than 50% of cases.[Citation12, Citation13] Therefore, PVAN is viewed as one of the leading causes of renal allograft loss in the first two years of transplantation. Unfortunately, there are no available licensed anti-polyoma viral drugs, and treatment relies on improving immune function to control BKV replication.[Citation7] Given BKV infection is a significant health threat to immunosuppressed renal transplant patients, patient outcomes might be improved with the use of mathematical modelling to predict the course of the disease in individuals and recommend optimized treatment strategies.

Mathematical modelling is widely used and historically accepted in the physical science and engineering communities as an aid in the understanding of complex phenomena. Specifically, we highlight the use of mathematical modelling with experimental investigations to enhance the understanding of biological processes. The process involves a sequences of steps: (i) empirical observations, experiments and data collection; (ii) formalization of the biological model; (iii) abstraction of mathematical model; (iv) formalization of uncertainty and use of a statistical model; (v) model analysis; (vi) changes in understanding; and (vii) design of new experiments.[Citation14] Given the highly iterative process of mathematical modelling and the recently developed quantitative techniques such as real-time PCR measurements of viral load, flow cytometry of T cell subsets and ELISPOT for virus-specifical T cell function, it is feasible to couple mathematical modelling with biological experimentation to investigate the dynamics of viral infection and the cellular immune response.

Here, we give a brief review of recent works that use mathematical modelling to investigate viral infection in relation to organ transplant dynamics. Funk et al. [Citation9], summarized investigations of a retrospective analysis of BKV plasma load in renal transplant recipients undergoing allograft nephrectomy or changes in immunosuppressive regimens. PCR measurements of viral DNA are applied to a standard mathematical model for viral load decay kinetics to estimate the half-life and doubling time of BKV as well as clearance and growth rates.[Citation9] This model addresses purely BKV replication. The next iteration in the modelling process requires the development of dynamic models that account for interactions between viral infection and host cell populations. Funk et al. [Citation15] extended a one-compartment model to a two-compartment model with six state variables describing BKV replication dynamics in renal tubular epithelial cells and in urothelial cells. Estimation of parameters was based on population level clearance, proliferation, etc., rates. The study presented a basic model integrating two replication sites, the kidney and the urinary tract, and derived four variants which incorporated coupled and decoupled dynamics of the two sites. It remains unknown whether the two replicate sites are in fact coupled; however, results in Funk et al. [Citation15] suggest that viral expansion was best explained by models where BKV replication started in the kidney followed by urothelial amplification and then reached an equilibrium amongst both replication sites. The model does not address the response of the immune system to viral infection and donor kidney and little to no statistically based model validation or calibration as proposed in our efforts here was carried out.

Other viral infections have been studied in relation to organ transplant dynamics. In particular, two models have been developed involving human cytomegalovirus (HCMV) infection. Kepler et al. [Citation16] developed a dynamic model describing the pathogenesis of primary HCMV infection in immunocompetent and immunosuppressed patients at the cellular/viral mechanistic scale for application to individual clinical data and patient health. The model incorporates dynamics of the viral load, immune response as well as actively infected, susceptible and latently infected cells. Results highlight the necessity of longitudinal data for multiple state variables for robust parameter estimation. In addition, Banks et al. [Citation17] developed a model to describe the immune response to both HCMV viral infection and introduction of a donor kidney in a renal transplant recipient. Dynamics of the viral load, susceptible and infected host cells, the immune response specific to viral infection and the transplant, and creatinine are incorporated into the model. Delineation between the cellular immune response to HCMV infection and the alloreactive immune response to the transplanted kidney as well as the incorporation of creatinine dynamics are vital additions to the dynamic model.

In the present effort, we develop a mathematical model of BKV infection and renal transplant dynamics at the cellular/viral mechanistic scale for application to renal transplant immunosuppressed individual clinical data. Specifically, we adapt dynamics of the HCMV model in Banks et al. [Citation17] to allot for more specific BKV infection features. We eliminate the use of an antiviral treatment term, incorporate the effect of the alloreactive immune response and the presence of susceptible host cells on the clearance of creatinine, and add the effect of susceptible host cells on the concentration of allospecific CD8+ T cells. In contrast to the model in Funk et al. [Citation15], we assume that the two BKV replication sites, the kidney and the urinary tract, are decoupled, focussing on the kidney as the main replication site. This choice was not only made given the inconclusive literature, but the data types available. Examining BKV replication within the urinary tract would require BKV urine data, which is not included in our available data-sets. We use a typically available data-set to pursue statistically based model validation or calibration in attempts to discern the specific information content one might expect in such a data-set.

The remainder of this paper is organized as follows. In Section 2, we present the biological model for which we base our mathematical model describing the immune response to both BKV infection and the introduction of a donor kidney in a renal transplant patient. An overview of clinical data acquired from collaborators is also given in Section 2. Model calibration and analysis is detailed in Section 3, where we give details regarding a log-transformed system, provide a detailed procedure for sensitivity analysis as well as outline our iterative inverse problem methodology. Details regarding computation of standard errors and confidence intervals are also discussed in Section 3. Results are given and discussed as well. In Section 4, we summarize efforts and conclusions drawn as well as suggest future research efforts.

2. Mathematical model description and data

In this section, we describe the dynamics of the viral load V, susceptible HS and infected HI host cells, BKV-specific EV and allospecific EK effector CD8+ T cells and serum creatinine C with a brief description of the underlying biological model for which we base our mathematical model. Table lists the state variables and Figure diagrams the intracellular dynamics embodied in the model.

Table 1. Description of state variables.

Figure 1. BKV model.

Figure 1. BKV model.

Active BKV infection targets both renal tubular epithelial cells and urothelial cells. For this model, however, we focus on one BKV target, the renal tubular epithelial cells. Susceptible host cells, the uninfected kidney tubular epithelial cells, HS, in the absence of infection, are assumed to proliferate through the term λHS1-HSκHSHS, indicating that new epithelial cells are derived from proliferation of existing HS. Proliferation is modelled by logistic dynamics with λHS being the maximum proliferation rate and κHS is the cell density at which proliferation shuts off. A loss of susceptible cells, HS, due to viral infection which occurs by cell-to-cell transmission, is represented by the term βHSV. Here, we assume that one copy of virion infects one cell. Infected host cells or BKV replicating cells, HI, lyse due to the cytopathic effect of BK virus, represented by the term δHIHI and produce ρV virions before death. In addition, infected host cells are eliminated by the BK-specific effector CD8+ T cells with rate term δEHEVHI. Free virus is naturally cleared at the rate δV by the body and a loss of viral concentration is experienced through the infection of susceptible host cells.

The cellular immune response is the key defence against the BK viral infection. The terms λEV and δEV represent the source and death rates of BK-specific effector CD8+ T cells. The concentration of BK-specific CD8+ T cells increases in response to the presence of free virus through the term ρEVEV, where ρEV is a bounded positive increasing function of free virus concentration. Specifically, ρEV(V)=(ρ¯EVV)/(V+κV) is a saturating nonlinearity with positive constants ρ¯EV and κV. The alloreactive immune response to the transplanted kidney is incorporated into the model via a state variable, EK, which denotes the effector CD8+ T cells that specifically target the transplant. The source rate for EK, λEK, is dependent upon the HLA donor/recipient matching conducted prior to transplantation. Similar to the BK-specific CD8+ T cells, the concentration of allospecific CD8+ T cells increases in response to the presence of susceptible host cells HS, since BK-virus targets kidney cells, represented by the term ρEKEK, where ρEK(HS)=(ρ¯EKHS)/(HS+κKH) with positive constants ρ¯EV and κKH. The death rate of EK is represented by δEK.

Finally, we discuss the role of creatinine in the model. Creatinine is a waste product in the blood resulting from muscle activity and is removed by the healthy kidney. Therefore, serum creatinine concentration C is used as a surrogate for glomerular filtration rate, a commonly used index of kidney function.[Citation18] The production rate of C is represented by λC and when the kidney is impaired, creatinine is not effectively filtered and its concentration increases. To account for the negative effect of the alloreactive immune response EK on the kidney and the positive effect of susceptible cells HS (Recall that the renal allograft is a site of replication. Hence, the concentration of susceptible cells reflects the health of the kidney.), the clearance rate δC is defined as followsδC(EK,HS)=δC0κEKEK+κEK·HSHS+κCH.

Table lists the parameters used in the model.

Table 2. Model parameters and their corresponding descriptions.

Based on the above discussion, the model is given as follows:(1) HS˙=λHS(1-HSκHS)HS-βHSV,(1) (2) HI˙=βHSV-δHIHI-δEHEVHI,(2) (3) V˙=ρVδHIHI-δVV-βHSV,(3) (4) EV˙=(1-ϵI)[λEV+ρEV(V)EV]-δEVEV,(4) (5) EK˙=(1-ϵI)[λEK+ρEK(HS)EK]-δEKEK,(5) (6) C˙=λC-δC(EK,HS)C.(6)

with initial conditions,(7) (HS(0),HI(0),V(0),EV(0),EK(0),C(0))=(HS0,HI0,V0,EV0,EK0,C0).(7)

We note that (Equation1)–(Equation4) describe the immune response to the viral infection coupled with (Equation5) and (Equation6) describing the immune response to the transplanted kidney. Here, εI represents the efficacy of immunosuppressive drugs and is assumed to be scaled to less than or equal to 1. This variable serves as the controller of the system to achieve balance between undersuppression and oversuppression of the patient’s immune system.

2.1. Data

The data for our investigation is from Massachusetts General Hospital and our discussions here consists of one patient record, TOS003. The data collection is performed as part of the patient’s routine medical care. Visits include pre-transplantation evaluation, day of transplant and post-transplantation visits. Day of transplant and post-transplantation visits are pertinent for model validation. Record TOS003 describes immunosuppression regimen, renal function recorded in plasma creatine (mg/dL) levels and infectious complications given in BKV viral plasma loads (in DNA copies/mL). TOS0003 associated data consists of eight viral load measurements and sixteen serum creatinine measurements. It should be noted that the recipient was diagnosed with ‘BK virus reactivation’ over the course of the first three months post-transplantation. It was documented that TOS003 was given immunosuppressive treatment upon organ transplantation and monitored accordingly. We note that the efficacy of the immunosuppression is a function of time, given the evidence in the data, however, it is difficult to quantify the efficacy of the immunosupression from the drug regimen records. The dosage, type and combination of drugs are changed after each visit. Hence, for simplicity, we assume that εI can be approximated by the following piecewise constant function.(8) εI(t)=ϵ1,t[0,21],ϵ2,t(21,60],ϵ3,t(60,120],ϵ4,t(120,450],(8)

where the time frames are chosen based on the drug regimen record. It is also important to note that we are assuming test results displaying ‘None detected’ equate to the detection limit 20 copies/mL of free virus in the system.[Citation9] In addition, no test quantifying the BK viral load was conducted pre-transplant or the day of transplant; however, due to the seroprevalence of BKV in the population, it is assumed that the patient had a latent infection.

3. Model calibration and analysis

The model contains 6 variables and 29 constant parameters (model parameters and initial conditions). Equations (Equation1)–(Equation6) were first written as a vector systemx¯˙=g¯(x¯;p¯,x¯0),x¯(0)=x¯0,

where x¯=[HS,HI,V,EV,EK,C]T is the vector of model states, p¯=[λHS,β,δEH,δV,ρ¯EV,δEV,δEK,δC0,κHS,δHI,ρV,λEV,κV,λEK,λC,κEK,κCH,ρ¯EK,κKH,ϵ1,ϵ2,ϵ3,ϵ4]TR23 is the vector of model parameters, and x¯0=[HS0,HI0,V0,EV0,EK0,C0]TR6 is the set of initial conditions. Due to the scale difference between model states, we adopted the approach used in [Citation19, Citation20]: solutions were determined for a log-transformed system. We remark that this is common in the biological literature to use log scale to deal with very large numbers; for example, in HIV studies and other viral infection studies both the viral load level (which can be as high as millions of copies/mL) and the changes in the viral load level can be large and hence they are often reported and analysed in log scale (e.g. see [Citation21Citation23]). Since model parameter values and initial conditions are in different scales, a subset of the model parameters and initial conditions are also log-transformed to the log scale. This log-transformation approach is used to convert all the analysed quantities roughly to the same scale. It is worth noting that whenever we carry out a log-transformation for a quantity, we only log-transform its corresponding numerical value. In other words, log10(x¯) is always understood as a shorthand notation for log10((x¯[units])/(1[units])) so the argument of log function is really dimensionless. An alternative approach, a common practice in engineering, to deal with the scale difference among analysed quantities is to normalize them so that the numerical values of the resulting quantities vary between 0 and 1, e.g. x=(x¯-x¯min)/(x¯max-x¯min), where x stands for quantity x¯ after normalization and subscripts min and max, respectively, denote the minimal and maximal values of x¯. However, for most biological applications, the minimal and maximal values of the analysed quantities are usually unknown and may vary among individuals, which is true for the problem we presented here. Thus, this normalization approach would result in a large number of extra unknown parameters that need to be identified (recall that the model contains 6 model states and 29 constant parameters). Considering the limited number of observations available to us, this approach can provide significant, if not impossible, challenges for the inverse problem investigated below and thus will not be considered in this paper.

Let,xi=log10(x¯i),i=1,2,3,4,5,x6=x¯6,x0i=log10(x¯0i),i=1,2,3,4,5,x06=x¯06,pj=log10(p¯j),j=1,2,,19,pj=p¯j,j=20,21,22,23.

Then we have the system(9) x˙=g(x;p,x0),x(0)=x0,(9)

where g=(g1,g2,g6)T is given bygi(x;p,x0)=10-xiln(10)g¯i(x¯;p¯,x¯0),i=1,2,5,g6(x;p,x0)=g¯6(x¯;p¯,x¯0).

We remark that by using the above log-transformed system (Equation9), one can resolve a problem of states becoming unrealistically negative in solving model (Equation1)–(Equation6) due to round-off error: nonnegative solutions of this model should stay so throughout numerical simulation. This log-transformation approach also enables the changes in model states due to the changes in parameters to be more measurable. In addition, this enables ready formulation of a reasonable stopping criterion for the inverse problem and can also make the associated optimization algorithm converge faster (vastly different magnitudes of analysed quantities increase condition numbers and hence the associated optimization algorithm may require more iterations to converge). From a statistical point of view, log-transformation is also a standard technique to render the observations more nearly normally distributed, and it is also commonly used to render heteroscedastic measurement errors more homoscedastic (e.g. see [Citation24Citation26]) so that the resulting inverse problem is easier to implement (in ordinary least squares (OLS) approaches as opposed to generalized least squares approaches, e.g. see [Citation26, Citation27] for details).

3.1. Forward simulations

Forward simulations were carried out by numerically solving the log-transformed version of model Equations (Equation9) in Matlab using the ODE variable order variable step-size solver ode15s over a time course of 450 days. Parameter values used are listed in Table . The values of these parameters were either derived from published experimental studies or through simulation to acquire a benchmark value. For those parameters whose values can be found or derived from the literature, specifics are detailed below.

(1)

The parameter ρV, which quantifies the number of virions released by BKV-infected cells, was taken to be ρV6000 copies/cell.[Citation8, Citation9, Citation15, Citation28]

(2)

The measured BK viral half-life was found to be t121.1h-9days. This implies that the total clearance rate of BK is from 0.077 to 15.1232 per day. In the simulations, we set δV=0.4 per day.[Citation8, Citation15]

Table 3. Initial guess for parameter values θ=(p¯,x¯0) used in the parameter estimation simulations.

3.2. Sensitivity analysis

In practice, one may be in a situation of estimating a large number of unknown parameters with a limited data-set. Such a situation is true in our case. To alleviate some of this difficulty, sensitivity analysis has been widely used in inverse problem investigations.[Citation14, Citation20, Citation29Citation33] This process identifies the model parameters and initial conditions to which the model outputs are most sensitive. Specifically, sensitivity analysis provides insight into how changes in the parameters can affect the model output. In addition, this framework can be used to construct confidence intervals for parameter estimates using asymptotic properties of estimators.

To compute the sensitivities of model outputs to the model parameters and initial conditions, one proceeds to determine the sensitivity of each model state xi to each parameter pj and initial condition x0j. These are defined as the derivatives of the model states with respect to the parameters, xi/pj and xi/x0j, which satisfy in our case(10) ddtxipj=l=16gixlxlpj+gipj,xipj(0)=0,i=1,2,6,j=1,2,,23,(10) (11) ddtxix0j=l=16gixlxlx0j,xix0j(0)=δij,i=1,2,6,j=1,2,,6,(11)

whereδij=1,i=j,0,ij.

The sensitivities xp and xx0 can be calculated by solving (Equation9)–(Equation11) simultaneously in Matlab using ode15s where the derivatives gx, gp are obtained through automatic differentiation using myAD and tssolve in Matlab. In our case, we have data for both the free virus, V and serum creatinine, C, states. Therefore, we only need to compute the sensitivities corresponding to states xi, i=3,6. Initial conditions and model parameter values used are listed in Table . Results of the sensitivity analysis informed us as to which parameters were to be estimated. These findings along with the corresponding confidence intervals for these estimated parameters are described below.

It is worth emphasizing that sensitivity coefficients obtained above are for the log-transformed system (Equation9). We note that in the case where xi=x¯i, pj=log10(p¯j) and x0j=log10(x¯0j) we have,xipj=xix¯ix¯ip¯jp¯jpj=log(10)p¯jx¯ip¯j,xix0j=xix¯ix¯ix¯0jx¯0jx0j=log(10)x¯0jx¯ix¯0j.

Hence, the sensitivity coefficients xipj and xix0j obtained here are really the so-called relative sensitivity coefficients in [Citation33]. We also note that in the case where xi=log10(x¯i), pj=log10(p¯j) and x0j=log10(x¯0j) we havexipj=xix¯ix¯ip¯jp¯jpj=p¯jx¯ix¯ip¯j,xix0j=xix¯ix¯ix¯0jx¯0jx0j=x¯0jx¯ix¯ix¯0j.

Hence, the sensitivity coefficients xipj and xix0j obtained here are really the so-called dimensionless sensitivity coefficients in [Citation33]. For more information on the relative sensitivity coefficients and dimensionless sensitivity coefficients, we refer the interested reader to [Citation33].

3.3. Parameter estimation

The observed amount of free virus DNA in the blood is represented by yi¯1, with corresponding measured time point ti1, i=1,2,,n1 and yi¯2 is the observed amount of serum creatinine at time point ti2, i=1,2,,n2. We define yi1=log10(yi¯1), i=1,2,,n1 and yi2=yi¯2, i=1,2,,n2. Following standard inverse problem procedures,[Citation14, Citation24Citation27, Citation33] we define random variables Yi1 and Yi2 and formulate the statistical model as follows:(12) Yi1=f1(ti1;q0)+ϵi1,i=1,2,,n1,Yi2=f2(ti2;q0)+ϵi2,i=1,2,,n2,(12)

where n1=8 and n2=16. Here f1(ti1;q0)=x3(ti1;q0) and f2(ti2;q0)=x6(ti2;q0), where q0Rκ denotes the hypothesized “true values” of the model parameters and initial conditions that need to be estimated (κ is a positive integer and denotes the number of model parameters and initial conditions to be estimated). The observation errors ϵi1, i=1,2,,n1 and ϵi2, i=1,2,,n2 are assumed to be independent and identically distributed (i.i.d) with zero mean and constant variance σ02. Therefore, q0 can be properly estimated by using an OLS technique(13) q^=argminqQi=1n1|f1(ti1;q)-yi1|2+i=1n2|f2(ti2;q)-yi2|2,(13)

where Q is some compact set of admissible values in Rκ.

Note that we have a large number of parameters (29 parameters) with little experimental data (n1+n2=24 time observations). Hence, in attempts to produce reliable estimates, the parameter estimation was implemented in an iterative process similar to that used in Adams et al. [Citation19], Banks et al. [Citation20]. This process entailed identifying the most influential or sensitive parameter subset at each step. Quantification of influence is determined by sensitivity rankings based on the quantities(14) SV(qj)=k=1n1(x3(tk1)qj)2,j=1,2,κ,SC(qj)=k=1n2(x6(tk2)qj)2,j=1,2,κ.(14)

Again, κ denotes the number of model parameters and initial conditions to be estimated.

Initially, we computed sensitivity rankings for all parameters with respect to the viral load and creatinine state variables using (Equation14), where parameters values are chosen as those given in Table . Our findings are given in Table , Appendix 1. Given these results in Table , we identified the parameters that are the most influential in the viral load state variables and the most influential in the creatinine state variables. We subsequently designed an iterative inverse problem algorithm that utilizes the sensitivity rank information to propose parameters sets that we wish to estimate. All sensitivity ranking results for the following algorithm are found in the tables in Appendix 1.

Parameters were estimated using the Matlab routine lsqnonlin. It is important to note that the use of overbar notation when referencing model parameters and initial conditions refers to the original scale. This is done to simplify notation when reporting estimated and fixed parameters within the algorithm. We remind the reader that all simulations were conducted using the log-transformed system with log-scaled parameter values as previously discussed.

Step 1=

We fixed model parameters and initial conditions q¯U1=[λEV,λEK,κEK,κCH,κKH,HS0,HI0,V0,EK0]TR9 using corresponding values found in Table based on sensitivity rankings in Table . Next, we estimated the remaining parameters q¯E1=[λHS,β,δEH,δV,ρ¯EV,δEV,δEK,δC0,κHS,δHI,ρV,κV,λC,ρ¯EK,ϵ1,ϵ2,ϵ3,ϵ4,EV0,C0]TR20 with initial values found in Table and obtained q^E1 using the OLS procedure in (Equation13). Results θopt1=[qU1,q^E1]TR29 are found in Table . We will refer to this experiment as ‘parameters estimation # 1’.

Step 2=

We performed a sensitivity analysis for the 20 estimated parameters in Step 1 using corresponding values in θopt1. The resulting sensitivity rankings are presented in Table . We then fixed an additional 5 parameters chosen from these 20 parameters based on the sensitivity rankings. That is, we fixed 14 model parameters and initial conditions q¯U2=[q¯U1,δEH,κV,ϵ1,EV0,C0]TR14 using corresponding values found in θopt1 . We estimated the remaining parameters q¯E2=[λHS,β,δV,ρ¯EV,δEV,δEK,δC0,κHS,δHI,ρV,λC,ρ¯EK,ϵ2,ϵ3,ϵ4]TR15 with initial values found in θopt1 and obtained q^E2 using OLS procedure. Results θopt2=[qU2,q^E2]TR29 are found in Table . We will refer to this experiment as ‘parameters estimation # 2’.

Step 3=

We performed a sensitivity analysis for the 15 estimated parameters in Step 2 using corresponding values in θopt2. The resulting sensitivity rankings are presented in Table . We then fixed an additional 5 parameters chosen from these 15 parameters based on the sensitivity rankings. That is, we fixed 19 model parameters and initial conditions q¯U3=[q¯U2,λHS,δC0,δHI,λC,ϵ2]TR19 using corresponding values found in θopt2. We then estimated the remaining parameters q¯E3=[β,δV,ρ¯EV,δEV,δEK,κHS,ρV,ρ¯EK,ϵ3,ϵ4]TR10 with initial values found in θopt2 and obtained q^E3 using the OLS procedure. Results θopt3=[qU3,q^E3]TR29 are found in Table . We will refer to this experiment as ‘parameters estimation # 3’.

Step 4=

We performed a sensitivity analysis for the 10 estimated parameters in Step 3 using corresponding values in θopt3. The resulting sensitivity rankings are presented in Table . We then fixed an additional 5 parameters chosen from these 10 parameters based on the sensitivity rankings. That is, we fixed 24 model parameters and initial conditions q¯U4=[q¯U3,δV,κHS,ρV,ϵ3,ϵ4]TR24 using corresponding values found in θopt3. Next, we estimated the remaining parameters q¯E4=[β,ρ¯EV,δEV,δEK,ρ¯EK]TR5 with initial values found in θopt3 and obtained q^E4 using the OLS procedure. Results θopt4=[qU4,q^E4]TR29 are found in Table . We will refer to this experiment as ‘parameters estimation # 4’.

Next, we outline a method to quantify the uncertainty in our parameter estimations. Two methods that have been widely used in the literature to quantify uncertainty in parameter estimates are asymptotic theory and bootstrapping. Both have been investigated and compared in Banks et al. [Citation34] for problems with different form and level of noise. It was found that asymptotic theory is always faster computationally than bootstrapping and there is no clear advantage in using bootstrapping versus asymptotic theory when the constant variance using OLS is assumed. For these reasons, we will use asymptotic theory [Citation26, Citation30] to quantify the uncertainty in our parameter estimations.

We calculated standard errors and confidence intervals [Citation14, Citation27] in order to quantify the uncertainty in parameter estimates. To compute these values, we must define some terms. Recall the statistical model defined in (Equation12). LetF(q)=f1(t11;q),f1(t21;q),,f1(tn11;q),f2(t12;q),f2(t22;q),,f2(tn22;q)T.

Then, the sensitivity matrix χ(q) is an (n1+n2)×κ matrix, where N=n1+n2 is the total number of viral and creatinine data points and κ is the number of estimated parameters, with its (i,j)th element being defined as(15) χij(q)=Fi(q)qj,i=1,2,n1+n2,j=1,2,,κ,(15)

where Fi is the ith element of F, and qj is the jth element of q. Given the data {yi}i=1N={y11,y21,,yn11,y12,y22,yn22} and the resulting parameter estimate q^, the variance σ02 can be approximated by(16) σ02σ^2=1N-κj=1N[yj-Fj(q^)]2.(16)

With these values, we can calculate(17) Σ^(q^)=σ^2[χ(q^)Tχ(q^)]-1.(17)

This matrix is known as the covariance matrix and is used to compute the standard errors for each element of q^ given by(18) SEk(q^)=Σ^kk(q^),k=1,2,κ.(18)

Hence, the 100(1-α)% confidence intervals are given by(19) [q^k-t1-α/2SEk(q^),q^k+t1-α/2SEk(q^)].(19)

We determine t1-α/2 by Prob{Tt1-α/2}=α/2, where T has a student’s t distribution tN-κ with N-κ degrees of freedom.

For the following results, we chose α=0.05. Results obtained for parameter estimation steps # 1, # 2, # 3 are presented in Appendix 2. We observe that parameter estimations # 1-3 produced good fits to the data (as seen in Figures ). However, there is substantial uncertainty in at least some components of the estimates for all three cases (see Tables , , and ). The reliability of parameter estimates depends on the number of parameters estimated. Specifically, maxi|γi| increases as κ increases, where γi denotes the absolute ratio of the standard error to the corresponding estimate for the ith estimated parameter. This is in agreement with common understanding that for a fixed number of observations, the estimation accuracy decreases as the number of parameters increases.[Citation27] Further discussions of these results are found in Appendix 2.

Results corresponding to parameter estimation # 4 associated with estimating 5 parameters are given in Figure , Tables , and . We observe from Figure that we obtain good fits to the data, where the model outputs are obtained using parameter values given in Table . Table illustrates the estimate, the standard error, the 95% confidence intervals and the absolute ratio of the standard error to the corresponding estimate (denoted by γ). From Table , we can see that γi<0.2,i=1,2,5. These parameter estimates for β,ρ¯EV,δEV,δEK,ρ¯EK are reliable. Therefore, by Figure and Table , we may infer that the goodness-to-fit is reasonable.

Figure 2. Using θopt4 in Table for parameter estimation # 4, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Figure 2. Using θopt4 in Table 4 for parameter estimation # 4, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Table 4. Parameter estimation # 4 results for TOS003 viral load and serum creatinine data on [0,450]. Top part (above the horizontal line) of the table gives fixed parameters whereas the bottom part ( below the horizontal line) of the table displays the estimated parameters.

Table 5. Parameter estimates, asymptotic standard errors, confidence intervals and the ratios γ of standard error to the corresponding estimate for parameter estimation # 4.

The next step is then to validate our model with these five free parameters. There are a number of methods in the literature that can be used for model validation (e.g. see [Citation35, Citation36] and the reference therein). These include data-splitting methods, cross-validation methods and bootstrapping methods. Specifically, both data-splitting methods and cross-validation methods use part of the data (training data) for parameter estimation and the rest of the data (test data) for validation. However, bootstrapping involves repeatedly fitting the model using the bootstrapping samples and evaluating the performance of the resulting fitted model with the original data. In other words, bootstrapping samples are used as training data and the original data is used as test data. We thus see that bootstrapping method uses the entire data-set for parameter estimation and hence it is very efficient and is especially useful for the case where one has a limited number of observations or even with a single (one patient only) clinical data-set. Recall that the sample size of our single patient data-set is small. Hence, bootstrapping is chosen to validate our model.

Let Jq;{yj}j=1N=1Nj=1N|yj-Fj(q)|2, where {yj}j=1N={y11,y21,,yn11,y12,y22,yn22},F(q)=(f1(t11;q),f1(t21;q),,f1(tn11;q),f2(t12;q),f2(t22;q),,f2(tn22;q))T and N=n1+n2. The procedures for using bootstrapping to calculate the predication error are given as follows.

(1)

Using the estimate q^ obtained in the parameter estimation # 4 to calculate Jq^;{yj}j=1N.

(2)

(a)

Define the standardized residualsr¯j=NN-κθyj-Fj(q^),j=1,2,3,,N. Set m=1.

(b)

Create a bootstrapping sample of size N using random sampling with replacement from {r¯1,...,r¯N} to form a bootstrapping sample {r1(m),,rN(m)}.

(c)

Create bootstrap sample pointsyj(m)=Fj(q^)+rj(m),j=1,2,3,,N.

(d)

Obtain a new estimate q^(m) from the bootstrapping sample {yj(m)}j=1N using OLS. Calculate J(q^(m);{yj(m)}j=1N) and J(q^(m);{yj}j=1N).

(e)

Set m=m+1 and repeat steps (b)–(d) until m>M (e.g. typically M=200 as in our calculations below).

(3)

Calculate the estimate of optimism (a measure of how much worse our model fits to the new data compared to the training data)O=1Mm=1MJ(q^(m);{yj}j=1N)-m=1MJ(q^(m);{yj(m)}j=1N).

(4)

Calculate the optimism adjusted predication error as J(q^;{yj}j=1N)+O.

Using the above procedures, we found that the optimism is 3.2667×10-4 and the optimism adjusted predication error is 3.4248×10-3. This implies that our model with five free parameters is reasonably validated.

To gain some information on the dynamics of other model states (that is, HS, HI, EV and EK), we plotted these states versus time in Figure , where the results were obtained with parameter values given in Table . From this figure, we observe that the number of infected cells (indicated in the left panel) increases at the very beginning and then decreases until it settles down to a small value. This agrees with the viral load data shown in Figure as the only source for production of new virus is from the infected cells. We also observe that the number of BK-specific CD8+ T cells EV (shown in the right panel of Figure ) increases at the beginning in response to the presence of large amount of free BK virus and then decreases sharply when the viral load decreases to a small level. The trend of increasing-decreasing-increasing of allospecific CD8+ T cells EK (indicated in the right panel of Figure ) perfectly explains the pattern shown in the serum creatinine data (see the right panel of Figure ). This is because EK has a negative effect on the health of the kidney and the damage to the kidney results in increases in the serum creatinine level (as explained in Section 2).

Figure 3. Model solution obtained with parameter values given in Table : (left) results for HS and HI, (right) results for EV and EK.

Figure 3. Model solution obtained with parameter values given in Table 4: (left) results for HS and HI, (right) results for EV and EK.

4. Concluding remarks and future research effort

Our efforts here to develop a mathematical model with statistically based model validation can be properly viewed as an algorithmic approach to determining the information content in a specific experimental data-set. That is, we illustrate one approach to determining how many and which model parameters can be estimated with confidence from a given data-set. In this context, we have developed a mechanistic mathematical model to describe the immune response to both BKV infection and a donor kidney, and have estimated model parameters and initial conditions using the clinical data. Due to the large number of unknown parameters and limited number of observations, one is unable to reliably estimate all the parameters. To alleviate this difficulty, we employed sensitivity analysis combined with asymptotic theory of estimators to determine the number of parameters that can be reliably estimated. Numerical results show that we were able to reliably estimate five parameters with the given eight viral load data points and sixteen creatinine data points. This clearly demonstrates that one requires more informative longitudinal data, particularly observations on other model variables, in order to fully validate the model. In this context, an immediate future effort would be to use optimal design methods (e.g. [Citation37, Citation38]) to determine when an experimenter should take measurements and what model variables to measure. The obtained results should then be used to guide the future data collection efforts.

Once we obtain sufficient amount of informative longitudinal data to reliably estimate all the unknown parameters, another immediate future effort is to quantify uncertainties of model response, which is due to uncertainty in parameter estimators. Based on the asymptotic theory, we know that parameter estimators are multivariate normally distributed. Thus, this would involve uncertainty quantification of the system of ordinary differential equations (Equation9) driven by a multivariate normally distributed random vector (referred to as a random differential equation). To do this, one could use the fact that the probability density function of solutions to random differential equations is described by the integral of the solution to a hyperbolic partial differential equation with integrator being the cumulative distribution function of a multivariate normally distributed random vector (see [Citation27, Section 7.3.2.2] for details). An alternative approach to do this is to use Monte Carlo methods or other numerical methods such as stochastic Galerkin methods and stochastic collocation methods (e.g. see [Citation27, Section 7.3] and the references therein for details).

Acknowledgements

The authors are grateful to the two anonymous referees for their helpful comments and constructive suggestions which led to an improved version of the paper.

Additional information

Funding

This research was supported in part by [grant number NIAID R01AI071915-10] from the National Institute of Allergy and Infectious Diseases, in part by the Air Force Office of Scientific Research under [grant number AFOSR FA9550-12-1-0188], and in part by the National Science Foundation under Research Training Grant (RTG) [DMS-1246991].

References

  • United States department of health and human services, (n.d.). OPTN, SRTR annual report. 2011. p. 8–10. Retrieved May 12, 2014 from the World Wide Web: http://optn.transplant.hrsa.gov.
  • Gardner SD, Field AM, Coleman DV, Hulme B. New human papovavirus (B.K.) isolated from urine after renal transplantation. Lancet. 1971;297:1253–1257.
  • Heritage J, Chesters PM, McCance DJ. The persistence of papovavirus BK DNA sequences in normal human renal tissue. J. Med. Virol. 1981;8:142–150.
  • Hirsch HH. BK virus: opportunity makes a pathogen. Clin. Infect. Dis. 2005;41:354–360.
  • Knowles WA, Pipkin P, Andrews N, Vyse A, Minor P, Brown DW, Miller E. Population-based study of antibody to the human polyomaviruses BKV and JCV and the simian polyomavirus SV40. J. Med. Virol. 2003;71:115–123.
  • Eash S, Querbes W, Atwood WJ. Infection of vero cells by BK virus is dependent on caveolae. J. Virol. 2004;78:11583–11590.
  • Egli A, Binggeli S, Bodaghi S, et al. Cytomegalovirus and polyomavirus BK post-transplant. Nephrol. Dial. Transplant. 2007;22:viii72–viii82.
  • Funk GA, Hirsch HH. From plasma BK viral load to allograft damage: rule of thumb for estimating the intrarenal cytopathic wear. Clin. Infect. Dis. 2009;49:989–990.
  • Funk GA, Steiger J, Hirsch HH. Rapid dynamics of polyomavirus type BK in renal transplant recipients. J. Infect. Dis. 2006;190:80–87.
  • Hammer MH, Brestrich G, Andreee H, et al. HLA type-independent method to monitor polyma BK virus-specific CD4+ and CD8+ T-cell immunity. Am. J. Transplant. 2006;6:625–631.
  • Nickeleit V, Hirsch HH, Binet IF, et al. Polyomavirus infection of renal allograft recipients: from latent infection to manifest disease. J. Am. Soc. Nephrol. 1999;10:1080–1089.
  • Hirsch HH, Steiger J. Polyomavirus BK. Lancet Infect. Dis. 2003;3:611–623.
  • Ramos E, Drachenberg CB, Portocarrero M, et al. BK virus nephrology diagnosis and treatment: experience at the University of Maryland Renal Transplant Program. In: Cecke JM, Terasaki PI, editors. Clinical Transplants. Los Angeles (CA): UCLA Immunogenetics Center; 2002. p. 143–153.
  • Banks HT, Tran HT. Mathematical and experimental modeling of physical and biological processes. Boca Raton (FL): Taylor/Francis-Chapman/Hall-CRC Press; 2009.
  • Funk GA, Gosert R, Comoli P, Ginevri F, Hirsch HH. Polyomavirus BK replication dynamics in vivo and in silico to predict cytopathology and viral clearance in kidney transplants. Am. J. Transplant. 2008;8:2368–2377.
  • Kepler GM, Banks HT, Davidian M, Rosenberg ES. A model for HCMV infection in immunosuppressed patients. Math. Comput. Model. 2005;49:1653–1663.
  • Banks HT, Hu S, Jang T, Kwon HD. Modeling and optimal control of immune response of renal transplant recipients. J. Biol. Dyn. 2012;6:539–567.
  • Levey AS, Bosch JP, Lewis JB, Greene T, Rogers N, Roth D. A more accurate method to estimate glomerular filtration rate from serum creatinine: a new prediction equation. Ann. Intern. Med. 1999;130:461–470.
  • Adams BM, Banks HT, Davidian M, Rosenberg ES. Model fitting and prediction with HIV treatment interruption data. Bull. Math. Biol. 2005;69:563–584.
  • Banks HT, Davidian M, Hu S, Kepler GM, Rosenberg ES. Modelling HIV immune response and validation with clinical data. J. Biol. Dyn. 2008;2:357–385.
  • Herbeck JT, Mittler JE, Gottlieb GS, Mullins JI. An HIV epidemic model based on viral load dynamics: value in assessing empirical trends in HIV virulence and community viral load. PLOS Comput. Biol. 2014;10:e1003673.
  • Herrero-Martinez E, Sabin CA, Evans JG, Griffioen A, Lee CA, Emery VC. The prognostic value of a single hepatitis C virus RNA load measurement taken early after human immunodeficiency virus seroconversion. J. Infect. Dis. 2002;186:470–476.
  • Ouma KN, Basavaraju SV, Okonji JA, Williamson J, Thomas TK, Mills LA, Nkengasong JN, Zeh C. Evaluation of quantification of HIV-1 RNA viral load in plasma and dried blood spots by use of the semiautomated cobas amplicor assay and the fully automated cobas Ampliprep/TaqMan assay, version 2.0, in Kisumu, Kenya. J. Clin. Microbiol. 2013;51:1208–1218.
  • Carrol RJ, Ruppert D. Transformation and weighting in regression. New York (NY): Chapman & Hall; 1988.
  • Davidian M, Giltinan D. Nonlinear models for repeated measurement data. London: Chapman & Hall; 1998.
  • Seber GA, Wild CJ. Nonlinear regression. Hoboken (NJ): Wiley; 2003.
  • Banks HT, Hu S, Thompson WC. Modeling and inverse problems in the presence of uncertainty. Boca Raton (FL): Taylor/Francis-Chapman/Hall-CRC Press; 2014.
  • Randhawa PS, Vats A, Zygmunt D, Swalsky P, Scantlebury V, Shapiro R, Finkelstein S. Quantification of viral DNA in renal allograft tissue from patients with BK virus nephropathy. Transplant. 2002;74:485–488.
  • Banks HT, Cintron-Arias A, Kappel F. Parameter selection methods in inverse problem formulation, CRSC Tech. Report CRSC-TR10-03 (2010), NCSU, Raleigh. In: Batzel JJ, Bachar M, Kappel F, editors. Mathematical modeling and Validation in physiology: application to the cardiovascular and respiratory systems. Lecture notes in mathematics. Vol. 2064, Berlin: Springer-Verlag; 2013. p. 43–73.
  • Banks HT, Davidian M, Samuels JR Jr, Sutton KL. An inverse problem statistical methodology summary, CRSC Tech. Report CRSC-TR08-01, (2008), NCSU, Raleigh. In: Chowell G, Hyman M, Hengartner N, Bettencourt LMA, Castillo-Chavez C, editors. Statistical Estimation Approaches in Epidemiology. Berlin, Heidelberg, New York: Springer; 2009. p. 249–302.
  • Banks HT, Dediu S, Ernstberger SL, Kappel F. Generalized sensitivities and optimal experimental design, center for research in scientific computation report, CRSC-TR08-12 (revised) (2008), NCSU, Raleigh. J. Inverse Ill-Posed Prob. 2010;8:25–83.
  • Banks HT, Holm KJ, Kappel F. Comparison of optimal design methods in inverse problems technical report CRSC-TR10-11 (2010), NCSU, Raleigh. Inverse Prob. 2011;27:075002.
  • Özisik MN, Orlande HRB. Inverse heat transfer: fundamentals and applications. New York (NY): Taylor & Francis; 2000.
  • Banks HT, Holm K, Robbins D. Standard error computations for uncertainty quantification in inverse problems: asymptotic theory vs. bootstrapping, CRSC technical report CRSC-TR09-13, NCSU, May 2010, Math. Comput. Model. 2010;52:1610–1625.
  • Efron B, Tibshirani RJ. An introduction to the bootstrap. Boca Raton (FL): Chapman & Hall/CRC Press; 1998.
  • Steyerberg EW, Harrell FE Jr, Borsboom GJJM, Eijkemans MJC, Vergouwe Y, Habbema JF. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J. Clin. Epidemiol. 2001;54:774–781.
  • Banks HT, Rehm KL. Experimental design for vector output systems, CRSC technical report CRSC-TR12-11, April 2012. Inverse Prob. Sci. Eng. 2013;22:1–34. doi:10.1080/17415977.2013.797973
  • Banks HT, Rehm KL. Experimental design for distributed parameter vector systems, CRSC technical report CRSC-TR12-17, August 2012. Appl. Math. Lett. 2013;26:10–14. http://dx.doi.org/10.1016/j.aml.2012.08.003.

Appendix 1

Results corresponding to the sensitivity analysis conducted during Step 1–Step 4 of the iterative inverse problem algorithm are given in Tables .

Table A1. Sensitivity rankings for 29 estimated parameters.

Table A2. Sensitivity rankings for 10 estimated parameters.

Table A3. Sensitivity rankings for 20 estimated parameters.

Table A4. Sensitivity rankings for 15 estimated parameters.

Appendix 2

Results corresponding to parameter estimation # 1 associated with estimating 20 parameters are found in Figure , Tables and . We can see from Figure that the estimate θopt1 produced relatively good fits for both the viral and serum creatinine data. However, we observe from Table that the ratio of standard errors to the corresponding estimate, γ, is huge for all parameters except C0. This means that the estimates for all the parameters except C0 are not reliable. One possible explanation for this result is the size of the condition number of the Fisher information, χ(q^)Tχ(q^). In this particular case, it is 2.0604×1017, obtained using Matlab function cond. This is considerably large and suggests that the matrix is very close to being singular. This presents an issue when calculating the standard errors, since the inverse of the Fisher matrix is required.

Results corresponding to parameter estimation # 2 associated with estimating 15 parameters are found in Figure , Tables and . Figure also shows relatively good fits for both the viral and serum creatinine data. We observe, however, from Table that γ is large for parameters λHS,β,ρ¯EV,δEV,δEK,δC0,κHS,δHI,ρV,ρ¯EK,ϵ2,ϵ3,ϵ4. Yet, 1<γ<2 for λHS,δC0 and 0.9<γ<1 for λC. We note that maxi|γi|=1074.3, which is much less than that corresponding to parameter estimation # 1. This suggests that there is less uncertainty in the estimates obtained compared to the previous case. It is important to note that the condition number of the Fisher matrix is 1.6620×1011 and has decreased.

Results corresponding to parameter estimation # 3 associated with estimating 10 parameters are given in Figure , Tables and . Figure shows relatively good fits for both the viral and serum creatinine data. However, we observe from Table that γ is large for parameter δV, 1<γ<4 for parameters ρ¯EV,δEV,ρV and 0.4<γ<1 for parameters β,δEK,κHS,ρ¯EK,ϵ3,ϵ4. Note that maxi|γi|=25.924, which is much less than that corresponding to parameter estimation # 1, 2. This means that we have less uncertainty in parameter estimates compared to that of the other two cases. Similar to the previous two cases, the condition number of the Fisher matrix is 1.9733×108 and has decreased.

Figure B1. Using θopt1 in Table for parameter estimation # 1, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Figure B1. Using θopt1 in Table B1 for parameter estimation # 1, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Table B1. Parameter estimation # 1 results for TOS003 viral load and serum creatinine data on [0,450]. Top part (above the horizontal line) of the table gives fixed parameters whereas the bottom part ( below the horizontal line) of the table displays the estimated parameters.

Table B2. Parameter estimates, asymptotic standard errors, confidence intervals and the ratios γ of standard error to the corresponding estimate for parameter estimation # 1.

Figure B2. Using θopt2 in Table for parameter estimation # 2, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Figure B2. Using θopt2 in Table B3 for parameter estimation # 2, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Table B3. Parameter estimation # 2 results for TOS003 viral load and serum creatinine data on [0,450]. Top part (above the horizontal line) of the table gives fixed parameters whereas the bottom part ( below the horizontal line) of the table displays the estimated parameters.

Table B4. Parameter estimates, asymptotic standard errors, confidence intervals and the ratios γ of standard error to the corresponding estimate for parameter estimation # 2.

Figure B3. Using θopt3 in Table for parameter estimation # 3, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Figure B3. Using θopt3 in Table B5 for parameter estimation # 3, (Left) Plot of model solution and viral load data on [0,450]. (Right) Plot of model solution and serum creatinine data on [0,450].

Table B5. Parameter estimation # 3 results for TOS003 viral load and serum creatinine data on [0,450]. Top part (above the horizontal line) of the table gives fixed parameters whereas the bottom part ( below the horizontal line) of the table displays the estimated parameters.

Table B6. Parameter estimates, asymptotic standard errors, confidence intervals and the ratios γ of standard error to the corresponding estimate for parameter estimation # 3.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.