6,169
Views
0
CrossRef citations to date
0
Altmetric
Research Article

A systems approach to a resilience assessment for agility

ORCID Icon
Pages 955-964 | Received 20 Sep 2022, Accepted 12 Nov 2022, Published online: 16 Dec 2022

Abstract

This work proposes a theoretical approach to assessing agility in terms of a modified version of resilience during large-scale crisis to sustain operational reliability. The proposed method could be used on subsystem optimization or eventually scaled up to global interconnectedness enabling decision makers to optimize resource allocation and so obtain resilience and agility in troubling times along with long-term sustained prosperity. Introducing weights to various parameters can also allow customizing outcomes such as insuring equitable outcomes, environmental stewardship and proper response to emergencies or any national crisis. The provided mathematical formalism can then become a decision maker tool to predict corrective action outcomes from various responses to a crisis or alternatively to determine sensitivity and potential risk for a crisis from apparently ambient or slowly changing conditions. Ad-hoc examples are considered to demonstrate the generality of the approach.

Glossary

Agility – The degree to which the second derivative of the system metric is close to zero, a kind of system auto-recovery.

Outcome risk – The first derivative of a metric for expected system conditions in a steady state or empirical condition.

Pliable states – These are input conditions which serve as having the potential to be biased by decision makers or some portion of society at large. The extent to which they are controlled or effected will bias actual driving decisions made by those in authority or some other collective societal behaviour that can be characterized in a categorical manner.

Resilience – The degree to which the first derivative of the system metric is close to zero.

Sensitivity parameter – A correlation estimate between 2 potentially disparate categorical data values

Uncontrollable states – These are driver or trigger actions which are considered ‘acts of nature’ such as hurricanes, earthquakes and other actions approximately outside the realm of local control which can challenge system agility and so be of interest for sensitivity analysis and risk assessment.

Unique drivers – These are those input conditions which may or may not be controllable but are distinct and to some extent efficacious in perturbing outcome risk.

1. Background

The National Academy of Engineering (2020) recently went on record pointing out that a novel holistic approach is required to mitigate, or even have any reasonable hope to tolerate future unexpected catastrophes (Madhavan et al. Citation2020). Here they clarify the requisite development of incorporating resiliency in our Systems of Systems (SoS) by front-end investments to mitigate unpredictable upset conditions of national and international interest. This is to say that covid-19 is not the only unexpected upset we will experience from an unknown suite of future insults to the SoS we rely on, not only to survive, but also to thrive and be sustainable.

As such, this work aims to place a mathematical framework on predicting how complex anthropogenic systems (e.g. food, energy, economy, health care, manufacturing etc.) all interact such that optimal bias on adjustable inputs can be applied to improve system resiliency. By defining an agility metric for withstanding large assaults on any one or more interconnected systems, the approach allows an optimization approach to be considered. This work is primarily just establishing the technical basis to eventually approach such a solution from a theoretical perspective.

Traditionally, probabilistic risk assessment (PRA) has been done using Fault Tree Analysis, HazOp or What if analyses (Keller & Modarres, Citation2005). Alternative methods include dynamic PRA (Mandelli et al., Citation2020) and Monte Carlo methods (Kelly & Smith, Citation2009) which have been applied to single industrial endeavours. In an attempt to make further progress in this direction for any SoS, a requisite ‘agility’ metric will be defined to allow predictive bias from decision maker actions. Here, this will be done on a contrived ad-hoc system to demonstrate the generality of the proposed approach. The eventual intent really being validation of the concept to later enable an approach for integrating agility into our larger societal SoS. More specifically, the intent here is to provide a tangible model for evaluating an arbitrary systems’ agility to mitigate and recover from an unexpected or previously unexperienced combination of inputs resulting in any form of catastrophe (public health, economic, supply chain, social disorder, etc).

Multiple methods have been proposed for these kind of resiliency metrics. The first of these methods (Johansson & Hassel, Citation2010) attempted to use a deterministic topological method for essential systems in railway systems. This approach looks at the number of links and nodes for an interconnected system. By following node failures caused by any single node taking down only attached nodes, propagation of successive failures can be approximated. Another method (Duenas-Osorio et al., Citation2007) attempted a probabilistic topological network where critical infrastructure and systems looked at all components of commodity flow to determine interdependencies of the same. This is Similar to that by Johansson and Hassel (Citation2010), in that only connected nodes can fail but this method has conditional connected node failure criteria which is probabilistic in nature based on expert or historical information. Yet another similar flow based model for assessing resilience by Lee et al. (Citation2007) gave each node a set of flow and capacity parameters to further predict dependency on connected nodes. This allows rerouting or limited service options to be realized and considered. Finally, resilience has also been evaluated by Trucco et al. (Citation2012) in not only considering flow and capacity for each node, but to look at minimum and maximum values and allow total performance to be considered accordingly. All of these methods have found application in industry and are reviewed accordingly by Mao and Li (Citation2018) providing multiple definitions of resilience, all conforming to some concept of maintaining the status quo under upset conditions.

System modeling theory does not only have to consider resiliency and/or agility, it can apply to manufacturing systems where optimal throughput, scheduling with potential waste, cost and energy minimization. These approaches can range from artificial intelligence and numerical modeling to data analytics and have been reviewed by Pistikopoulos et al. (Citation2021). When including interdependencies which involve software considerations and other embedded systems, model based system engineering has many options which are also thoroughly reviewed by Rashid et al. (Citation2015) although the current work will not be calling out these facets at this time. Perhaps the most similar efforts to that proposed here focus on the target SoS by Nielsen et al. (Citation2015). Here, an SoS is typically assumed to have both boundaries and parameter independence (neither of which is a-priori required for the present work) but the various approaches are diverse and often system specific.

Other research has looked specifically at agile system engineering (Abrahamsson et al., Citation2009; Dove, Citation2005; Hummel et al., Citation2013; & Turner, Citation2007) but unlike all these other methods, the proposed approach in this work is highly linearized, entirely novel and yet to find real-world application. Literature reviews on agility include computer software (Erickson et al., Citation2005; Pereira & de FSM Russo, Citation2018), manufacturing (Gunasekaran, Citation1999; Sanchez & Nagi, Citation2001) and transportation (Takahashi & Nakamura, Citation2000, Wu & Barnes, Citation2011) are all available. Any of these can be folded up into larger systems of systems which also have conceptual reviews available (Gorod et al., Citation2008; Yin et al., Citation2019, Anacker et al., Citation2022). The present work proposes combining an agility metric with a generic system of systems approach which is unique to that previously found in the literature. The methods described by Nakagawa et al. (Citation2013) include a variety of approaches which do not focus on agility whereas Chiprianov et al. (Citation2014) do focus on the related topic of security, which does feed into an important component of agility as defined in this work, still, these do not encompass all aspects of agility. The present work is intended to introduce a novel approach to optimizing overall agility in any arbitrary SoS.

2. Theory

To simplify the construct, let’s initially consider the sensitivity dependence of a system comprising only 3 initial conditions or uncontrollable states, Us and another 3 pliable states Ps which would result in some risk of a particular failure mode of interest. This contribution to the literature for agility in decision making is considered novel and so will only be worked out in theory with testing and validation left for later efforts.

To begin with, lets take an example set of uncontrollable initial conditions given by Us which represent particular scenarios such as any selection of natural disasters, overseas trade embargo(s) or foreign wars etc. Basically anything outside the control of actions taken within the purview of control from the state or smaller entity encompassing the SoS of interest. That trade embargoes may be initiated by independent local actions is accepted but the grouping is convenient to discriminate strong from weak local control parameters

The potential pliable states (Ps) of these state variables are given by Psn which represent those states we can influence directly if not completely. In this sense, Ps1 might be a large US troop deployment, Ps2 might be a policy changes (embargo, federal interest rates), and so on. With this, attention is warranted to correlate the potentially high dimensional interdependency of controllable responses to situational driver events.

Any mismatch in dimensionality of Us and Ps must then be caught by their interdependent sensitivities S as given by Equation Equation1. The multi-dimensionality, data-type mismatch challenge (some variables will be naturally geospatial, others political, others market, economies, etc.) will be addressed later as a part of this very limited scope on a small subsystem of any SoS. This will require reasonable estimation of the values, the cross-correlation between these variable system states as shown in Equation Equation1 and ideally also their uncertainties. From this approach, the weaknesses and strengths can be identified to enable investments or changes of any kind in the Ps elements to make our societal resilience and agility robust and effective. (1) outcome risk=ddt(PsSUs)=ddt[(Ps1Ps2Ps3)(S11S12S13S21S22S23S31S32S33)(Us1Us2Us3)](1)

2.1. Risk metrics

The desired result from Equation Equation1 is an outcome risk value close to zero in that all conditions do not change much with changes to the state vector components, such a system would then be considered robust and therefore resilient (things continue largely without change). This alone is a noble goal, resilience, but another metric can be determined which is the time derivative of the outcome risk as shown in Equation Equation2. The desire being that agility is high when the time rate of change of outcome risk is low. This agility metric again would be desired to be zero for an agile system. If for any reason, the outcome risk can change quickly from any combination of state vector components, then the system would be lacking agility and so warranting investment consideration of some form to increase overall system agility. (2) agility=(ddtoutcome risk)1(2) This all results in various risk-related metrics whose variability or potential to give high risk can be mitigated through identification of those needed response states Psn via planning and proper investment by decision makers in things such as resources, training and infrastructure (or possibly just redundancy). Given the nature of Equations 1 & 2, the scale in actual use would be amenable to logarithmic levels but for the remainder of this analysis, the forms will remain as presented above.

2.2. State vectors

Each state vector Psi represents a single value as used in Equation Equation1. Any one or more of these could come from subsystems of the overall system being represented by the outcome risk. These state vectors would not utilize the time derivative but rather represent their current condition with the time derivative only coming in when outcome risk or agility is to be assessed. In this way, the methodology could be scaled down to as fine of detail as one has data to support and theoretically scaled up to a global system of systems. An example might be if the system being assessed for outcome risk was car manufacturing in the USA, then each PSi might be a specific manufacturer. Each manufacturer will have their own associated outcome risk based on subsystem state variables such as union issues (strikes), stock on hand, factory maintenance, profit margins, etc which will be different for each manufacturer and so changing the risk they fold into the overall outcome risk.

In principle, the uncontrollable states Usi do not have to be truly uncontrollable such as solar flares or asteroid impacts. Rather, they could be insensitive to small decisions or agility investment. Examples include climate change, international commerce or global health (e.g. pandemics). The elements of these state vectors are intended to give a large amount of independence between the Psi and the Usi but each vector could contain all elements of Us and Ps with appropriate unity entries in the sensitivity matrix to accommodate this modification.

If the outcome risk is defined by Equation Equation1, then perturbation analysis on the pliable states credible variations will enable an optimization to minimize risk contributions from this value. However, the pliable states might be linearly dependent on other specific variables a for some small range although they will more likely have some nonlinearity (many not even being continuous) in at least some cases so this functionality can be represented by a(β) where β could be highly multivariate but will only be represented by single variables in this description as shown in Equation Equation3. Here, cross correlations are assumed zero for simplicity although this is not required. In a simple description, if α2 is an average home loan interest rate, then β2 could be average credit scores, annual salary, home cost etc. As an example of higher dimensionality, if α1 is foreign nation sentiment on US troop deployments, the β1 variable could be the number of these countries or alternatively a weighted average depending on country status such as global economic status etc. The effects from some change in the federal reserve, foreign policy or stimulus etc. can then all be assessed using the metric from Equation Equation1 to then identify potential resiliency for input state conditions and responses. (3) Ps=(α(β1)1000α(β2)2000α(β3)3)(Ps1Ps2Ps3)(3) Another way of looking at Equation Equation3 is to recognize that depending on the scale of systems being modeled by Equation Equation1, finer detail can be infused into any component of the state vectors using Equation Equation3 instead of just adding new columns and rows into Equation Equation1. More specifically this enables greater parametric detail to incorporate the potential for one or more components to be defined as functionally dependent on others rather than relying strictly on the sensitivity matrix to correlate all contributing states. If the pliable states in Equation Equation3 were advertising platforms (e.g. social media, television and billboards) then the vectors a could be cost effectiveness metrics for each where those are all dependent on variables β such as your ability to target viable customers (β1), your ability to generate quality content (β2, video or photo etc.) and longevity utility (e.g. transience or other quality metric) of the selected media (β3, will it retain long term viability and applicability such as a social media influencer/creator keeping a sales video up on their social media platform or be shared on YouTube due to entertainment content etc.). Other options of higher dimensionality are actually expected for a comprehensive approach where cross terms will generally be non-zero unlike the simplified version shown in Equation Equation3. This might also be used to correlate one or more pliable states to others.

2.3. Sensitivity matrix

Here, the elements of the sensitivity matrix S could be initialized using expert estimates from historical data categories of Us and Ps. Preferably, big data would inform such variables but much of this could be proprietary for commercial systems. Note the actual matrix represented by Equation Equation1 would realistically have substantially higher dimensionality that that shown here.

With the sensitivities expressed by Equation Equation1 in the elements of the matrix operator S, we can individually assess these cross term dependencies (the Sm,n element would be a proportionality metric of the Usn external initial condition state to the controllable Psm state). The initial estimate for the sensitivity coefficients in S would be the proportionalities between the Us and Ps state vector components.

The sensitivities Si,j could even be broken down to include covariances C, such that Sij=m=1maimCijbjm for any appropriate set of status or system interdependent properties a and b. When these inputs (e.g. geospatial dependent societal or environmental variables, political or technological events, natural catastrophic upsets, gradual biases such as climate change conditions, etc.) and outputs (e.g. loss of electricity, stock market upsets, energy prices, etc.). This approach would enable avoiding functional dependencies as espoused by Equation Equation3 at the cost of a larger matrix in Equation Equation1.

2.4. Systems of systems (SoS)

Furthermore, many outputs could become further inputs to follow-on events which might not start out as catastrophic events (a financial crisis) but could evolve into a catastrophe to the extent that they can trigger high consequence events, within or across any SoS, depending on system variables at that time. This is largely identical to using each element of a state vector as the antiderivative of the outcome risk for that subsystem. As an example, if the PsSUs term in Equation Equation1 represents the electrical grid supply in Texas, any given state vector element of the Us or Ps vectors could then be some basic facet of that energy mix such as that provided only by nuclear. Here, the Texas grid supply from nuclear would then be its own system of systems such that one of these pliable states Psm could be given by Psm=PmSmUm where the pliable states for the Texas nuclear grid Pm would then be composed of such elements as fuel cost, time for core reloading, maintenance etc. The unpliable states Um then might include, grid accessibility, renewables output, grid demand, next scheduled refueling campaign etc. Many of these elements would again be SoS such as fuel cost, maintenance and grid demand so that the model scale can encompass the very small and the very large. Like in Equation Equation1, the sensitivity matrix Sm here for Texas electricity does not need to be a square matrix but scales in rows and columns accordingly.

2.5. Time dependency

The time dependency can be factored into the sensitivity matrix itself, such that each Sij corresponds to the specific time interval expected for the defined variables. In this way, if Psi is an annual tax rate on food and Umj is the resulting demand on a staple processed food ingredient, the sensitivity matrix element correlating these would be the Sij annual predicted demand per tax rate. In other words, Sij would estimate if the consumers demand on a particular staple would scale with the annual tax rate on that item, and to what extent. This time resolution may be too large for some effects which might have a much more rapid evolution. With this, either smaller time steps can be taken or nonlinear models could be considered.

Rather than fixing the sensitivity matrix on uniform time intervals, we could utilize a highly simplified (linear) time propagator function of the form outcome risk(Δt)=(UsS(Δt)Ps) where S(Δt) can take on any generic form of S(Δt)=ST, where the matrix element St)ij = SinTmj such that the time matrix T would fold in the various individual proportional time dependent sensitivity components between state variables Us and Ps. In this way, successive multiplications by the time matrix will propagate the risk prediction forward in time accordingly. Alternatively, one could model these changes with a linear or even quadratic temporal variable such that Sim(t)=ϱimt2+ςimt+φim for appropriate constant coefficients of ϱ, ς, and φ (here assuming t0 = 0 s.t. Sim(Δt)=Sim(t)), this could simplify the time derivatives of PsSUs. Of course, other individual functional approaches can be used if they cannot be adequately approximated by a quadratic in the time interval Δt.

2.6. Data types

The state variable components of Us and Ps are expected to take all categorical forms spanning binary, ordinal, various nominal categories, integer, real etc. Example types of the Us and Ps categories include political party changes (or remaining in power) and their geospatial and rank distributions, changing government policies (foreign and domestic), arising regional conflicts, or production method and supply chain changes, technology shifts, cryptocurrency adoptions, etc. In all of these instances, the same model can be used by replacing elements of S with appropriate categories of the solution to Equation Equation1. Clearly, not all scenarios of interest involve upsetting conditions of national interest (and some would expectedly be subtle) but if a large comprehensive SoS represented by Equation Equation1 were properly developed, this could substantially improve multiscale optimization including desired equitable outcomes.

The end goal would then be to expand this approach from theoretical to a real SoS or eventually even all SoS’s. If this can be used to incorporate all of societal variables, it would enable predicting and mitigating what previously might have been considered unpredictable or even incredible for decision makers or the public at large (such as supply chain issues from covid or even covid itself).

2.7. Uniqueness and correlation

The selection of elements for the sensitivity matrix must be carefully chosen to insure uniqueness, particularly if nonlinear temporal variable dependencies are utilized (sec 2.5). Depending on volume and quality in selected data sets, it may also be possible to result in an over or under-fitted solution regarding the parameters of interest having sufficient definition and diversity to span the space despite potentially disparate variable types.

If the matrix elements of S span the space, then the number of independent variables being modeled needs to be equivalent to the rank of the matrix in order for it not to be over or underdetermined. In other words, all columns of this n × n matrix must be linearly independent or equivalently, the matrix must have a non-zero determinant. This can mean that two or more parameters could have some codependency provided sufficient functionally dependent variables are present incorporating all parameters in the space. If this is attained, then the formalism of using the vector product as a single number for use in Equation Equation1 forces a properly meaningful solution provided all elements of Equation Equation1 are real which in principle guarantees existence of an agility metric (Equation Equation2) despite the potentially vast diversity in dimensions and event data categories of each vector element.

Not only are there differences in dimension and data categories, some parameters may be highly discontinuous or disjoint given the potential categories or outcomes associated with those selections. Examples may include effects from climate change or declaration of war etc. where extremely rapid changes can occur over a very small time after following a gradual dynamic phase. The process of accounting for these dependencies would have to occur on a case by case basis but does pose a clear potential weakness of the approach if not properly accounted for and so warrants considerable attention when constructing the sensitivity matrix. Other issues which may be encountered include multiple iterative discontinuities in data or simply incorporating chaotic data which would not correlate easily with causality. All of these aspects are topics for future research in this new systems modeling approach.

3. Customizing impacts

The proposed concept will allow apparently disparate data forms to contribute to the improved modeling of any complex SoS. These data types can be demographics (including religious, political and other worldview metrics), production (and consumption rates), regional, national and global etc. Without knowing or even considering the sensitivity from all such inputs, the predictability of any SoS may be inadequate for practical utility. By tolerating any and all credible inputs, the system complexity can be expanded to a very high dimensionality until its reliability could be benchmarked based on historical data and so potentially updated and validated for use. Given the requisite computing capabilities for Equation Equation1 (even with vast dimensionality), is within reason for modern systems. Two trivial examples will be given using only a 3 × 3 system with only one walked through parametrically.

3.1. Simplified Ad-Hoc example

By recognizing the potential for this risk assessment tool to be used by decision makers for optimizing an outcome based on biasing the pliable states Psi, these state vector elements could be weighted accordingly for that optimized result. As an example, consider a definition of the base case being a predictable and stable economy following historical long term trends. Here, Equation Equation1 might (in this example remaining drastically truncated to a 3 × 3 system) for simplicity, have some example elements. In this example, an ad-hoc array of uncontrollable inputs (Ui) of U1 = date (time of year and day), U2 = average rainfall (cm/y) and U3 = average wind magnitude (m/s) with the example pliable states being P1 = food production (MT/day), P2 = current traffic rates (cars/km) and P3 = Federal Reserve interest rates (%) as shown in Equation Equation4. The values would be chosen for a specific year to allow the time derivative in Equation Equation1 to apply. The solution here is trivial as the real power comes in obtaining good sensitivity coefficients. If then the sensitivity coefficients were normalized such that i,jSij=ϑ, then the relative changes can be obtained by multiplying through with ϑ1. (4) (PsSUs)=(MT/day, cars/km, %)date1MT/dayy/cmMT/days/mMT/daydate1cars/kmy/cmcars/kms/mcars/kmdate1%y/cm%s/m%(Julian datecm/ym/s)(4) Note that the purpose of Equation Equation4 is only to demonstrate how the vast array of dimensionality in parameters of interest readily combine using a properly defined sensitivity matrix. What is not shown are the magnitudes of the sensitivity matrix as those would initially require expert estimates to then be refined by various methods such as machine learning using big data.

Taking time derivatives of Equation Equation4 is straightforward using the definitions given in section 2.5 but if the sensitivity coefficients do not have explicit functional time dependence, and only annual averages are available as assumed for Equation Equation4, then the time derivative will have no effect if it is only rate of change per year. Taking a derivative on a different interval will require other means such as definitions described in 2.5.

This ad hoc example was specifically chosen to emulate how both uncontrollable and pliable states can have substantially heterogeneous variability. Furthermore, the definition of an uncontrollable state may be used in a fluid sense if it were pliable but substantially more stable than the defined pliable states such that the effect of one transient pliable state could be used to predict its bias on a more stable pliable state but again, both vectors could be combined requiring appropriate unity elements in S.

Now the real world array dimensionality would be very much larger than this 3 × 3 state system but this limited case will serve as an adequate example. Placing weights on pliable states can then allow optimization by decision makers to incorporate equity in select outcomes.

3.2. Uncertainties

Of additional interest to the sensitivity elements in Equation Equation4, are their contributing uncertainties as these could really drive the model significance, utility and applicability. The initial conditions for this system would have the generic ranges or confidence intervals listed in Table . The resultant sensitivities will have comparable variability in range as the states and so defining a base case will allow comparisons for variability

Table 1. Hypothetical state vector ranges for assessing outcome risk uncertainty, note these range values are assumed to be confidence limits appropriate to whatever distribution type is present in the state vector component.

Even with the type of ranges given in Table , the distribution types are not already known but can initially be estimated by expert assessments (although these would preferably come from fits to historical data or machine learning assuming adequate database sets are available). Examples of public databases are available for such training sets [5,6,7] but proprietary large data sets could entirely limit the power from machine learning in finding all such sensitivity components of interest (hence the need for expert estimates prior to large data availability). By inspection, the magnitude of some sensitivities (those tied to the last column in Table ) would be quite small as this ad-hoc choice of parameters are not intended to be highly correlated.

Still, these sensitivity magnitudes (Sij) could be made ever more complex if desired by incorporating additional dependencies such as annual interest rates for various loans a and/or biennial wage averages for various groups of individuals b through Sij=m=1maimCijbjm such that there is always sufficient definition flexibility to insure appropriate modeling to potential functional dependencies.

Using this approach, the actual numerical value from Equation Equation1 may take highly varied outcomes depending on the state vectors chosen and their associated time intervals or dependencies (column and row headers in Table ). This can be normalized to give the formal base case a value of unity by multiplying the right-hand side of Equation Equation1 with (i,jPimSijUjm)1. This does require the nominal values are properly estimated. This would result in a normalized risk estimate given by Equation Equation5 which can be perturbed to assess system agility as described in Section 4. (5) normalized risk=ddt(Ps(α)1Ps(α)2Ps(α)3)(S11S12S13S21S22S23S31S32S33)(Ui1Ui2Ui3)/i,jPimSijUjm(5)

3.3. Generic example for US nuclear energy

Effectively this example can start with just the first-order proportionalities such as the parameter Ps1 being the number of US corporations with reactor designs submitted to the Nuclear Regulatory Commission for regulatory approval to construct and operate a nuclear reactor, its proportionality α1 to the outcome state Uj of that company obtaining adequate investors to provide the financing would be nonzero but positive. The range of the Uj parameter being [−1,1] with −1 being absolutely not obtaining adequate financing and a + 1 being a certainty that they would build and operate a nuclear reactor. This parameter can itself be dependent on other parameters of the form a(β) such that the βi values would be a linear combination of such things as the energy needs of any given state or region over the time period of interest, the credit rating of that company (to procure financing), the social bias on nuclear energy and so forth. This would cycle over all potential siting locations and principalities which might procure a reactor and then the rest of the αi parameters would have to be developed similarly. Clearly, this would be a large list of parameters requiring substantial effort to generate a spanning set. Even after this portion was completed, this would generate a single input for sustainability of US nuclear reactor designs, manufacture and operations as it does not force domestic retention, political regulatory changes or new fuel cycle developments.

4. Uncertainty, limitations and sensitivity assessment

An essential element of the simplest hypothesis testing is the variance (σ2) of a parameter x¯ to be compared or assessed for any statistically significant differences to some standard value μ when cast as a z-score or z=(x¯μ)/σ. In order to obtain this variance approximation for the normalized risk, one can use a Monte Carlo approach to determine the total propagated uncertainty based on estimated individual parameter means and variances. To do this, each element of Ps, both the means, variances for each input type can be estimated based on expertise or preferably big data if accessible as given in Table .

4.1. Markov chain Monte Carlo theory for generic nuclear example

Still, from the particular example input range (that local utilities would procure US reactors), the initial parameter might be estimated to be 0.2. The rest of the Sij parameters would be independently perturbed according to their Table equivalent ranges and distribution types allowing a large set of outcome risks to be calculated via Equation Equation1. Doing this a few 10’s of thousands of times will give a clear distribution with its own mean, standard deviation σ, skew etc.

How then the sales of US reactor designs would affect the outcome risk for US nuclear energy would be factored in to include maintaining native talent, manufacturing, maintenance and so forth in such a way as to contribute to the desired resultant metric (defined in Section 4.3). Assuming this were taken out to include all dependencies for US nuclear energy, this then might be used as an input to the overall US energy mix for a larger SoS. The energy SoS could then be folded into a larger US SoS to include all anthropogenic systems known (e.g. economy, health care, food, manufacturing, distribution etc). Although a very large set, it would be finite but the question remains as to how one would obtain a complete basis set.

4.2. Expanding the set to span the range

Obtaining sufficient values for the Psi values to span the entire space may require using nonorthogonal values so that many basis vectors will have nonzero projections. Examples might be various economic parameters such as inflation, wage increases and consumer spending (which would expectedly have codependencies). These would all include various aspects of new growth financing dependencies for various industrial and commercial ventures but would also be potentially strongly correlated making them have some redundancy in Equation Equation1. This does not mean that the independent aspects of these measures cannot be further refined but by definition, so long as the range is spanned by the basis vectors, the solution to Equation Equation1 will not have degeneracies.

The resulting dimensionality of the system of equations will be expectedly large and so generating a complete set is expected to take some time and include clearly needed development criticism, guided instruction and of course thorough independent review. The very thoughtful and deliberate accounting for these variables, their range, mean and variance will all have to be documented, defended and then account for dynamic changes to include later optimization. One risk in this approach is that a model which predicts historical values well would not necessarily include all variables which can affect future systems. In other words, the dominant variables in one time period may not dominate all future time intervals.

4.3. Alternative agility metric

As given in Equation Equation2, the agility scales with ever improving the potential for outcome risk to have negligible dependencies on state variable changes. An alternative metric for the agility would be inversely proportional to this resulting standard deviation σ obtained above (section 4.1), meaning σ(PsSUs) is minimized such that if a large range of outputs are possible, then predictability is low. By maximizing predictability, then the bias to desired outcomes becomes tenable. The end goal will be to understand which of these complex dependencies will allow biased (promoting social equity, sustainability and being environmentally friendly) predictable outcomes. The obvious outcomes desired would be reliable services, cost competitiveness and no accidents but this can be expanded to include social justice values as well. This will allow determining the initial normalization factor N for the agility metric given in Equation Equation6. In principle, there should be a very strong correlation between the agility definitions of Equations 2 & 6 and the choice will likely depend on the tools used to obtain the parameters. (6) agility~ N/σ(PsSUs)(6) With this, the eventual intent would be to demonstrate that by adjusting the Ps values, the maximization of the SoS agility can be obtained with a proscribed set of Ps changes for a decision maker support tool. Perhaps far more important are the investments required to adjust the sensitivity factors Sij to increase agility as given by Equations 2 & 6.

This process would allow a focus on identifying cost effective investments into biasing the sensitivities S and possibly even the uncontrollable states Us to further improve SoS agility. In order to attain equality in Equation Equation7, we will have to do the optimization process of identifying investments which will cost effectively maximize that value. This can include social programmes (education, housing, financing opportunities etc.), infrastructure (roads, electrical grid, manufacturing etc.), planning (federal and state training, online guides, communication improvements etc.) such that the largest gain in agility for the least amount of money in the shortest time can be obtained. (7) agility~=opt[N/σ(PsSUs)](7) In principle then, this will incorporate any and all upset conditions, credible and incredible. The initial approach for scaling a time to a monetary equivalent for the optimization process of Equation Equation7 might be that of an annual interest payment on a large-scale energy facility build which is delayed by a year or 1 yr ≈ 1E9$ which is incurred by the large initial investment of 1E10$ at a 10% annual interest rate (the real case is far more complicated but this should suffice for a reasonably scaled metric). With this approach, the time dependence for making any changes can be converted to money as a means to visualize the most cost effective methods for quickly improving SoS agility.

5. Discussion and conclusions

This product could ultimately form the technical basis for a grand-scale SoS encompassing all potential risks to society. By considering a pair of trivial ad-hoc examples, the concept to test and evaluate the challenges and opportunities to provide agility to our SoS has been described in the hopes that future work in this effort will allow upset events to be weathered with little or no impact. Further, the method may allow for optimizing long-term planning. This does not minimize the remaining importance of evaluating the application of this method to chaotic or discontinuous data which will require further research to mitigate or even incorporate. These will remain future topics for further research.

An example of readily accessible next steps in this work include small energy or economic sectors in terms of societal risks from weakly and strongly dependent states but eventually would be applied to any SoS subsystem. Particularly the overarching SoS components of food, economy, public health, manufacturing, distribution and all of energy. If the method is properly successful in this sense, it could also serve to optimize sustainability and societal standard of living based on pliable state selections.

The high dimensionality for the system may be simplified using machine learning on subsystems already preconditioned via the form in Equation Equation1 if they were made available for such an effort. As an example, the following list of systems could generate an outcome risk value to then be used as a basis element in a larger SoS again using Equation Equation1; finance, accounting, engineering, marketing, law, information management, information retrieval, stock trading, network management, telecommunications, education, medicine, chemistry, human resources management, human capital, business, production management, economics, energy, and defence. In principle, the method could be intensely powerful or just as useless as its inputs if not set up properly.

Acknowledgments

This work was produced NCSU and Battelle Savannah River Alliance, LLC under Contract No. 89303321CEM000080 and/or a predecessor contract with the U.S. Department of Energy.

Disclosure statement

No potential conflict of interest was reported by the author(s).

References

  • Abrahamsson, P., Conboy, K., & Wang, X. (2009). ‘Lots done, more to do’: the current state of agile systems development research. European Journal of Information Systems, 18(4), 281–284. https://doi.org/10.1057/ejis.2009.27
  • Anacker, H., Günther, M., Wyrwich, F., & Dumitrescu, R. (2022, June). Pattern based engineering of System of Systems-a systematic literature review. In 2022 17th annual system of systems engineering conference (SOSE) (pp. 178–183). IEEE.
  • Chiprianov, V., Gallon, L., Munier, M., Aniorte, P., & Lalanne, V. (2014, June). Challenges in security engineering of systems-of-systems. In Troisième Conférence en IngénieriE du Logiciel (pp. 143).
  • Dove, R. (2005, March). Fundamental principles for agile systems engineering. In Conference on systems engineering research (CSER). Stevens Institute of Technology.
  • Duenas-Osorio, L., Craig, J. I., Goodno, B. J., & Bostrom, A. (2007). Interdependent response of networked systems. Journal of Infrastructure Systems, 13(3), 185–194. https://doi.org/10.1061/(ASCE)1076-0342(2007)13:3(185)
  • Erickson, J., Lyytinen, K., & Siau, K. (2005). Agile modeling, agile software development, and extreme programming. Journal of Database Management, 16(4), 88–100. https://doi.org/10.4018/jdm.2005100105
  • Gorod, A., Sauser, B., & Boardman, J. (2008). System-of-systems engineering management: A review of modern history and a path forward. IEEE Systems Journal, 2(4), 484–499. https://doi.org/10.1109/JSYST.2008.2007163
  • Gunasekaran, A. (1999). Agile manufacturing: a framework for research and development. International Journal of Production Economics, 62(1-2), 87–105. https://doi.org/10.1016/S0925-5273(98)00222-9
  • Hummel, M., Rosenkranz, C. and Holten, R., 2013. The role of communication in agile systems development. Business & Information Systems Engineering, 5(5), 343–355. https://doi.org/10.1007/s12599-013-0282-4
  • Johansson, J., & Hassel, H. (2010). An approach for modelling interdependent infrastructures in the context of vulnerability analysis. Reliability Engineering & System Safety, 95(12), 1335–1344. https://doi.org/10.1016/j.ress.2010.06.010
  • Keller W, Modarres M. (2005). A historical overview of probabilistic risk assessment development and its use in the nuclear power industry: a tribute to the late Professor Norman Carl Rasmussen, Reliability Engineering & System Safety, 89(3), 271-285. https://doi.org/10.1016/j.ress.2004.08.022
  • Kelly, D. L., & Smith, C. L. (2009). Bayesian inference in probabilistic risk assessment—The current state of the art. Reliability Engineering & System Safety, 94(2), 628–643. https://doi.org/10.1016/j.ress.2008.07.002
  • Lee II, E. E., Mitchell, J. E., & Wallace, W. A. (2007). Restoration of services in interdependent infrastructure systems: a network flows approach. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 37(6), 1303–1317. https://doi.org/10.1109/TSMCC.2007.905859
  • Madhavan, G., Poste, G., & Rouse, W. (2020). Editors’ note: Systemic vistas. Bridge, 50(4), 3–5.
  • Mandelli, D., Wang, C., Parisi, C., Maljovec, D., Alfonsi, A., Ma, Z., & Smith, C. (2020). Linking classical PRA models to a dynamic PRA. Annals of Nuclear Energy, 149, Article 107746. https://doi.org/10.1016/j.anucene.2020.107746
  • Mao, Q., & Li, N. (2018). Assessment of the impact of interdependencies on the resilience of networked critical infrastructure systems. Natural Hazards, 93(1), 315–337. https://doi.org/10.1007/s11069-018-3302-3
  • Nakagawa, E. Y., Gonçalves, M., Guessi, M., Oliveira, L. B., & Oquendo, F. (2013, July). The state of the art and future perspectives in systems of systems software architectures. In Proceedings of the First International Workshop on Software Engineering for Systems-of-Systems (pp. 13–20).
  • Nielsen, C. B., Larsen, P. G., Fitzgerald, J., Woodcock, J., & Peleska, J. (2015). Systems of systems engineering. ACM Computing Surveys, 48(2), 1–41. https://doi.org/10.1145/2794381
  • Pereira, J. C., & de FSM Russo, R. (2018). Design thinking integrated in agile software development: A systematic literature review. Procedia Computer Science, 138, 775–782. https://doi.org/10.1016/j.procs.2018.10.101
  • Pistikopoulos, E. N., Barbosa-Povoa, A., Lee, J. H., Misener, R., Mitsos, A., Reklaitis, G. V., Venkatasubramanian, V., You, F., Gani, R. (2021). Process systems engineering – The generation next? Computers & Chemical Engineering, 147, Article 107252. https://doi.org/10.1016/j.compchemeng.2021.107252
  • Rashid, M., Anwar, M. W., & Khan, A. M. (2015). Toward the tools selection in model based system engineering for embedded systems—A systematic literature review. Journal of Systems and Software, 106, 150–163. https://doi.org/10.1016/j.jss.2015.04.089
  • Sanchez, L. M., & Nagi, R. (2001). A review of agile manufacturing systems. International Journal of Production Research, 39(16), 3561–3600. https://doi.org/10.1080/00207540110068790
  • Takahashi, K., & Nakamura, N. (2000). Agile control in JIT ordering systems. International Journal of Agile Management Systems, 2(3), 242–252. https://doi.org/10.1108/14654650010356158
  • Trucco, P., Cagno, E., & De Ambroggi M. (2012). Dynamic functional modelling of vulnerability and interoperability of critical infrastructures. Reliability Engineering & System Safety, 105, 51–63. https://doi.org/10.1016/j.ress.2011.12.003
  • Turner, R. (2007). Toward Agile systems engineering processes. Crosstalk. The Journal of Defense Software Engineering, 11–15.
  • Wu, C., & Barnes, D. (2011). A literature review of decision-making models and approaches for partner selection in agile supply chains. Journal of Purchasing and Supply Management, 17(4), 256–274. https://doi.org/10.1016/j.pursup.2011.09.002
  • Yin, X., Hu, X., Rong, M., Liu, H., & Yang, Y. (2019). Review of evaluation methods of contribution rate to system of systems. Journal of System Simulation, 31(6), 1027.