A Survey of Big Data Issues in Electronic Health Record Analysis

ABSTRACT

The Electronic Health Record (EHR) groups all digital documents related to a given patient such as anamnesis, results of the laboratory tests, prescriptions, recorded medical signals as ECG or images, etc. Dealing with such data representation incurs a plethora of problems, such as different data types, even unstructured data (i.e., doctor’s notes), huge and fast-growing volume, etc. Therefore. EHR should be considered as one of the most complex data objects in the information processing industry. Accordingly, taking into consideration its complexity, heterogeneity, fast growth, and size, the analysis of EHR data increasingly needs big data tools. Such tools should be able to analyze datasets characterized by the so-called 4Vs (volume, velocity, variety, and veracity). These notwithstanding, we should also add the fifth V—value—because analytics tool deployment makes sense only if it leads to health-care improvement (as personalized patient care, decreasing unnecessary hospitalization, or reducing patient readmissions). In this study, we focus on the selected aspects of EHR analysis from the big data perspective.

Introduction

The Electronic Health Record (EHR) collects a digital representation of individual patient documents as results of the medical tests, doctor’s descriptions, prescriptions, images, etc. Its implementation is expected to improve the procedures and quality of health care, help to control the costs, and reduce fraud (Graña and Jackwoski 2015). Its implementation induced changes in the way physicians deal with information and knowledge, forcing documentation procedures that are resented by some care providers as cumbersome and a waste of efforts (Reich 2012), producing resistances to its implantation. The paradigmatic example is its recent implementation in Greece in the middle of a significant economical crisis (Vijayakrishnan et al. 2014) It has been also proposed that EHR is a key instrument to support public health (Kukafka et al. 2007), which requires extending its contents to nonmedical environmental information, providing standard protocols and processing pipelines to make the EHR information broadly available. The health information exchange system connects primary care, hospitals, pharmacies, and laboratories to the health-care information management network, in order that the EHR might play a central role in communications between health institutions and companies, as well as inside smart hospitals (Mertz 2014). Nevertheless, working EHR implementations produce massive interactions and generation of data. Therefore, EHR data management should be seen as large-scale data processing of complex data representations. In 2012, worldwide digital health-care data was estimated to be equal to 500 petabytes and is expected to reach 25,000 petabytes in 2020.¹ These huge quantities of data require specific computational methods, such as distributed processing architecture (El-Sappagh and El-Masri 2014). Accordingly, taking into consideration its complexity, heterogeneity, fast growth, and size, health-care data can be categorized as a big data problem, requiring special tools to analyze it (Mathew and Pillai 2015)in order to deal efficiently with the 5Vs:

• Volume has been identified as the main feature of big data problems, because data scale requires innovative tools to store and access it. Let’s imagine that a doctor’s note could be stored as text file, which is a few kilobytes, but a raw image requires a few megabytes, and more-sophisticated diagnostic tools, such as MRI, might easily be a few gigabytes. If we multiple such volume size by the number of tests carried out in the hospital, we should be ready to deal with tera- and petabytes, but we should be also ready for exabytes volumes, because each reasonable analysis should have access to entire patient histories. Taking into consideration medical data, we should also address the following issues:

  • Data distribution versus privacy. Dealing with distributed data could potentially cause numerous problems such as diversity of data source qualities, and we also should notice that using distributed databases might easily clash against legal or commercial limitations that do not allow sharing raw data among databases or merging them in the common repository. Therefore, developing privacy-preserving versions of the lived-in data analysis techniques has been the subject of intense research efforts, and it primarily aims to offer a possibility of distributed data analysis with a minimum risk of information disclosure.

  • Response time of analytic systems. The problem is not the volume to store (prices are going down), but the response time. Users do not agree to wait too long for the answer of a typical query.

  • Special structure for data, i.e., migrating from operating database systems to analytic databases (data warehouses)

  • Standards for data collections (such as the Digital Imaging and Communications in Medicine (DICOM)² used by picture archiving and communication system (PACS)) and transfer (such as HL7³).

• Velocity. Data is in motion, new information about patients is added, some medical records are updated. Therefore, smart analytic tools applied to EHR analysis require that the models need not be rebuilt from scratch when new data come, but only improved. Thus, the analysis of the data stream rises as one of the crucial topics for data scientists.

• Variety. The EHR groups’ inhomogeunous data: on the one hand, structured data in the form of standardized medical info, such as DICOM or International Classification of Diseases (ICD) codes, but on the other hand, highly valuable information that is buried in the doctor’s notes, which are written in natural language, waiting to be mined and assigned value. Therefore, methods of data structuralization, natural language processing, automatic image/video analysis are the focus of intense research. The following problem should be also considered:

  • Data privacy versus data integration. The EHR inhomogeneity extends to the personal nature of the data, which is protected by law in most civilized countries. Therefore, security and privacy are mandatory in EHR processing and transmission through communication channels.

  • Cost of data integration and structuralization versus profits.

• Veracity. Data quality seems to be the crucial issue, because only high-quality data can produce credible models. Unfortunately, most of the data is biased and noisy. Often, we may find anomalies, such as outliers in data or missing items. Human errors are an important issue as well. Kohn et al. (Kohn, Corrigan, and Donaldson 2000) report that, each year, 44.000–98.000 patients die in the United States due to avoidable clinical mistakes. Therefore, development of methods that are able to detect wrong treatments and/or diagnoses, and clean the EHR of errors are strongly required.

• Value. Analytic tool deployment makes sense only if it leads to improvement in health care (such as personalized patient care, decreasing unnecessary hospitalizations, or reducing patient readmissions). Apart from the economic aspects concerned with ways of reducing health-care costs while assuring the same or higher quality level of service, we should also remember that such methods should focus on the issue of how to improve patient well-being. For instance, the detection of disease outbreaks (Dawson, Gailis, and Meehan 2015) or general/specific population conditions, such as prevalence of infections (Shang et al. 2015) in specific populations, has a high public-health value and can readily be computed from EHR data.

In this work, we focus on selected aspects of EHR analysis from the big data perspective (Cyganek et al. 2015), such as efficient computer system structures, data quality and data privacy, data mining, cost-sensitive approaches, and, especially, the medical data stream analysis. The analysis of huge volumes of fast-arriving data is recently the focus of intense research, because such methods could provide a competitive advantage to a given company. A most promising approach is data stream classification, which is employed to solve nonstationary problems related to discovery of client preference changes, spam filtering, fraud detection, and medical diagnosis, to enumerate only a few.

Efficient computing system structure

With regard to the volume dimension of big data, an important performance metric is the response time of the computer system utilized to process the data. Some big data applications are local oriented, i.e., the results of data processing are needed only locally. In the local-oriented scenario, the computer system required to process the data includes local servers that are responsible for running the algorithms that analyze the data, storage devices that store the data, and a local area network (LAN) connecting all elements of the system. In turn, some big data applications require computational resources of scale greater than a local system to store, analyse, and deliver results. For ease of reference, we call these scenarios global-oriented. In particular, in the mainstream of global-oriented big data applications, the system produces results that are requested by a number of users distributed in a geographical area exceeding the local system. Another scenario is when big data analysis needs huge computational power that cannot be provided locally, and consequently, specialized data centers are utilized to process data. Moreover, in order to reduce both capital expenditures (CAPEX) and operational expenditures (OPEX), global-oriented big data analysis can be achieved by using the concept of cloud computing. More specifically, usually data centers that provide various processing and computing services are of very large scale, which facilitates the deployment process in terms of the investment cost reduction. For instance, purchasing thousands of pieces of equipment for the data center (computers, storage, switches, cables, etc.) significantly reduces the required CAPEX cost. Second, the data centers are commonly deployed in places that allow low costs in terms of operational expenditures, which follow mostly from energy consumption. For example, data centers that are located in places with relatively inexpensive energy (e.g., close to big power plants or sources renewable energy) and low values of power usage effectiveness (PUE) requirements resulting from the reduced needs for cooling (e.g., cold climate).

Note that the common element of all global-oriented big data applications is the requirement of data transmissions across large distances via wide area networks (WAN). Nowadays, the state-of-art WAN solution is given by optical networks including technologies such as wavelength division multiplexing (WDM) and elastic optical networks (EONs). Usually, it is a two-layer network architecture with an optical network in the lower layer and packet-switched network in the upper layer. In the case of big data applications, two scenarios are possible to provision big data network traffic, i.e., service network provisioning by the packet-switched network layer, which ensures delivery of the big data traffic, and optical network provisioning, with which big data traffic is transmitted directly over the optical layer with the use of short-lived lightpaths. In more detail, in the service network provisioning scenario, the traffic related to big data analysis is carried in the packet-switched network layer using the existing virtual topology of lightpaths, i.e., the underlying optical network is assumed to be overprovisioned, so that extra capacity is left to enable big data traffic according to new requests. However, limited resources of network capacity can significantly increase the system response time to unacceptable values for some big data applications that require very fast reaction. The optical network provisioning scenario allows establishing dynamic short-lived lightpaths directly in the optical layer, fully dedicated to carrying the big data traffic. More specifically, each new data transfer request related to the big data analysis is provisioned by a new lightpath that connects directly the client node and the computing resources. This strategy supports transmissions with very high bandwidths (for instance, up to 1 Tb/s in EONs) ensuring very short response time of the system. As well as the response time, other important performance metrics to be addressed in the context of big data processing are: scalability, resilience, security, low cost (including both CAPEX capital expenditures and OPEX operating cost), energy efficiency (Erl, Puttini, and Mahmood 2013; Klinkowski and Walkowiak 2013; Venters and Whitley 2012; Walkowiak et al. 2015).

Today, big data analytic solutions are becoming widely accessible (Asri et al. 2015; Cyganek et al. 2015; Mathew and Pillai 2015), including the well-known Hadoop and MapReduce for massive job execution in scalable clusters of servers and many other free, open source tools for distributing data processing, data visualization, and data mining of large-scale datasets, allowing applications such as predictive epidemics, personalized health care, minimization of medication errors (Nguyen et al. 2013), and evidence-based medicine. As well, the big investment on computational power can be avoided by using commercial cloud services, which allow a smooth scale-up of the system, even to the point of mobile app development (Martíez-Pérez et al. 2015). However, cloud-based implementations using third-party services raise strong security and privacy issues (Katt 2014; Mohammed et al. 2015), and their use, it has been found, is still immature (Hu and Bai 2014). Development of secret sharing schema and algorithms will improve the opportunities to benefit from cloud services in EHR systems (Ermakova and Fabian 2013).

Data quality

We have also to keep in mind that the quality of the results of any analytical tool will depend on the quality of data used for the model building process and/or the a priori knowledge that we have at our disposal. It is obvious that an unrepresentative learning dataset, or incomplete (or inconsistent) rules will potentially lead to the construction of a low quality classifier. To boost the quality of the learning system input information, it is positive to use knowledge and data from different sources, e.g., data from numerous databases. In addition, we also have to answer the following questions:

1. What is the quality of each source? To what extent does merging data or knowledge from unreliable sources (as from a low-quality expert or unrepresentative dataset or database that includes numerous mistakes) decrease the quality of classification?

2. How to merge learning materials of different quality? We usually have a learning set and rules at our disposal, then we have to answer the question: “Do we use each source separately, e.g., we train individual classifiers on the basis of each type of learning material, and then we will find a method to combine their decisions?” Also, we can transform various forms of learning materials into one unified form (meaning that datasets are transformed into rules using, e.g., well-known machine learning methods), or generate a set of learning examples on the basis of rules. Regardless of the choice, there still exist problems involving the quality of learning material.

3. Is the learning data material consistent? Usually we meet the problem of knowledge consistency. However, if we would like to merge learning materials form different sources, then, this merged one could be inconsistent.

The quality of knowledge plays a key role for machine learning systems. In designing a decision support system, we are able to gain knowledge from different sources (i.e., from different experts or on the basis of different databases), and we also have to take different qualities into consideration.

This problem was partly described for induction learning (Bruha and Kocková 1993; Dean and Famili, 1997; Gur-Ali and Wallace 1993) and concept description (An and Cercone 1999; Bergadano et al. 1988), and we can find several works dealing with decision making on the basis of the learning set quality; for instance (Schapire 2001) aims to construct classifiers based on a very small learning set. Those problems might be regarded as the decision making based on the unreliable (nonrepresentative) sources.

Let’s consider the problem of how to assess the quality of a rule for probabilistic reasoning; in the next section, we propose a kind of measure that can be applied to knowledge acquisition process for diverse forms of rule-based models. The use of semantic resources, such as ad hoc ontologies, and data correctness verification based on semantic processing has been proposed to achieve data quality in the big data scenario of EHR (Gai et al. 2015). The alignment of the semantic expectations with the actual data extracted from the EHR annotations allows detection of errors at a much more abstract level, before their exploitation by decision support systems’ (Kashfi 2011) data mining. Also, paying special attention to usability issues has been pointed out as a way to achieve much better quality data (Villa and Cabezas 2014). If the health-care provider is not overwhelmed by the data input process, the probability of errors will be much smaller (Vedanthan et al. 2015).

Cost and benefit

Today, for most of the practical decision problems coming from different areas of human activities, we can choose an appropriate analytical tool as a classification method to make a high-quality decision. However, its choice is usually limited by several constraints related to data gathering and classification of computation costs. We can convert this problem into one of training and exploitation of classifiers. First, and probably the best-known criterion, is the misclassification cost, which is widely discussed in the literature (Duda, Hart, and Stork 2001) and is the key criterion in Bayesian decision theory. It proposes to define a so-called loss function, which informs about the misclassification costs between each pair of classes. According to this theory, the optimal classifier makes decisions minimizing the expectation value of a loss function. We can also find several propositions on how to deal with it. For instance, Peng et al. (2005) discussed how to create a cost-sensitive classifier ensemble for a medical decision support system. Of course, the misclassification cost is important, but we can find another source that might generate additional costs during training or exploitation of classifiers:

• Cost of training is usually high for classifiers that build structures used during classification, e.g., decision trees (Alpaydin 2010).

• Cost of testing.

• Cost of decision making usually takes the cost of necessary feature acquisition into consideration.

Let us focus here on the last type of cost sources; however, the discussion will be continued in the next sections as well, where we will concentrate on the cost-sensitive approach applied to combined classifiers. Meanwhile, the cost may be measured as a price of medical tests, or time, in the case of measurement repetitions that require significant time. Such a meaning of cost-sensitive classification arises frequently in many fields of human activities such as the industrial production process (Verdenius 1991), robotics (Tan and Schlimmer 1989), and technological diagnosis (Lirov and Yue 1991), to enumerate only a few. However, this problem is most clearly visible for medical diagnosis (Núñez 1988). As mentioned, for most decision problems we have the necessary tools to make a high-quality decision, but, from a practical point of view, we have to notice that a physician has a limited budget within which to make tests for diagnosis. Therefore, in real cases, physicians have to balance the costs of various tests with the expected benefits. Physicians usually have to make the diagnosis fast on the basis of (low cost) features acquired from measurements that do not require much time to conduct—because the therapeutic action has to be taken without any delay, e.g., in the case of a rescue operation.

In addition to benefits that might be obtained by the patient, the practitioner, and the general public from the big data processing of EHR information (Michel-Verkerke, Stegwee, and Spil 2015), there is a strong interest in the health-care industry where the EHR-related products are becoming a big economic sector. The EHR implementations alone were estimated at $15.5 billion in 2010 and are projected to grow to $19.7 billion (Nguyen, Bellucci, and Nguyen 2014). The current global mobile health-care market was valued at $6.6 billion in 2013 (Ben-Assuli 2015). The pharmaceutical industries, as well as other health-related industries, might be directly interested in EHR data mining (Kim, Labkoff, and Holliday 2008; Moor et al. 2015), looking for innovations, assessment of the effect of drugs, identification of target health-care markets, and real data for the development of medicines (Berger et al. 2015). Also, government agencies are interested in expense prediction in order to allocate resources (Vivas-Consuelo et al. 2014). All of them need consistent, updated, anonymous, and error-free data.

Data mining

The availability of extensive EHR databases, allows data mining analysis performed by a wide variety of applied views. On-the-fly construction of probabilistic models from the stored EHR data allowed obtaining detectors of epidemic outbreaks (Dawson, Gailis, and Meehan 2015), using Bayesian modeling and particle filtering. The evaluation of the prevalence of infection features in home health-care data (Shang et al. 2015) allowed identifying an increasingly serious problem due to population aging. Similarly feasible is the use of longitudinal studies using past EHR information (Oakkar et al. 2015), such as the one covering the acculturation of Asian American men and its correlation with the body mass index. Also, retrospective analysis of EHR data allows improved definition of new clinical trials, or observation studies (Kreuzthaler, Schulz, and Berghold 2015), for an optimal design of the trial. Natural language processing allows extracting meaningful information from semistructured data, wherein some information is written down by the clinician in the form of an informal report. This has been exploited more in some specific areas, such as digestive diseases (Hou, Imler, and Imperiale 2014), or in the discovering of surgical site infection risk (Michelson, Pariseau, and Paganelli 2014). A mixture of natural language processing of free text writing in EHR and statistical analysis of some of the data fields allowed for the detection of past history of indigents leading to heart failure in a population of 50,000 primary care patients (Vijayakrishnan et al. 2014). Data mining can be extended to the relationships among physicians, nurses, and other roles in the health-care system, discovering the underlying social network structure (Malin, Nyemba, and Paulett 2011), which can be useful to set the proper access rights and channel the information in a proactive way. HER-derived phenotyping allows for mapping of the EHR data into specific and meaningful medical concpets. Ho, Ghosh, and Sun (2014), as well as Wang et al. (2015), propose to use the nonnegative tensor factorization for extraction of phenotypes from the patient claim records. This way, obtained tensor factors can be used as phenotype candidates that automatically indicate patient clusters on specific diagnoses or treatments. Finally, the use of EHR allows building patient summaries, such as noted frequent medication, which allow treatment in emergency situations when the patient has little cognitive ability and cannot answer questions (Remen and Grimsmo 2011).

Data mining from EHR has been done in some specific areas, often related to highly prevalent diseases that ask for some public governance understanding of the facts in order to define appropriate health policies. This is the case of childhood obesity, which is a growing concern in both developed and underdeveloped countries. The analysis of EHR data allows determining demographic factors with heavy influence in the rise of the abnormal condition (Baer et al. 2013; Brown et al. 2015; Flood et al. 2015). Another general public concern is cardiac conditions, which are a major cause of death. There have been a number of data mining studies based on EHR data aiming to identify risk factors (Green et al. 2012; Rubbo et al. 2015). The final example is diabetes, whose prevalence is steadily increasing due to eating and exercise habits and increasing older populations. EHR data mining looks for risk factors as well as comorbidities and cost analysis of effective treatment (Chung et al. 2015; Mashayekhi et al. 2015; Zhong et al. 2015).

Data stream analysis

The analysis of huge volumes of fast-arriving data is recently the focus of intense research, because such methods allow building a competitive advantage for a given company. There are some works devoted to data visualization oriented toward interactive manipulation of the data (Gotz, Wang, and Perer 2014), with some emphasis on the search for patterns of clinical events. One useful approach is the data stream classification, which is employed to solve problems related to discovery of customer preference changes, spam filtering, fraud detection, and medical diagnosis to enumerate only a few. Basically, there are two classifier design approaches:

• build and use, which first focuses on training a model quickly and then using the trained classifier to make decisions;

• after-model-construction tries to tune the model parameters continuously while making decisions.

The first approach is very traditional and is useful only under the assumptions that data used for training and being recognized come from the same distribution, and the number of training examples ensures that a model is well trained (i.e., it is not undertrained). Of course, for many practical tasks, such assumptions could be accepted; nevertheless, many contemporary problems do not allow its acceptance and should take into consideration that the statistical dependencies describing the classification task as prior probabilities and conditional probability density functions could change. As a result of the changes, the posterior probabilities (responsible for the decision boundary shapes) could change as well. Additionally, we should take into account the fact that data come very fast; therefore, it is impossible to label arriving examples manually by a human expert, but each object should be labeled by a classifier.

The first problem is called concept drift (Widmer and Kubat 1996) and the efficient methods that are able to deal with it are still the focus of intense research, because appearance of this phenomenon could potentially cause a significant accuracy deterioration of a classifier under exploitation (Wozniak, Kasprzak, and Cal 2013).

Therefore, developing methods, which are able to effectively deal with this phenomenon has become a vital issue of pattern classification. Basically, the following approaches may be considered to deal with the problem:

1. detecting the drift and retraining the classifier,

2. rebuilding the model frequently, or

3. adapting the classification model to data population (distribution) changes.

The last approach, “build, use, and improve,” is usually dedicated to data stream analysis, in which the apparent changes are rather slow and gradual. The methods related to this topic usually use continuous model updating after each incoming example (online learners) or collect a batch of data to update the model (sliding windows).

In many pattern classification tasks there exist dependencies among patterns to be classified. It is typical, e.g., for chronic disease diagnosis (for which historical data plays the crucial role for high-quality medical assessment), to name only a few.

The formalization of a classification task requires successively classified objects that are related one to another, but also could take into consideration the occurrence of external factors changing the character of these relationships. Let’s illustrate this task, using an example of medical diagnosis. The aim of this task is to classify the successive states of a patient. Each diagnosis could be the basis for a certain therapeutic procedure. Thus, we obtain a closed-loop system in which the object under observation is simultaneously subjected to control (treatment) dependent on the classification. The mentioned control could be recognized as the cause of the concept drift. As distinct from the traditional concept drift model, where drift appears randomly, in this case the drift has a deterministic nature.

Direct learning from EHR data is a challenge, because both the heterogeneous nature and the mixture of structured and nonstructured information of the EHR data ares quite inappropriate for conventional machine learning. The current trend to apply deep learning approaches to many signal processing problems (face recognition, object recognition, strategy in games) finds difficulties when applied to EHR data due to coding it as a real-valued vector. As an example, a predictive study about suicide tried to apply restricted Boltzmann Machines (Tran et al. 2015) to embed the EHR data into a vector space that might be subject to further analytical process; it required ad hoc modifications in the form of rules for the preservation of data structure and the constraint of preserving nonnegative weights, in order to produce meaningful results. Another pressing issue that hinders machine learning application is that of privacy. The data must be freed from variable values that might allow user identification or its imputation to some group of people with negative implications (e.g., HIV infected), although the data might still be useful for applications such as recommendation of treatment based on similarity with other patients. The approach of Chignell et al. (2013) performs clustering of some features, retaining the residuals after removing the cluster centroids, thus obtaining a shifted representation that is quite local and difficult to invert. Residual data can effectively be used for recommendations based on patient similarity.

Multidimensional tools for EHR data representation and analysis

As previously alluded, EHR can contain different types of data, represented in various formats. However, what is the most important for EHR data are the ways of extracting vital information to facilitate patient health-care processes. Frequently, EHR data depend on a multitude of factors that render their analyses even more difficult. However, there are recently developed methods of representation and analysis of multidimensional data with the help of tensor methods (Kolda and Bader 2009; Lathauwer et al. 2000b). These can be used for representation and analysis of some types of EHR data, as will be discussed in this section.

A tensor is a multilinear real function operating on a vector space and its dual. However, for data mining, a tensor can be perceived as a multidimensional cube of numeric values that extend such well-known objects as vectors and matrices. For a short introduction to tensors for data analysis and software implementations, see, e.g., Cyganek (2013). Examples of multidimensional medical data are ample, such as magnetic resonance imaging (MRI) data, X-ray images, or EHR-derived phenotyping, which goal is to map EHR data to some medical concepts (Ho, Ghosh, and Sun 2014; Wang et al. 2015), to name a few. Interestingly enough, diffusion MRI, which aims at measuring the diffusion of water molecules in biological tissues, as well as diffusion tensor imaging (DTI), heavily rely on tensor analysis (Lauterbur 1973; Le Bihan et al. 1986).

In the case of EHR data, tensors can be used mostly for data compression and data analysis. For both applications, a proper tensor decomposition method should be employed. In the following, we provide a short overview of some principal tensor decomposition methods with information on potential application to EHR data processing (Cyganek and Wozniak 2015).

1. Higher Order Singular Value Decomposition (HOSVD) is a multidimensional extension of the well-known SVD matrix decomposition to decomposition of tensors. It allows tensor representation as a product of a core tensor and mode matrices (Kolda and Bader 2009; Lathauwer et al. 2000a). HOSVD is related to the Tucker decomposition, which was introduced into psychometrics by the pioneering work of Ledyard Tucker in the 1960s (Tucker 1966). HOSVD has a number of interesting properties. For instance, it builds orthogonal tensor subspaces, which can be used for pattern recognition (Cyganek 2013). It also can be used for data compression, although there are better methods (Wang and Ahuja 2007).

2. Best rank-1 tensor decomposition, also known as CANDECOMP/PARAFAC (CP). This decomposition aims at approximating a tensor as a sum of vector products which are all of rank 1 (Lathauwer, Moor, and Vandewalle 2000b). For instance, in a phenotyping problem, an initial tensor is composed of the counts of the co-occurences between diagnosis and a chosen medical procedure, and for a given patient. This way, a 3D tensor is composed, for which its element at position (a,b,c) is a simple counter of the number of times a procedure “a” was used on a patient “b” having symptoms “c.” Finally, the nonnegative rank-1 decomposition allows extraction of the procedure, patient, and symptom factors, respectively.

3. The best all-rank decomposition, also called the best rank-(R1, R2, RP) decomposition (Lathauwer, Moor, and Vandewalle 2000b). For a P-dimensional tensor, this decomposition aims at constructing an approximating tensor that fulfills the P-rank constraint. The best-rank tensor decomposition can be used for data compression and classification (Cyganek and Wozniak 2015; Wang and Ahuja 2007). However, the algorithm is iterative and frequently initiated with the HOSVD method.

4. Nonnegative tensor decomposition with optional sparsity and orthogonality constraints (Hazan, Polak, and Shashua 2005). These types of tensor decompositions assume tensor decomposition into only nonnegative factors. They found applications in biology and medicine due to clear interpretation of the decomposing factors. They write well into the idea of the purely additive influence of each of the components discovered in the data (Cichocki, Zdunek, and Amari 2008).

5. Tensor optimal subspace projections aim at decompositions into tensor subspaces that exhibit properties suitable for optimal classification. The optimal conditions are obtained by embedding proper constraints, such as sparsity or locality preserving, during the optimization process (Liu, Liu, and Chan 2010).

Apart, there are many other methods and variants of tensor decompositions, which also can be considered for efficient processing of various EHR data (Chi and Kolda 2012; Kolda and Bader 2009).

Access and medical data protection

An important aspect of the health record system is to ensure the confidentiality of the patient personal data (Fernandez-Aleman et al. 2013). These data have today a more virtual-based existence than a physical one, therefore, protecting them is difficult. The most basic security technological requirements are related to data encryption and certification as well as identity authentication to reach data access. From a practical point of view, protection is provided by multilevel electronic technologies and computer science techniques. Integration of these services is governed in many countries (e.g., Australia, Japan, United States) by rule-based acts, which describe, among other issues, patients’ rights, obligatory information-protection steps by the medical staff, and policies of data storage. Because patients’ medical data are read, entered, and collected by many people (e.g., nurses, medical technicians, and doctors) the multilevel protection is required:

• (1) The first, simplest level of protection:

  • (a) installation of specialized management software with username and PIN or password according to personalized access rules,

  • (b) execution of security measures (password or PIN changing after periodic intervals, number of characters in password, etc.).

• (2) The second level of protection is based on biometric verification systems, where face, signature, retina, finger or palm recognition increase the level of security of the system (Doroz, Porwik, and Orczyk 2016).

Unfortunately, the aforementioned access restrictions are still insufficient. A serious data security issue paradigm is a one-time authorization made usually when starting work with a device or an IT system. This type of user authentication can result in a serious threat, especially in open-type areas such as medical facilities, causing systems and EHRs being vulnerable to an intruder attack. The nature of the work of medical staff is such that workstations with appropriate equipment are, from time to time, unsupervised, although earlier access to them had been granted correctly. It is a moment when access to a workstation can be taken over by illegitimate persons. To ensure a high level of security, especially of medical data, information systems should be continuously monitored. For this reason, the next security level is strongly recommended. Flexible and fine-grained data access rights of individual users can be achieved based on semantic analysis of the EHR semistructure data (Amato et al. 2015). Some works aim to assess the appropriate access control policies, in other words, access rights must be tailored to the actual job profile, which can be ascertained from the analysis of the social network structure derived from actual logs of EHR system access (Malin, Nyemba, and Paulett 2011).

• (3) The third level of protection involves continuously monitoring users’ activities associated with keyboard-based interfaces. Using these interfaces, intruders can break into a system and gain unauthorized access to protected data. It can be averted by computer-user profiling. Such a profiling is performed by intrusion detection systems that constantly monitor all operations performed by users and then try to verify a user’s identity by comparing his/her activity to his/her profile. In this approach, a computer users’ activity is constantly analyzed by performing a real-time keyboard monitoring. The process of collecting a user’s activity data is carried out in the background, practically not involving any user attention. The encryption of the alphanumeric characters entered via a keyboard is performed in order to prevent an access to the user’s private and sensitive information (passwords, PIN codes). It means that unauthorized access is quickly detected and alarmed (Kudlacik, Porwik, and Wesolowski 2016).

Smartphones bring new challenges to the security and privacy of health-care data. It has been reported that 80% of health-care personnel use their smartphones for work-related tasks (Burns and Johnson 2015), so that the health information exchange system has become a bring-your-own-device (BYOD) environment, where a whole ecology of applications might be coexisting in the devices used for EHR data visualization and manipulation, opening the door to many unintentional but malicious threats. System managers must implement new policies to minimize these threats, such as installing approved security applications in personal devices, which implies intruding in the personal devices. Moreover, widespread mobile health-care applications lack security certifications, although standards are hurriedly being setup by the American National Institute of Standards and Technology (NIST) and companion institutions. Some works are directed toward building in the mobile ecosystems a secure layer based on the trustworthiness of entities, applying trust-assessment systems imported from e-commerce (Bahtiyar and Çağlayan 2014). Trust can be measured by direct communication and through indirect reputation measures of third parties. Trust assessment allows secure identity management systems without a central authority to overcome the difficulties found in establishing such central authority in health information ecosystems (Dolera Tormo et al. 2013). Solutions for security enhancement in mobile health-care ecosystems (Simplicio et al. 2015) must contemplate loss of connectivity or transmission delays, as well as theft and device sharing. Hence, offline authentication must be added, as well as secure storage and transmission by encryption with authenticated keys. Mobile information ecosystems can take the form of mobile health social networks, i.e., among patients suffering the same disease (Zhou et al. 2013), offering benefits such as opportunistic computing to carry out some big data operations, and communication links sharing, as well as social comfort. These social networks require new cryptographic security strategies as well as trust-assessment methods to avoid attacks at a diversity of levels, coming from the body area network to the communication grid. To this already complicated scenario, the advent of pervasive computing and the Internet of Things (IoT) adds new factors in a health-care system’s ecology (Trcek and Brodnik 2013), which require new, distributed authentication protocols, which must be robust even to power failures in the IoT devices.

A key technological element involved in the quest for achieving data privacy while granting access for research and public interest information processing is the De-identification (a.k.a. anonymization) of patient records. The National Institutes for Health (NIH) classifies research as Human Subjects Research (HSR) or Not Human Subjects Research (NHSR). The NHSR has much less administrative oversight and can be more effectively exploited. Privacy threads come from the ability to deduce direct identifiers, quasi-identifiers, and sensitive attributes from the data (Gkoulalas-Divanis, Loukides, and Sun 2014), which can be subjects of identity, membership, or attribute disclosures. Privacy models, such as k-Anonimity, allow designing algorithms and to assess the privacy risks of already anonymized data. Some processes achieve anonymization by statistical analysis, i.e., removing high-entropy variables (Chignell et al. 2013). AS well as structured information in fields, EHR might also contain free-form text, which, to some extent, might be cleared of identity information by some basic natural language processing (NPL) analysis, such as syntactic analysis (Huang et al. 2009). However, further guaranties need more sophisticated language-dependent NPL tools in addition to pattern recognition (Zuccon et al. 2014). For example, Chazard et al. (2014) builds the list of forbidden words in French following an incremental process without a prior dictionary. Similarly, Hanauer et al. (2013) uses an open source solution for statistical identity scrubbing with little human effort. Disassociation (Loukides et al. 2014) is another approach to impede the identification of the patient. It is achieved by partitioning the EHR in several pieces, allowing separate processing. When considering image data, the anonymization could require image processing to assess that no identity information is slipped into printed images (Newhauser et al. 2014). However, anonymous sharing and cooperative processing of clinical and signal data via a web service on a multicenter study on deep brain stimulation for Parkinson’s Disease has been demonstrated (Rossi et al. 2014). Most anonymization approaches scale badly, failing for large datasets. Recent works are achieving a big data scale of anonymization techniques (Zhang et al. 2015) by the use of standard big data tools.

Finally, we should discuss a scenario in which valuable EHR data is distributed among a number of servers or computing systems. In modern health care, such databases might belong to private medical companies that are not interested in sharing their resources. However, using information stored in multiple EHR warehouses might be much more efficient for data mining purposes, offering a wider perspective on analyzed trends and phenomena on a global scale. Therefore, privacy issues in EHR are directly related to distributed data mining from heterogeneous sources. The aim of such procedures is to extract knowledge and value from distributed data without releasing any information with regard to the nature of data stored in any of the sources. This is known as privacy-protecting data mining (Xu et al. 2014) and is of crucial importance in the big data scenarios of EHRs.

As EHR data mining resources should be oriented toward medical experts, machine learning methods that offer interpretable decisions and can explain their predictions are of primary focus. Here, one must mention privacy-preserving decision trees (Vaidya et al. 2014) and rule-based methods (Alwatban and Emam 2014). They offer a good trade-off between accuracy and ease of interpretation while offering advanced privacy protection techniques. However, these models have not been discussed in the context of big data. Therefore, there is a need for combining these techniques with efficient computing environments that would allow for fast induction of these classifiers when faced with massive, evolving, and nonstationary EHR data (del Río et al. 2015).

Here also, one should mention other privacy-preserving machine learning techniques, such as nearest neighbor methods (Krawczyk and Woźiak, 2013), Bayesian learning (Liu et al. 2016), or ensemble solutions (Li, Bai, and Reddy 2016). They all, however, still lack true efficient implementation for big data that will allow for real-time analytics, online learning, and constant model updating. New proposals on how to run privacy-preserving EHR mining tools in environments such as MapReduce or Spark (Fernádez et al. 2014) are crucial for taking a step toward efficient and secure EHR processing.

Final remarks

Though EHR implementation efforts started a couple of decades ago, their use is still problematic and questioned in some areas of health care, whereas other areas are very keen on it and use EHR data positively. One of the areas of work that is more promising is the creation of intelligent systems based on the EHR stored information, i.e., the realization of retrospective studies, once legal and ethical problems are solved by proper anonymization algorithms. In this area the following are the main challenges:

• understanding doctors’ notes (unstructured text analysis);

• handling huge volumes of medical images that are part of the EHR and continuing to increase storage requirements steadily; medical imaging is big data problem by itself, requiring sophisticated mathematical modeling and numerical algorithms at many levels;

• backtracking the effect of medical decisions, i.e., treatments, so that the physicians can improve, over time, their clinical guidelines and procedures. Up to now, data analysis has been limited to the statistical inference over the data and/or the construction of some prognosis-predictive systems, which do not take into account medical decisions and how they interact with patient evolution.

Funding

This work was supported by the Polish National Science Center under the grant no. DEC-2013/09/B/ST6/02264 and by EC under FP7, Coordination and Support Action, Grant Agreement Number 316097, ENGINE - European Research Centre of Network Intelligence for Innovation Enhancement (http://engine.pwr.wroc.pl/).

Additional information

Funding

This work was supported by the Polish National Science Center under the grant no. DEC-2013/09/B/ST6/02264 and by EC under FP7, Coordination and Support Action, Grant Agreement Number 316097, ENGINE - European Research Centre of Network Intelligence for Innovation Enhancement (http://engine.pwr.wroc.pl/).

Notes

1. Jimeng Sun and Chandan K. Reddy, Big Data Analytics for Healthcare, Tutorial presentation at the SIAM International Conference on Data Mining, Austin, TX, 2013. http://dmkd.cs.wayne.edu/TUTORIAL/Healthcare/

2. http://medical.nema.org/standard.html

3. http://www.hl7.org/