Multimodal data fusion for systems improvement: A review

References

• Abavisani, M. and Patel, V.M. (2018) Multimodal sparse and low-rank subspace clustering. Information Fusion, 39, 168–177.
   Web of Science ®Google Scholar
• Acar, E., Bro, R. and Smilde, A.K. (2015) Data fusion in metabolomics using coupled matrix and tensor factorizations. Proceedings of the IEEE, 103(9), 1602–1620.
   Web of Science ®Google Scholar
• Acar, E. Papalexakis, E.E., Gürdeniz, G., Rasmussen, M.A., Lawaetz, A.J., Nilsson, M. and Bro, R. (2014) Structure-revealing data fusion. BMC Bioinformatics, 15(1), 1–17.
   Google Scholar
• Acar, E., Rasmussen, M.A. Savorani, F., Naes, T. and Bro, R. (2013) Understanding data fusion within the framework of coupled matrix and tensor factorizations. Chemometrics and Intelligent Laboratory Systems, 129, 53–63.
   Web of Science ®Google Scholar
• Adhikari, S. Lecci, F., Becker, J.T., Junker, B.W., Kuller, L.H., Lopez, O.L. and Tibshirani, R.J. (2019) High-dimensional longitudinal classification with the multinomial fused lasso. Statistics in Medicine, 38(12), 2184–2205.
   Web of Science ®Google Scholar
• Albers, D.J., Levine, M.E., Stuart, A., Mamykina, L., Gluckman, B. and Hripcsak, G. (2018) Mechanistic machine learning: How data assimilation leverages physiologic knowledge using Bayesian inference to forecast the future, infer the present, and phenotype. Journal of the American Medical Informatics Association, 25(10), 1392–1401.
   Web of Science ®Google Scholar
• Argelaguet, R., Velten, B., Arnol, D., Dietrich, S., Zenz, T., Marioni, J.C., Buettner, F., Huber, W. and Stegle, O. (2018) Multi-Omics Factor Analysis–A framework for unsupervised integration of multi-omics data sets. Molecular Systems Biology, 14(6), e8124.
   Web of Science ®Google Scholar
• Atrey, P.K., Hossain, M.A., El Saddik, A. and Kankanhalli, M.S. (2010) Multimodal fusion for multimedia analysis: A survey. Multimedia Systems, 16(6), 345–379.
   Web of Science ®Google Scholar
• Brentan, B.M., Luvizotto Jr, E., Herrera, M., Izquierdo, J. and Pérez-García, R. (2017) Hybrid regression model for near real-time urban water demand forecasting. Journal of Computational and Applied Mathematics, 309, 532–541.
   Web of Science ®Google Scholar
• Bro, R. (1996) Multiway calibration. multilinear PLS. Journal of Chemometrics, 10(1), 47–61.
   Web of Science ®Google Scholar
• Bro, R. et al. (1997) Parafac. tutorial and applications. Chemometrics and Intelligent Laboratory Systems, 38(2), 149–172.
   Web of Science ®Google Scholar
• Calhoun, V.D. and Sui, J. (2016) Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 1(3), 230–244.
   Google Scholar
• Candès, E.J., Li, S., Ma, Y. and Wright, J. (2011) Robust principal component analysis? Journal of the ACM (JACM), 58(3), 1–37, 2011.
   Web of Science ®Google Scholar
• Castellano, G., Kessous, L. and Caridakis, G. (2008) Emotion recognition through multiple modalities: Face, body gesture, speech, in Affect and Emotion in Human-Computer Interaction, Springer, Milwaukee, WI, pp. 92–103.
   Google Scholar
• Chehade, A., Song, C., Liu, K., Saxena, A. and Zhang, X. (2018) A data-level fusion approach for degradation modeling and prognostic analysis under multiple failure modes. Journal of Quality Technology, 50(2), 150–165.
   Web of Science ®Google Scholar
• Chen, D. and Irwin, D. (2017) Black-box solar performance modeling: Comparing physical, machine learning, and hybrid approaches. ACM SIGMETRICS Performance Evaluation Review, 45(2), 79–84.
   Google Scholar
• Chlaily, S., Amblard, P.-O., Michel, O. and Jutten, C. (2016) Impact of noise correlation on multimodality, in 2016 24th European Signal Processing Conference (EUSIPCO), IEEE Press, Piscataway, NJ, pp. 195–199.
   Google Scholar
• Clifton, S.M., Kang, C., Li, J.J., Long, Q., Shah, N. and Abrams, D.M. (2017) Hybrid statistical and mechanistic mathematical model guides mobile health intervention for chronic pain. Journal of Computational Biology, 24(7), 675–688.
   Web of Science ®Google Scholar
• DenseNet121. (2018) Densenet121. https://github.com/keras-team/keras-applications/blob/master/keras_applications/densenet.py (accessed 27 December 2020).
   Google Scholar
• Dong, D., Li, Z., Rahman, S.M.M. and Vega, R. (2016) A hybrid model approach for forecasting future residential electricity consumption. Energy and Buildings, 117, 341–351.
   Web of Science ®Google Scholar
• Elhamifar, E. (2016) High-rank matrix completion and clustering under self-expressive models, in Advances in Neural Information Processing Systems, Barcelona, Spain, pp. 73–81.
   Google Scholar
• Fang, X., Gebraeel, N.Z. and Paynabar, N. (2017) Scalable prognostic models for large-scale condition monitoring applications. IISE Transactions, 49(7), 698–710.
   Web of Science ®Google Scholar
• Fang, X., Paynabar, K. and Gebraeel, N. (2019) Image-based prognostics using penalized tensor regression. Technometrics, 61(3), 369–384.
   Web of Science ®Google Scholar
• Fang, X., Zhou, R. and Gebraeel, N. (2015) An adaptive functional regression-based prognostic model for applications with missing data. Reliability Engineering & System Safety, 133, 266–274.
   Web of Science ®Google Scholar
• Fernandez-Beltran, R., Haut, J.M., Paoletti, M.E., Plaza, J., Plaza, A. and Pla, F. (2018a) Multimodal probabilistic latent semantic analysis for sentinel-1 and sentinel-2 image fusion. IEEE Geoscience and Remote Sensing Letters, 15(9), 1347–1351.
   Google Scholar
• Fernandez-Beltran, R., Haut, J.M., Paoletti, M.E., Plaza, J., Plaza, A. and Pla, F. (2018b) Remote sensing image fusion using hierarchical multimodal probabilistic latent semantic analysis. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(12), 4982–4993.
   Web of Science ®Google Scholar
• Gahrooei, M.R., Paynabar, K., Pacella, M. and Colosimo, M.B. (2019) An adaptive fused sampling approach of high-accuracy data in the presence of low-accuracy data. IISE Transactions, 51(11), 1251–1264.
   Web of Science ®Google Scholar
• Gahrooei, M.R., Paynabar, K., Pacella, M. and Shi, J. (2019) Process modeling and prediction with large number of high-dimensional variables using functional regression. IEEE Transactions on Automation Science and Engineering, 17(2), 684–696.
   Web of Science ®Google Scholar
• Gahrooei, M.R., Yan, H., Paynabar, K. and Shi, J. (2020) Multiple tensor-on-tensor regression: An approach for modeling processes with heterogeneous sources of data. Technometrics, 63(2), 147–159.
   Web of Science ®Google Scholar
• Galán, C.O., Lasheras, F.S., de Cos Juez, F.J. and Sánchez, A.B. (2017) Missing data imputation of questionnaires by means of genetic algorithms with different fitness functions. Journal of Computational and Applied Mathematics, 311, 704–717.
   Web of Science ®Google Scholar
• Gao, F., Wu, T., Li, J., Zheng, B., Ruan, L., Shang, D. and Patel, B. (2018) SD:CNN: A shallow-deep CNN for improved breast cancer diagnosis. Computerized Medical Imaging and Graphics, 70, 53–62.
   PubMed Web of Science ®Google Scholar
• Gao, H. (2020) Coordination and Privacy Preservation in Multi-Agent Systems (Doctoral dissertation, Clemson University).
   Google Scholar
• Gao, J., Li, P., Chen, Z. and Zhang, J. (2020) A survey on deep learning for multimodal data fusion. Neural Computation, 32(5), 829–864.
   PubMed Web of Science ®Google Scholar
• Gaskin, C.J. and Happell, B. (2014) On exploratory factor analysis: A review of recent evidence, an assessment of current practice, and recommendations for future use. International Journal of Nursing Studies, 51(3), 511–521.
   PubMed Web of Science ®Google Scholar
• Gaw, N., Hawkins-Daarud, A., Hu, L.S., Yoon, H., Wang, L., Xu, Y., Jackson, P.R., Singleton, K.W., Baxter, L.C., Eschbacher, J. et al. (2019) Integration of machine learning and mechanistic models accurately predicts variation in cell density of glioblastoma using multiparametric mri. Scientific Reports, 9(1), 1–9.
   PubMedGoogle Scholar
• Gaw, N., Schwedt, T.J., Chong, C.D., Wu, T. and Li, J. (2018) A clinical decision support system using multi-modality imaging data for disease diagnosis. IISE Transactions on Healthcare Systems Engineering, 8(1), 36–46.
   Google Scholar
• Georghiades, A.S., Belhumeur, P.N. and Kriegman, D.J. (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.
   Web of Science ®Google Scholar
• Gorgannejad, S., Gahrooei, M.R., Paynabar, K. and Neu, R.W. (2019) Quantitative prediction of the aged state of ni-base superalloys using pca and tensor regression. Acta Materialia, 165, 259–269.
   Web of Science ®Google Scholar
• Guo, H., Wang, S., Tie, Y., Qi, K. and Guan, L. (2018) Joint intermodal and intramodal correlation preservation for semi-paired learning. Pattern Recognition, 81, 36–49.
   Web of Science ®Google Scholar
• He, C., Liu, Q., Li, H. and Wang, H. (2010) Multimodal medical image fusion based on ihs and pca. Procedia Engineering, 7, 280–285.
   Google Scholar
• He, D., Wang, Z., Yang, L. and Dai, W. (2017) Study on missing data imputation and modeling for the leaching process. Chemical Engineering Research and Design, 124, 1–19.
   Web of Science ®Google Scholar
• Ho, J.C., Ghosh, J. and Sun, J. (2014) Marble: high-throughput phenotyping from electronic health records via sparse nonnegative tensor factorization, in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 115–124.
   Google Scholar
• Hotelling, H. (1936) Relations between two sets of variates. Biometrika, 28(3/4), 321–377.
   Google Scholar
• Ilarshman, RA. (1970) Foundations of the parafac procedure: Models and methods for an “explanatory” multi-mode factor analysis. UCLA Working Papers in Phonetics, 16, 1–84.
   Google Scholar
• InceptionV3. Inceptionv3. https://github.com/keras-team/keras-applications/blob/master/keras_applications/inception_v3.py (accessed 27 December 2020).
   Google Scholar
• Joze, H.R.V., Shaban, A., Iuzzolino, M.L. and Koishida, K. (2020) MMTM: Multimodal transfer module for CNN fusion, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE/CVF, Seattle, WA, pp. 13289–13299.
   Google Scholar
• Kahou, S.E., Bouthillier, X., Lamblin, P., Gulcehre, C., Michalski, V., Konda, K., Jean, S., Froumenty, P., Dauphin, Y., Boulanger-Lewandowski, N., et al. (2016) Emonets: Multimodal deep learning approaches for emotion recognition in video. Journal on Multimodal User Interfaces, 10(2), 99–111.
   Web of Science ®Google Scholar
• Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R. and Fei-Fei, L. (2014) Large-scale video classification with convolutional neural networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE/CVF, Columbus, OH, pp. 1725–1732.
   Google Scholar
• Kefayati, M., Talebi, M.S., Khalaj, B.H. and Rabiee, H.R. (2007a) Secure consensus averaging in sensor networks using random offsets, in 2007 IEEE International Conference on Telecommunications and Malaysia International Conference on Communications, IEEE, Penang, Malaysia, pp. 556–560.
   Google Scholar
• Kefayati, M., Talebi, M.S., Rabiee, H.R. and Khalaj, B.H. (2007b) On secure consensus information fusion over sensor networks, in 2007 IEEE/ACS International Conference on Computer Systems and Applications, IEEE/ACS, Amman, Jordan, pp. 108–115.
   Google Scholar
• Kettenring, J.R. (1971) Canonical analysis of several sets of variables. Biometrika, 58(3), 433–451.
   Web of Science ®Google Scholar
• Khaleghi, B., Khamis, A., Karray, F.O. and Razavi, S.N. (2013) Multisensor data fusion: A review of the state-of-the-art. Information Fusion, 14(1), 28–44.
   Web of Science ®Google Scholar
• Khan, S.A., Virtanen, S., Kallioniemi, O.P., Wennerberg, K., Poso, A. and Kaski, S. (2014) Identification of structural features in chemicals associated with cancer drug response: A systematic data-driven analysis. Bioinformatics, 30(17), i497–i504.
   Web of Science ®Google Scholar
• Khasha, R., Sepehri, M.M. and Mahdaviani, S.A. (2019) An ensemble learning method for asthma control level detection with leveraging medical knowledge-based classifier and supervised learning. Journal of Medical Systems, 43(6), 1–15.
   Web of Science ®Google Scholar
• Kibble, M., Khan, S.A., Saarinen, N., Iorio, F., Saez-Rodriguez, J., Mäkelä, S. and Aittokallio, T. (2016) Transcriptional response networks for elucidating mechanisms of action of multitargeted agents. Drug Discovery Today, 21(7), 1063–1075.
   Web of Science ®Google Scholar
• Klami, A., Virtanen, S. and Kaski. S. (2013) Bayesian canonical correlation analysis. Journal of Machine Learning Research, 14(Apr), 965–1003.
   Google Scholar
• Kolda, T.G. (2006) Multilinear operators for higher-order decompositions. Technical report, Sandia National Laboratories.
   Google Scholar
• Kumar, M., Garg, D.P. and Zachery, R.A. (2007) A method for judicious fusion of inconsistent multiple sensor data. IEEE Sensors Journal, 7(5), 723–733.
   Web of Science ®Google Scholar
• Kumar, P. and Diwakar, M. (2021) A novel approach for multimodality medical image fusion over secure environment. Transactions on Emerging Telecommunications Technologies, 32(2), e3985.
   Web of Science ®Google Scholar
• Lahat, D., Adalý, T. and Jutten, C. (2014) Challenges in multimodal data fusion, in 2014 22nd European Signal Processing Conference (EUSIPCO), IEEE Aerospace and Electronic Systems Society (AESS), Lisbon, Portugal, pp. 101–105.
   Google Scholar
• Lahat, D., Adali, T. and Jutten, C. (2015) Multimodal data fusion: an overview of methods, challenges, and prospects. Proceedings of the IEEE, 103(9), 1449–1477.
   Web of Science ®Google Scholar
• Lahti, L., Schäfer, M., Klein, H.-U., Bicciato, S. and Dugas, M. (2013) Cancer gene prioritization by integrative analysis of MRNA expression and DNA copy number data: A comparative review. Briefings in Bioinformatics, 14(1), 27–35.
   Web of Science ®Google Scholar
• Lampert, C.H. and Krömer, O. (2010) Weakly-paired maximum covariance analysis for multimodal dimensionality reduction and transfer learning, in European Conference on Computer Vision, Springer, Heraklion, Crete, Greece, pp. 566–579.
   Google Scholar
• Li, D., Dimitrova, N., Li, M. and Sethi, I.K. (2003) Multimedia content processing through cross-modal association, in Proceedings of the Eleventh ACM International Conference on Multimedia, ACM, Berkeley, CA, pp. 604–611.
   Google Scholar
• Li, J., Choi, J., Perros, L., Sun, J. and Vuduc, R. (2017a) Model-driven sparse CP decomposition for higher-order tensors, in 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS), IEEE Press, Piscataway, NJ, pp. 1048–1057.
   Google Scholar
• Li, J. and Shi, J. (2007) Knowledge discovery from observational data for process control using causal Bayesian networks. IIE Transactions, 39(6), 681–690.
   Web of Science ®Google Scholar
• Li, N., Gebraeel, N., Lei, Y., Fang, X., Cai, X. and Yan, T. (2021) Remaining useful life prediction based on a multi-sensor data fusion model. Reliability Engineering & System Safety, 208, 107249.
   Web of Science ®Google Scholar
• Li, Q. and Li, L. (2019) Integrative factor regression and its inference for multimodal data analysis. arXiv preprint arXiv:1911.04056.
   Google Scholar
• Li, R., Zhang, W., Suk, H.-Il, Wang, L., Li, J., Shen, D. and Ji, S. (2014) Deep learning based imaging data completion for improved brain disease diagnosis, In International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI, Boston, MA, pp. 305–312.
   Google Scholar
• Li, S., Wang, W., Qi, H., Ayhan, B., Kwan, C. and Vance, S. (2015) Low-rank tensor decomposition based anomaly detection for hyperspectral imagery, in 2015 IEEE International Conference on Image Processing (ICIP), IEEE Press, Piscataway, NJ, pp. 4525–4529.
   Google Scholar
• Li, W., Cerise, J.E., Yang, Y. and Han, H. (2017) Application of t-sne to human genetic data. Journal of Bioinformatics and Computational Biology, 15(04),1750017.
   Google Scholar
• Li, Y., Wu, F.-X. and Ngom, A. (2018) A review on machine learning principles for multi-view biological data integration. Briefings in Bioinformatics, 19(2), 325–340.
   PubMed Web of Science ®Google Scholar
• Liberman, Y., Samuels, R., Alpert, P. and Messer, H. (2014) New algorithm for integration between wireless microwave sensor network and radar for improved rainfall measurement and mapping. Atmospheric Measurement Techniques, 7(10), 3549–3563.
   Web of Science ®Google Scholar
• Liu, D., Clemente, L., Poirier, C., Ding, X., Chinazzi, M., Davis, J.T., Vespignani, A. and Santillana, M. (2020a) A machine learning methodology for real-time forecasting of the 2019-2020 covid-19 outbreak using internet searches, news alerts, and estimates from mechanistic models. arXiv preprint arXiv:2004.04019.
   Google Scholar
• Liu, K., Chehade, A., and Song, C. (2015) Optimize the signal quality of the composite health index via data fusion for degradation modeling and prognostic analysis. IEEE Transactions on Automation Science and Engineering, 14(3), 1504–1514.
   Web of Science ®Google Scholar
• Liu, J., Li, T., Xie, P., Du, S., Teng, F. and Yang, X. (2020) Urban big data fusion based on deep learning: An overview. Information Fusion, 53, 123–133.
   Web of Science ®Google Scholar
• Liu, K., Gebraeel, N.Z. and Shi, J. (2013) A data-level fusion model for developing composite health indices for degradation modeling and prognostic analysis. IEEE Transactions on Automation Science and Engineering, 10(3), 652–664.
   Web of Science ®Google Scholar
• Liu, K. and Huang, S. (2014) Integration of data fusion methodology and degradation modeling process to improve prognostics. IEEE Transactions on Automation Science and Engineering, 13(1), 344–354.
   Web of Science ®Google Scholar
• Liu, M., Zhang, J., Yap, P.-T. and Shen, D. (2016) Diagnosis of Alzheimer’s disease using view-aligned hypergraph learning with incomplete multi-modality data, in International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI, Athens, Greece, pp. 308–316.
   Google Scholar
• Liu, X., Chen, K., Wu, T., Weidman, D., Lure, F. YM, Li, J., Alzheimer’s Disease Neuroimaging Initiative (ADNI), et al. (2020c) A novel transfer learning model for predictive analytics using incomplete multimodality data. medRxiv.
   Google Scholar
• Liu, Z. and Guo, Y. (2018) A hybrid approach to integrate machine learning and process mechanics for the prediction of specific cutting energy. CIRP Annals, 67(1), 57–60.
   Web of Science ®Google Scholar
• Liu, Z., Zhang, W., Quek, T.Q.S. and Lin, S. (2017) Deep fusion of heterogeneous sensor data, in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE Press, Piscataway, NJ, pp. 5965–5969.
   Google Scholar
• Lock, E.F. (2018) Tensor-on-tensor regression. Journal of Computational and Graphical Statistics, 27(3), 638–647.
   PubMed Web of Science ®Google Scholar
• Mak, S., Sung, C.-L., Wang, X., Yeh, S.-T., Chang, Y.-H., Joseph, V.R., Yang, V. and Wu, C.F.J. (2018) An efficient surrogate model for emulation and physics extraction of large eddy simulations. Journal of the American Statistical Association, 113(524), 1443–1456.
   Web of Science ®Google Scholar
• Mascheroni, P., Alfonso, J.C.L., Meyer-Hermann, M. and Hatzikirou, H. (2020) Bayesian combination of mechanistic modeling and machine learning (BAM3): Improving clinical tumor growth predictions. bioRxiv.
   Google Scholar
• Meng, Y., Yu, S., Zhang, J., Qin, J., Dong, Z., Lu, G. and Pang, H. (2019) Hybrid modeling based on mechanistic and data-driven approaches for cane sugar crystallization. Journal of Food Engineering, 257, 44–55.
   Web of Science ®Google Scholar
• Moin, A., Bhateja, V. and Srivastava, A. (2016) Weighted-PCA based multimodal medical image fusion in contourlet domain, in Proceedings of the International Congress on Information and Communication Technology, Springer, IEEE, Udaipur, India pp. 597–605.
   Google Scholar
• Neverova, N., Wolf, C., Taylor, G. and Nebout, F. (2015) Moddrop: Adaptive multi-modal gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(8), 1692–1706.
   Web of Science ®Google Scholar
• Ng, A. (2011) Sparse autoencoder. CS294A Lecture Notes, 72(2011), 1–19.
   Google Scholar
• Paynabar, K., Jin, J. and Reed, M.P. (2015) Informative sensor and feature selection via hierarchical nonnegative garrote. Technometrics, 57(4), 514–523.
   Web of Science ®Google Scholar
• Peng, Y., Qi, J., Huang, X. and Yuan, Y. (2017) CCL: Cross-modal correlation learning with multigrained fusion by hierarchical network. IEEE Transactions on Multimedia, 20(2), 405–420.
   Web of Science ®Google Scholar
• Pérez-Rúa, J.-M., Vielzeuf, V., Pateux, S., Baccouche, M. and Jurie, F. (2019) MAFS: Multimodal fusion architecture search, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway, NJ, pp. 6966–6975.
   Google Scholar
• Prakash, O., Park, C.M., Khare, A., Jeon, M. and Gwak, J. (2019) Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform. Optik, 182, 995–1014.
   Web of Science ®Google Scholar
• Qi, J., Yang, P., Newcombe, L., Peng, X., Yang, Y. and Zhao, Z. (2020) An overview of data fusion techniques for internet of things enabled physical activity recognition and measure. Information Fusion, 55, 269–280.
   Web of Science ®Google Scholar
• Rajalingam, B. and Priya, R. (2017) Multimodality medical image fusion based on hybrid fusion techniques. International Journal of Engineering and Manufacturing Science, 7(1), 22–29.
   Google Scholar
• Ramachandram, D., Lisicki, M., Shields, T.J., Amer, M.R. and Taylor, G.W. (2018) Bayesian optimization on graph-structured search spaces: Optimizing deep multimodal fusion architectures. Neurocomputing, 298, 80–89.
   Web of Science ®Google Scholar
• Ramachandram, D. and Taylor, G.W. (2017) Deep multimodal learning: A survey on recent advances and trends. IEEE Signal Processing Magazine, 34(6), 96–108.
   Web of Science ®Google Scholar
• Rankawat, S.A. and Dubey, R. (2017) Robust heart rate estimation from multimodal physiological signals using beat signal quality index based majority voting fusion method. Biomedical Signal Processing and Control, 33, 201–212.
   Web of Science ®Google Scholar
• Gilan, S.S., Goyal, N. and Dilkina, B. (2016) Active learning in multi-objective evolutionary algorithms for sustainable building design, in Proceedings of the Genetic and Evolutionary Computation Conference 2016, ACM, Denver, CO, pp. 589–596.
   Google Scholar
• Sagi, O. and Rokach, L. (2018) Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1249.
   Web of Science ®Google Scholar
• Said, A.B., Mohamed, A., Elfouly, T., Harras, K. and Wang, Z.J. (2017) Multimodal deep learning approach for joint EEG-EMG data compression and classification, in 2017 IEEE Wireless Communications and Networking Conference (WCNC), IEEE Press, Piscataway, NJ, pp.1–6.
   Google Scholar
• Samareh, A., Jin, Y., Wang, Z., Chang, X. and Huang, S. (2018) Detect depression from communication: How computer vision, signal processing, and sentiment analysis join forces. IISE Transactions on Healthcare Systems Engineering, 8(3), 196–208.
   Google Scholar
• Schmitt, M. and Zhu, X.X. (2016) Data fusion and remote sensing: An ever-growing relationship. IEEE Geoscience and Remote Sensing Magazine, 4(4), 6–23.
   Web of Science ®Google Scholar
• Seyyedi, H. (2010) Comparing satellite derived rainfall with ground based radar for Northwestern Europe. University of Twente Faculty of Geo-Information and Earth Observation (ITC).
   Google Scholar
• Si, B., Schwedt, T.J., Chong, C.D., Wu, T. and Li, J. (2020) A novel hierarchically-structured factor mixture model for cluster discovery from multi-modality data. IISE Transactions, 53(7), 1–13.
   Web of Science ®Google Scholar
• Siegel, M. and Wu, H. (2004) Confidence fusion, in IEEE International Workshop on Robot Sensing, IEEE Press, Piscataway, NJ, pp. 96–99.
   Google Scholar
• Simonyan, K. and Zisserman, A. (2014) Two-stream convolutional networks for action recognition in videos. Advances in Neural Information Processing Systems, 27, 568–576.
   Google Scholar
• Şimşekli, U., Ermiş, B., Cemgil, A.T. and Acar, E. (2013) Optimal weight learning for coupled tensor factorization with mixed divergences, in 21st European Signal Processing Conference (EUSIPCO 2013), IEEE Press, Piscataway, NJ, pp. 1–5.
   Google Scholar
• Song, C. and Liu, K. (2018) Statistical degradation modeling and prognostics of multiple sensor signals via data fusion: A composite health index approach. IISE Transactions, 50(10), 853–867.
   Web of Science ®Google Scholar
• Song, C., Liu, K. and Zhang, X. (2019) A generic framework for multisensor degradation modeling based on supervised classification and failure surface. IISE Transactions, 51(11), 1288–1302.
   Web of Science ®Google Scholar
• Srivastava, N. and Salakhutdinov, R. (2012) ldquo, multimodal learning with deep Boltzmann machines, in Proceedings of Neural Information and Processing Systems. Neural Information Processing Systems, Lake Tahoe, Nevada, pp. 2222–2230.
   Google Scholar
• Suk, H.-Il, Lee, S.-W., Shen, D., Alzheimer’s Disease Neuroimaging Initiative, et al. (2017) Deep ensemble learning of sparse regression models for brain disease diagnosis. Medical Image Analysis, 37, 101–113.
   Web of Science ®Google Scholar
• Tmazirte, N.A., El Najjar, M.E., Smaili, C. and Pomorski, D. (2013) Dynamical reconfiguration strategy of a multi sensor data fusion algorithm based on information theory, in 2013 IEEE Intelligent Vehicles Symposium (IV), IIE Press, Piscataway, NJ, pp. 896–901.
   Google Scholar
• Tucker, L.R. (1958) An inter-battery method of factor analysis. Psychometrika, 23(2),111–136.
   Web of Science ®Google Scholar
• Tucker, L.R. (1964) The extension of factor analysis to three-dimensional matrices. Contributions to Mathematical Psychology, 110119.
   Google Scholar
• Van der Maaten, L. and Hinton, G. (2008) Visualizing data using t-SNE. Journal of Machine Learning Research, 9, 2579–2605.
   Web of Science ®Google Scholar
• Van Mechelen, I. and Smilde, A.K. (2010) A generic linked-mode decomposition model for data fusion. Chemometrics and Intelligent Laboratory Systems, 104(1), 83–94.
   Web of Science ®Google Scholar
• Vielzeuf, V., Lechervy, A., Pateux, S. and Jurie, F. (2018) Centralnet: A multilayer approach for multimodal fusion, in Proceedings of the European Conference on Computer Vision (ECCV), ECCV, Munich, Germany, pp. 1–15.
   Google Scholar
• Virtanen, V., Klami, A., Khan, S. and Kaski, S. (2012) Bayesian group factor analysis, in Artificial Intelligence and Statistics, PMLR, La Palma, Canary Islands, pp. 1269–1277.
   Google Scholar
• Wang, J., Ma, Y., Zhang, L., Gao, R.X. and Wu, D. (2018) Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 48, 144–156.
   Web of Science ®Google Scholar
• Wang, Y., Chen, R., Ghosh, J., Denny, J.C., Kho, A., Chen, Y., Malin, B.A. and Sun, J. (2015) Rubik: Knowledge guided tensor factorization and completion for health data analytics, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1265–1274.
   Google Scholar
• Wang, Y., Guan, L. and Venetsanopoulos, A.N. (2012) Kernel cross-modal factor analysis for information fusion with application to bimodal emotion recognition. IEEE Transactions on Multimedia, 14(3), 597–607.
   Web of Science ®Google Scholar
• White, M. and Schuurmans, D. (2012) Generalized optimal reverse prediction, in Artificial Intelligence and Statistics, PMLR, La Palma, Canary Islands, pp. 1305–1313.
   Google Scholar
• White, M., Zhang, X., Schuurmans, D. and Yu, Y.-l. (2012) Convex multi-view subspace learning, in Advances in Neural Information Processing Systems, Neural Information Processing Systems, Lake Tahoe, Nevada, pp. 1673–1681.
   Google Scholar
• Wu, D., Pigou, L., Kindermans, P.-J., Le, N.D.-H., Shao, L., Dambre, J. and Odobez, J.-M. (2016) Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(8), 1583–1597.
   PubMed Web of Science ®Google Scholar
• Xiang, S., Yuan, L., Fan, W., Wang, Y., Thompson, P.M., Ye, J., Alzheimer’s Disease Neuroimaging Initiative, et al. (2014) Bi-level multi-source learning for heterogeneous block-wise missing data. NeuroImage, 102, 192–206.
   PubMed Web of Science ®Google Scholar
• Xiao, Y., Wang, Y. and Ding, Z. (2018) The application of heterogeneous information fusion in misalignment fault diagnosis of wind turbines. Energies, 11(7), 1–15.
   Web of Science ®Google Scholar
• Yadav, B., Gopalacharyulu, P., Pemovska, T., Khan, S.A., Szwajda, A., Tang, J., Wennerberg, K. and Aittokallio, T. (2015) From drug response profiling to target addiction scoring in cancer cell models. Disease Models & Mechanisms, 8(10),1255–1264.
   Web of Science ®Google Scholar
• Yan, H., Paynabar, K. and Pacella, M. (2019) Structured point cloud data analysis via regularized tensor regression for process modeling and optimization. Technometrics, 61(3), 385–395.
   Web of Science ®Google Scholar
• Yan, H., Paynabar, K. and Shi, J. (2014) Image-based process monitoring using low-rank tensor decomposition. IEEE Transactions on Automation Science and Engineering, 12(1), 216–227.
   Web of Science ®Google Scholar
• Yokoya, N., Ghamisi, P. and Xia, J. (2017) Multimodal, multitemporal, and multisource global data fusion for local climate zones classification based on ensemble learning, in 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), IEEE Press, Piscataway, NJ, pp. 1197–1200.
   Google Scholar
• Yoon, H. and Li, J. (2018) A novel positive transfer learning approach for telemonitoring of Parkinson’s disease. IEEE Transactions on Automation Science and Engineering, 16(1), 180–191.
   Web of Science ®Google Scholar
• Yue, S., Park, J.G., Liang, Z. and Shi, J. (2020) Tensor mixed effects model with application to nanomanufacturing inspection. Technometrics, 62(1), 116–129.
   Web of Science ®Google Scholar
• Zhang, C. and Ma, Y. (2012) Ensemble Machine Learning: Methods and Applications. Berlin/Heidelberg, Germany: Springer.
   Google Scholar
• Zhang, C., Yan, H., Lee, S. and Shi, J. (2018) Weakly correlated profile monitoring based on sparse multi-channel functional principal component analysis. IISE Transactions, 50(10), 878–891.
   Web of Science ®Google Scholar
• Zhang, C., Yan, H., Lee, S. and Shi, J. (2020) Dynamic multivariate functional data modeling via sparse subspace learning. Technometrics, 1–33.
   Web of Science ®Google Scholar
• Zhang, L., Xie, Y., Xidao, L. and Zhang, X. (2018a) Multi-source heterogeneous data fusion, in 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), IEEE Press, Piscataway, NJ, pp. 47–51.
   Google Scholar
• Zhang, C., Yan, H., Lee, S. and Shi, J. (2018b) Weakly correlated profile monitoring based on sparse multi-channel functional principal component analysis. IISE Transactions, 50(10), 878–891.
   Google Scholar
• Zhang, X., Yu, Y., White, M., Huang, R. and Schuurmans, D. (2011) Convex sparse coding, subspace learning, and semi-supervised extensions, in Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI, San Francisco, CA.
   Google Scholar
• Zhao, Q., Caiafa, C.F., Mandic, D.P., Chao, Z.C., Nagasaka, Y., Fujii, N., Zhang, L. and Cichocki, A. (2012) Higher order partial least squares (hopls): A generalized multilinear regression method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1660–1673.
   Web of Science ®Google Scholar
• Zhou, H., Li, L. and Zhu, H. (2013) Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association, 108(502), 540–552.
   PubMed Web of Science ®Google Scholar
• Zhu, Z., Yin, H., Chai, Y., Li, Y. and Qi, G. (2018) A novel multi-modality image fusion method based on image decomposition and sparse representation. Information Sciences, 432, 516–529.
   Web of Science ®Google Scholar