Low cost network traffic measurement and fast recovery via redundant row subspace-based matrix completion

References

• Aksoy, S., & Haralick, R. M. (2001). Feature normalization and likelihood-based similarity measures for image retrieval. Pattern Recognition Letters, 22(5), 563–582. https://doi.org/10.1016/S0167-8655(00)00112-4
   Web of Science ®Google Scholar
• Balcan, M. F. F., & Zhang, H. (2016). Noise-tolerant life-long matrix completion via adaptive sampling. In Advances in neural information processing systems (pp. 2955–2963).
   Google Scholar
• Barford, P., Kline, J., Plonka, D., & Ron, A. (2002). A signal analysis of network traffic anomalies. In Proceedings of the 2nd acm sigcomm workshop on internet measurment (pp. 71–82). ACM Press.
   Google Scholar
• Cai, J. F., Candès, E. J., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982. https://doi.org/10.1137/080738970
   Web of Science ®Google Scholar
• Candès, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717–772. https://doi.org/10.1007/s10208-009-9045-5
   Web of Science ®Google Scholar
• Chen, Y., Bhojanapalli, S., Sanghavi, S., & Ward, R. (2014). Coherent matrix completion. In International conference on machine learning (pp. 674–682). JMLR.org.
   Google Scholar
• Chen, Y., Bhojanapalli, S., Sanghavi, S., & Ward, R. (2015). Completing any low-rank matrix, provably. The Journal of Machine Learning Research, 16(1), 2999–3034. https://dl.acm.org/doi/10.5555/2789272.2912096
   Google Scholar
• Chen, Y. C., Qiu, L., Zhang, Y., Xue, G., & Hu, Z. (2014). Robust network compressive sensing. In Proceedings of the 20th annual international conference on mobile computing and networking (pp. 545–556). ACM.
   Google Scholar
• Claise, B. (2004). Cisco systems netflow services export version 9. RFC 3954. Internet Engineering Task Force.
   Google Scholar
• Diao, C., Zhang, D., Liang, W., Li, K. C., Hong, Y., & Gaudiot, J. L. (2022). A novel spatial-temporal multi-scale alignment graph neural network security model for vehicles prediction. IEEE Transactions on Intelligent Transportation Systems, 24(1), 904–914. https://doi.org/10.1109/TITS.2022.3140229
   Web of Science ®Google Scholar
• Hu, N., Zhang, D., Xie, K., Liang, W., Diao, C., & Li, K. C. (2022). Multi-range bidirectional mask graph convolution based GRU networks for traffic prediction. Journal of Systems Architecture, 133, 102775. https://doi.org/10.1016/j.sysarc.2022.102775
   Web of Science ®Google Scholar
• Hu, N., Zhang, D., Xie, K., Liang, W., & M. Y. Hsieh (2022). Graph learning-based spatial-temporal graph convolutional neural networks for traffic forecasting. Connection Science, 34(1), 429–448. https://doi.org/10.1080/09540091.2021.2006607
   Web of Science ®Google Scholar
• Huang, W., Xie, K., & Li, J. (2022). A novel sequence tensor recovery algorithm for quick and accurate anomaly detection. IEEE Transactions on Network Science and Engineering, 9(5), 3531–3545. https://doi.org/10.1109/TNSE.2022.3189365
   Web of Science ®Google Scholar
• Jin, K., Xie, K., Wang, X., Tian, J., Xie, G., Wen, J., & Li, K. (2022). Low cost online network traffic measurement with subspace-based matrix completion. IEEE Transactions on Network Science and Engineering, 10(1), 53–67. https://doi.org/10.1109/TNSE.2022.3201624
   Web of Science ®Google Scholar
• Krishnamurthy, A., & Singh, A. (2013). Low-rank matrix and tensor completion via adaptive sampling. In Advances in neural information processing systems (pp. 836–844).
   Google Scholar
• Krishnamurthy, A., & Singh, A. (2014). On the power of adaptivity in matrix completion and approximation. preprint arXiv:1407.3619.
   Google Scholar
• Lakhina, A., Crovella, M., & Diot, C. (2004). Diagnosing network-wide traffic anomalies. ACM SIGCOMM Computer Communication Review, 34(4), 219–230. https://doi.org/10.1145/1030194.1015492
   Web of Science ®Google Scholar
• Li, X., Xie, K., Wang, X., Xie, G., Li, K., Cao, J., & Wen, J. (2023a). Neighbor graph based tensor recovery for accurate internet anomaly detection. IEEE Transactions on Parallel and Distributed Systems, 34(2), 655–674. https://doi.org/10.1109/TPDS.2022.3227570
   Web of Science ®Google Scholar
• Li, X., Xie, K., Wang, X., Xie, G., Li, K., Cao, J., & Wen, J. (2023b). Tripartite graph aided tensor completion for sparse network measurement. IEEE Transactions on Parallel and Distributed Systems, 34(1), 48–62. https://doi.org/10.1109/TPDS.2022.3213259
   Web of Science ®Google Scholar
• Li, Y., Liang, W., Xie, K., Zhang, D., Xie, S., & Li, K. C. (2023). LightNestle: quick and accurate neural sequential tensor completion via meta learning. In IEEE Infocom 2023-ieee conference on computer communications.
   Google Scholar
• Liang, W., Li, Y., Xie, K., Zhang, D., Li, K. C., Souri, A., & Li, K. (2022). Spatial-temporal aware inductive graph neural network for C-ITS data recovery. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/tits.2022.3156266
   Web of Science ®Google Scholar
• Lin, Z., Chen, M., & Ma, Y. (2010). The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. preprint arXiv:1009.5055.
   Google Scholar
• Liu, G., & Li, P. (2016). Low-rank matrix completion in the presence of high coherence. IEEE Transactions on Signal Processing, 64(21), 5623–5633. https://doi.org/10.1109/TSP.2016.2586753
   Web of Science ®Google Scholar
• Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., & Ma, Y. (2012). Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 171–184. https://doi.org/10.1109/TPAMI.2012.88
   Web of Science ®Google Scholar
• Liu, S., & Wang, Q. (2020). Adaptive coherent sampling for network delay measurement. In ICC 2020-2020 ieee international conference on communications (icc) (pp. 1–6). IEEE.
   Google Scholar
• Markovsky, I. (2018). Low-rank approximation: algorithms, implementation, applications. Springer.
   Google Scholar
• Roughan, M., Thorup, M., & Zhang, Y. (2003). Traffic engineering with estimated traffic matrices. In Proceedings of the 3rd ACM sigcomm conference on internet measurement (pp. 248–258). ACM Press.
   Google Scholar
• Roughan, M., Zhang, Y., Willinger, W., & Qiu, L. (2011). Spatio-temporal compressive sensing and internet traffic matrices (extended version). IEEE/ACM Transactions on Networking, 20(3), 662–676. https://doi.org/10.1109/TNET.2011.2169424
   Web of Science ®Google Scholar
• Tang, B., Tang, M., Xia, Y., & Hsieh, M. Y. (2021). Composition pattern-aware web service recommendation based on depth factorisation machine. Connection Science, 33(4), 870–890. https://doi.org/10.1080/09540091.2021.1911933
   Web of Science ®Google Scholar
• Uhlig, S., Quoitin, B., Lepropre, J., & Balon, S. (2006). Providing public intradomain traffic matrices to the research community. ACM SIGCOMM Computer Communication Review, 36(1), 83–86. https://doi.org/10.1145/1111322.1111341
   Web of Science ®Google Scholar
• Wan, Y., Yi, J., & Zhang, L. (2018). Matrix completion from non-uniformly sampled entries. preprint arXiv:1806.10308.
   Google Scholar
• Wang, M., Li, B., & Li, Z. (2004). sFlow: towards resource-efficient and agile service federation in service overlay networks. In 24th International conference on distributed computing systems, 2004. proceedings (pp. 628–635). IEEE.
   Google Scholar
• Wang, Y., & Singh, A. (2015). Column subset selection with missing data via active sampling. In Artificial intelligence and statistics (pp. 1033–1041). JMLR.org.
   Google Scholar
• Wang, Y., & Singh, A. (2017). Provably correct algorithms for matrix column subset selection with selectively sampled data. The Journal of Machine Learning Research, 18(1), 5699–5740.
   Google Scholar
• Xie, K., Chen, Y., Wang, X., Xie, G., Cao, J., Wen, J., & Sun, J. (2020). Accurate and fast recovery of network monitoring data with GPU-Accelerated tensor completion. IEEE/ACM Transactions on Networking, 28(4), 1601–1614. https://doi.org/10.1109/TNET.90
   Web of Science ®Google Scholar
• Xie, K., Li, X., Wang, X., Xie, G., Wen, J., Cao, J., & Zhang, D. (2017). Fast tensor factorization for accurate internet anomaly detection. IEEE/ACM Transactions on Networking, 25(6), 3794–3807. https://doi.org/10.1109/TNET.2017.2761704
   Web of Science ®Google Scholar
• Xie, K., Li, X., Wang, X., Xie, G., Wen, J., & Zhang, D. (2019). Active sparse mobile crowd sensing based on matrix completion. In Proceedings of the 2019 international conference on management of data (pp. 195–210). ACM.
   Google Scholar
• Xie, K., Ning, X., Wang, X., Xie, D., Cao, J., Xie, G., & Wen, J. (2016). Recover corrupted data in sensor networks: a matrix completion solution. IEEE Transactions on Mobile Computing, 16(5), 1434–1448. https://doi.org/10.1109/TMC.2016.2595569
   Web of Science ®Google Scholar
• Xie, K., Peng, C., Wang, X., Xie, G., & Wen, J. (2017). Accurate recovery of internet traffic data under dynamic measurements. In IEEE infocom 2017-ieee conference on computer communications (pp. 1–9). IEEE.
   Google Scholar
• Xie, K., Tian, J., Xie, G., Zhang, G., & Zhang, D. (2021). Low cost sparse network monitoring based on block matrix completion. In IEEE infocom 2021-ieee conference on computer communications (pp. 1–10). IEEE.
   Google Scholar
• Xie, K., Wang, L., Wang, X., Xie, G., & Wen, J. (2017). Low cost and high accuracy data gathering in WSNs with matrix completion. IEEE Transactions on Mobile Computing, 17(7), 1595–1608. https://doi.org/10.1109/TMC.7755
   Web of Science ®Google Scholar
• Xie, K., Wang, L., Wang, X., Xie, G., Zhang, G., Xie, D., & Wen, J. (2015). Sequential and adaptive sampling for matrix completion in network monitoring systems. In 2015 IEEE conference on computer communications (infocom) (pp. 2443–2451). IEEE.
   Google Scholar
• Xie, K., Xie, R., Wang, X., Xie, G., Zhang, D., & Wen, J. (2022). NMMF-Stream: a fast and accurate stream-processing scheme for network monitoring data recovery. In IEEE infocom 2022-ieee conference on computer communications (pp. 2218–2227). IEEE.
   Google Scholar
• Yu, M., Jose, L., & Miao, R. (2013). Software defined traffic measurement with opensketch. In 10th USENIX symposium on networked systems design and implementation (NSDI 13) (pp. 29–42). USENIX Association.
   Google Scholar
• Zhang, Y., Roughan, M., Duffield, N., & Greenberg, A. (2003). Fast accurate computation of large-scale IP traffic matrices from link loads. ACM SIGMETRICS Performance Evaluation Review, 31(1), 206–217. https://doi.org/10.1145/885651.781053
   Google Scholar
• Zhang, Y., Roughan, M., Lund, C., & Donoho, D. L. (2005). Estimating point-to-point and point-to-multipoint traffic matrices: an information-theoretic approach. IEEE/ACM Transactions on networking, 13(5), 947–960. https://doi.org/10.1109/TNET.2005.857115
   Web of Science ®Google Scholar