A fast H.266/QTMT intra coding scheme based on predictions of learned models

References

• Abadi, M., P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, et al. 2016. “TensorFlow: A System for Large-Scale Machine Learning.” In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, Savannah, GA, USA, 2–4 November, 2016. 265–283. New York: ACM Digital Library.
   Google Scholar
• Bossen, F., X. Li, and K. Suehring. 2019. AHG Report: Test Model Software Development (AHG3). Report No. JVET-M0003Marrakech: Joint Video Exploration Team.
   Google Scholar
• Bossen, F., K. Sühring, A. Wieckowski, and S. Liu. 2021. “VVC Complexity and Software Implementation Analysis.” IEEE Transactions on Circuits and Systems for Video Technology 31 (10): 3765–3778. doi:10.1109/TCSVT.2021.3072204.
   Google Scholar
• Boyce, J. X. Li, V. Seregin, and K. Sühring. 2018. JVET Common Test Conditions and Software Reference Configurations. Report No. JVET-J1010. San Diego: Joint Video Exploration Team.
   Google Scholar
• Bross, B. 2018. Versatile Video Coding (Draft 1). Report No. JVET-J1001. San Diego: Joint Video Exploration Team.
   Google Scholar
• Chen, J., H. Sun, J. Katto, X. Zeng, and Y. Fan. 2019. “Fast QTMT Partition Decision Algorithm in VVC Intra Coding Based on Variance and Gradient.” In 2019 Proceedings of IEEE Visual Communications and Image Processing (VCIP), Sydney, NSW, Australia, 01-04 December 2019. 1–4. Washington, DC: IEEE. doi:10.1109/VCIP47243.2019.8965674.
   Google Scholar
• Chen, J., Y. Ye, and S. Kim. 2019. Algorithm Description for Versatile Video Coding and Test Model 7 (VTM 7). Report No. JVET-P2002. Geneva: Joint Video Exploration Team.
   Google Scholar
• Cho, S., and M. Kim. 2013. “Fast CU Splitting and Pruning for Suboptimal CU Partitioning in HEVC Intra Coding.” IEEE Transactions on Circuits and Systems for Video Technology 23 (9): 1555–1564. doi:10.1109/TCSVT.2013.2249017.
   Google Scholar
• Choi, K., T. V. Le, Y. Choi, and J. Y. Lee. 2022. “Low Complexity Intra Coding in Versatile Video Coding.” IEEE Transactions on Consumer Electronics 68 (2): 119–126. doi:10.1109/TCE.2022.3145397.
   Google Scholar
• Correa, G., P. A. Assuncao, L. V. Agostini, and L. A. da Silva Cruz. 2015. “Fast HEVC Encoding Decisions Using Data Mining.” IEEE Transactions on Circuits and Systems for Video Technology 25 (4): 660–673. doi:10.1109/TCSVT.2014.2363753.
   Google Scholar
• Dong, X., L. Shen, M. Yu, and H. Yang. 2021. “Fast Intra Mode Decision Algorithm for Versatile Video Coding.” IEEE Transactions on Multimedia 24:400–414. doi:10.1109/TMM.2021.3052348.
   Google Scholar
• Fan, Y., J. Chen, H. Sun, J. Katto, and M. Jing. 2020. “A Fast QTMT Partition Decision Strategy for VVC Intra Prediction.” Institute of Electrical and Electronics Engineers Access 8:107900–107911. doi:10.1109/ACCESS.2020.3000565.
   Google Scholar
• Farhat, I., W. Hamidouche, A. Grill, D. Ménard, and O. Déforges. 2022. “Adaptive Loop Filter Hardware Design for 4K ASIC VVC Decoders.” IEEE Transactions on Consumer Electronics 68 (2): 107–118. doi:10.1109/TCE.2022.3146272.
   Google Scholar
• Hamidouche, W., P. Philippe, S. A. Fezza, M. Haddou, F. Pescador, and D. Menard. 2022. “Hardware-Friendly Multiple Transform Selection Module for the VVC Standard.” IEEE Transactions on Consumer Electronics 68 (2): 96–106. doi:10.1109/TCE.2022.3163345.
   Google Scholar
• Huang, Y. H., J. J. Chen, and Y. H. Tsai. 2021. “Speed Up H.266/QTMT Intra-Coding Based on Predictions of ResNet and Random Forest Classifier.” 2021 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 10-12 January 2021. 1–6. Washington, DC: IEEE. doi:10.1109/ICCE50685.2021.9427626.
   Google Scholar
• Jin, Z., P. An, C. Yang, and L. Shen. 2018. “Fast QTBT Partition Algorithm for Intra Frame Coding Through Convolutional Neural Network.” Institute of Electrical and Electronics Engineers Access 6:54660–54673. doi:10.1109/ACCESS.2018.2872492.
   Google Scholar
• Kingma, D. P., and J. Ba. 2014. “Adam: A Method for Stochastic Optimization.” ArXiv Preprint 1412:6980. doi:10.48550/arXiv.1412.6980.
   Google Scholar
• Lin, H. K., C. C. Sun, M. H. Sheu, and M. Berecovic. 2020. “A New Low-Complexity Approximate DCT for Image and Video Compression.” Journal of the Chinese Institute of Engineers 43 (6): 580–591. doi:10.1080/02533839.2020.1786463.
   Web of Science ®Google Scholar
• Lin, T. L., H. Y. Jiang, J. Y. Huang, and P. C. Chang. 2020. “Fast Intra Coding Unit Partition Decision in H. 266/FVC Based on Spatial Features.” Journal of Real-Time Image Processing 17:493–510. doi:10.1007/s11554-018-0794-8.
   Google Scholar
• Liu, Z., T. Li, Y. Chen, K. Wei, M. Xu, and H. Qi. 2023. “Deep Multi-Task Learning Based Fast Intra-Mode Decision for Versatile Video Coding.” IEEE Transactions on Circuits and Systems for Video Technology 33 (10): 6101–6116. doi:10.1109/TCSVT.2023.3262733.
   Google Scholar
• Liu, Z., X. Yu, Y. Gao, S. Chen, X. Ji, and D. Wang. 2016. “CU Partition Mode Decision for HEVC Hardwired Intra Encoder Using Convolution Neural Network.” IEEE Transactions on Image Processing 25 (11): 5088–5103.doi:10.1109/TIP.2016.2601264.
   PubMedGoogle Scholar
• Mercat, A., M. Viitanen, and J. Vanne. 2020. “UVG Dataset: 50/120fps 4K Sequences for Video Codec Analysis and Development.” In the 11th Proceedings of ACM Multimedia Systems Conference, Istanbul, Turkey, 8 - 11 June, 2020. 297–302. New York: ACM Digital Library.doi:10.1145/3339825.3394937.
   Google Scholar
• Min, B., and R. C. C. Cheung. 2015. “A Fast CU Size Decision Algorithm for the HEVC Intra Encoder.” IEEE Transactions on Circuits and Systems for Video Technology 25 (5): 892–896. doi:10.1109/TCSVT.2014.2363739.
   Google Scholar
• Mu, F., L. Song, X. Yang, and Z. Luo. 2014. “Fast Coding Unit Depth Decision for HEVC.” In Proceedings of 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Chengdu, China, 14-18 July 2014. 1–6. Washington, DC:. IEEE. doi:10.1109/ICMEW.2014.6890647.
   Google Scholar
• Pateux, S., and J. Jung. 2007. “An Excel Add-In for Computing Bjontegaard Metric and Additional Performance Analysis.” In ITU-T Q.6/16 Video Coding Experts Group (VCEG) 31st Meeting, Marrakech, Morocco, 15-16 January 2007. VCEG–AE07. USA: Video Coding Experts Group. https://docs.google.com/viewer?url=https%3A%2F%2Fwww.itu.int%2Fwftp3%2Fav-arch%2Fvideo-site%2F0701_Mar%2FVCEG-AE01r1.doc.
   Google Scholar
• Pedregosa, F., G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, et al. 2011. “Scikit-Learn: Machine Learning in Python.” Journal of Machine Learning Research 12 (85): 2825–2830.
   Google Scholar
• Peng, Z., L. Shen, Q. Ding, X. Dong, and L. Zheng. 2023. “Block-Dependent Partition Decision for Fast Intra Coding of VVC.” IEEE Transactions on Consumer Electronics, 70 (1): 277–289. doi:10.1109/TCE.2023.3324794.
   Google Scholar
• Shen, L., H. Yang, and S. Wang. 2022. “Effective QTMT Partition Decision Algorithm for VVC Intercoding.” In Proceedings of 2022 IEEE 2014 International Workshop on Multimedia Signal Processing (MMSP) Shanghai, China, 26–28 September 2022. 1–6. Washington, DC: IEEE. doi:10.1109/MMSP55362.2022.9949302.
   Google Scholar
• Shen, L., Z. Zhang, and Z. Liu. 2014. “Effective CU Size Decision for HEVC Intracoding.” IEEE Transactions on Image Processing 23 (10): 4232–4241. doi:10.1109/TIP.2014.2341927.
   PubMedGoogle Scholar
• Sobel, I., and G. Feldman. 1968. “An Isotropic 3x3 Image Gradient Operator.” Presentation at Stanford AI Project 2014 (02).
   Google Scholar
• Suehring, K., and X. Li. 2016. JVET Common Test Conditions and Software Reference Configurations. Report No. JVET-B1010. San Diego: Joint Video Exploration Team.
   Google Scholar
• Sühring, K. 2019. Guidelines for VVC Reference Software Development. Geneva: Joint Video Exploration Team.
   Google Scholar
• Sullivan, G. J., J.-R. Ohm, W.-J. Han, and T. Wiegand. 2012. “Overview of the High Efficiency Video Coding (HEVC) Standard.” IEEE Transactions on Circuits and Systems for Video Technology 22 (12): 1649–1668. doi:10.1109/TCSVT.2012.2221191.
   Web of Science ®Google Scholar
• Tang, G., M. Jing, X. Zeng, and Y. Fan. 2019. “Adaptive CU Split Decision with Pooling-Variable CNN for VVC Intra Encoding.” In Proceedings of 2019 IEEE Visual Communications and Image Processing (VCIP) Sydney, NSW, Australia, 01–04 December 2019. 1–4. Washington, DC: IEEE. doi:10.1109/VCIP47243.2019.8965679.
   Google Scholar
• Tang, T., C. You, Z. Li, R. Zhang, and H. Zou. 2023. “Ultra-Lightweight CNN Based Fast Intra Prediction for VVC Screen Content Coding.” In Proceedings of 2023 IEEE International Symposium on Circuits and Systems (ISCAS), Monterey, CA, USA, 21–25 May 2023. 1–5. Washington, DC: IEEE. doi:10.1109/ISCAS46773.2023.10181706.
   Google Scholar
• Tech, G., J. Pfaff, H. Schwarz, P. Helle, A. Wieckowski, D. Marpe, and T. Wiegand. 2021. “Fast Partitioning for VVC Intra-Picture Encoding with a CNN Minimizing the Rate-Distortion-Time Cost.” In Proceedings of 2021 Data Compression Conference (DCC), Snowbird, UT, USA, 23–26 March 2021. 3–12. Washington, DC: IEEE.doi:10.1109/DCC50243.2021.00008.
   Google Scholar
• Tissier, A., W. Hamidouche, J. Vanne, F. Galpin, and D. Menard. 2020. “CNN Oriented Complexity Reduction of VVC Intra Encoder.” In Proceedings of 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 25–28 October 2020. 3139–3143. Washington, DC: IEEE. doi:10.1109/ICIP40778.2020.9190797.
   Google Scholar
• Viitanen, M., J. Sainio, A. Mercat, A. Lemmetti, and J. Vanne. 2022. “From HEVC to VVC: The First Development Steps of a Practical Intra Video Encoder.” IEEE Transactions on Consumer Electronics 68 (2): 139–148. doi:10.1109/TCE.2022.3146016.
   Google Scholar
• Wen, Y., K. Zhang, Z. Li, and Y. Qiao. 2016. “A Discriminative Feature Learning Approach for Deep Face Recognition.” In Computer Vision–ECCV 2016 14th European Conference, edited by B. Leibe, J. Matas, N. Sebe, and M. Welling, Amsterdam, The Netherlands, 11–14 October 2016, Vol. 9911, 499–515. Cham: Springer. doi:10.1007/978-3-319-46478-7_31.
   Google Scholar
• Wieckowski, A., J. Brandenburg, T. Hinz, C. Bartnik, V. George, G. Hege, C. Helmrich, et al. 2021. “VVenC: An Open and Optimized VVC Encoder Implementation.” In Proceedings of 2021 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Shenzhen, China, 05–09 July 2021.1–2. Washington, DC: IEEE. doi:10.1109/ICMEW53276.2021.9455944.
   Google Scholar
• Xu, M., T. Li, Z. Wang, X. Deng, R. Yang, and Z. Guan. 2018. “Reducing Complexity of HEVC: A Deep Learning Approach.” IEEE Transactions on Image Processing 27 (10): 5044–5059. doi:10.1109/TIP.2018.2847035.
   Web of Science ®Google Scholar
• Yang, H., L. Shen, X. Dong, Q. Ding, P. An, and G. Jiang. 2020. “Low-Complexity CTU Partition Structure Decision and Fast Intra Mode Decision for Versatile Video Coding.” IEEE Transactions on Circuits and Systems for Video Technology 30 (6): 1668–1682. doi:10.1109/TCSVT.2019.2904198.
   Google Scholar
• Zhou, B., A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. 2016. “Learning Deep Features for Discriminative Localization.” In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016.2921–2929. Washington, DC: IEEE. doi:10.1109/CVPR.2016.319.
   Google Scholar
• Zhu, L., Y. Zhang, Z. Pan, R. Wang, S. Kwong, and Z. Peng. 2017. “Binary and Multi-Class Learning Based Low Complexity Optimization for HEVC Encoding.” IEEE Transactions on Broadcasting 63 (3): 547–561. doi:10.1109/TBC.2017.2711142.
   Web of Science ®Google Scholar