A review of challenges and solutions in the design and implementation of deep graph neural networks

Abstract

The study of graph neural networks has revealed that they can unleash new applications in a variety of disciplines using such a basic process that we cannot imagine in the context of other deep learning designs. Many limitations limit their expressiveness, and researchers are working to overcome them to fully exploit the power of graph data. There are a number of publications that explore graph neural networks (GNNs) restrictions and bottlenecks, but the common thread that runs through them all is that they can all be traced back to message passing, which is the key technique we use to train our graph models. We outline the general GNN design pipeline in this study as well as discuss solutions to the over-smoothing problem, categorize the solutions, and identify open challenges for further research.

Abbreviations: CGNN: Continuous Graph Neural Networks; CNN: Convolution NeuralNetwork; DeGNN: Decomposition Graph Neural Network; DGN: Directional GraphNetworks; DGN: Differentiable Group Normalization; DL: Deep Learning; EGAI:Enhancing GNNs by a High-quality Aggregation of Beneficial Information; GAT: GraphAttention Network; GCN: Graph Convolutional Network; GDC: Graph Drop Connect; GDR: Group Distance Ratio; GNN: Graph Neural Network; GRAND: GraphRandom Neural Networks; IIG: Instance Information Gain; MAD: Man AverageDistance; PDE-GCN: Partial Differential Equations-GCN; PTDNet: ParameterizedTopological Denoising network; TDGNN: Tree Decomposition Graph NeuralNetwork;

Keywords:

• Geometric deep learning
• graph neural networks
• over-smoothing
• over-squashing

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Aafaq Mohi ud din

Aafaq Mohi Ud Din received B. Tech Degree in CSE from University of Kashmir, India in 2014 and M.Tech Degree in CS from University of Kashmir India in 2019. He is currently a Research Scholar in the Department of CSE at National Institute of Technology, Srinagar, India. His research interests are in the field of Geometric Deep Learning, Representation Learning.

Shaima Qureshi

Dr. Shaima Qureshi received her Ph.D. from National Institute of Technology Srinagar (NIT Srinagar). She has completed her B.E. (Hons.) Computer Science degree from BITS Pilani, India in 2004. She completed her M.S. in Computer Science from Syracuse University, NY, USA in 2006. She has been working as an Associate Professor in NIT Srinagar since 2008. She is currently guiding Ph.D., Masters and Bachelor of Engineering (B.E.) students. She has 6 patents and 32 publications to her credit. Prior to joining the academic field, she worked as a Senior QA Engineer for two years in the software industry in the USA. Her areas of research include Mobile Networks, Machine learning, and Deep learning.