KG4Py: A toolkit for generating Python knowledge graph and code semantic search

Abstract

In the era of big data, there are numerous duplicate code snippets on the Internet, it is especially necessary to make use of them to build new software projects. In this paper, we present a toolkit (KG4Py) for generating a knowledge graph of Python files in GitHub repositories and conducting semantic search with the knowledge graph. In KG4Py, we remove all duplicate files in 317 K Python files and perform static code analyses of these files by using a concrete syntax tree (CST) to build a code knowledge graph of Python functions. We integrate a pre-trained model with an unsupervised model to generate a new model, and combine this new model with a code knowledge graph for the purpose of searching code snippets with natural language descriptions. The experimental results show that KG4Py achieves good performance in both the construction of the code knowledge graph and the semantic search of code snippets.

KEYWORDS:

• Knowledge graph
• static code analysis
• information extraction
• deep learning
• code semantic search

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 https://github.com/Instagram/LibCST.

2 https://www.sbert.net.

3 https://github.com/saltudelft/CD4Py.

4 https://www.nltk.org.

Additional information

Funding

This work is supported by the Xinjiang Tianshan Youth Project of China [grant number 2020Q019], the National Natural Science Foundation of China [grant number 61562087], and the Doctoral Scientific Research Foundation of Xinjiang Normal University [grant number XJNUBS1905].