Crop field extraction from high resolution remote sensing images based on semantic edges and spatial structure map

Abstract

Crop field boundary extraction is crucial to remote sensing images attained to support agricultural production and planning. In recent years, deep convolutional neural networks (CNNs) have gained significant attention for edge detection tasks. Moreover, transformers have shown superior feature extraction and classification capabilities compared to CNNs due to their self-attention mechanism. We proposed a novel structure that combines full edge extraction with CNNs and enhances connectivity with transformers, consisting of three stages: a) preprocessing the training data; b) training the semantic edge and spatial structure graph models; and c) vectorizing the fusion of semantic edge and spatial structure graph outputs. To cater specifically to high-resolution remote sensing image crop-field boundary extraction, we developed a CNN model called Densification D-LinkNet. Its full-scale skip connections and edge-guided module adapted well to different crop-field boundary features. Additionally, we employed a spatial graph structure generator (Relationformer) based on object detection that directly outputs the structural graph of the crop field boundary. This method relies on good connectivity to repair fragmented edges that may appear in semantic edge detection. Through multiple experiments and comparisons with other edge-detection methods, such as BDCN, DexiNed, PidiNet, and EDTER, we demonstrated that our proposed method can achieve at least 9.77% improvement in boundary intersection over union (IoU) and 2.07% improvement in polygon IoU on two customized datasets. These results indicate the effectiveness and robustness of our approach.

Keywords:

• High-resolution remote sensing images
• image segmentation
• semantic edge detection
• transformer

1. Introduction

Acquiring comprehensive information on crop field plots, including land plotting, crop yield calculations, and agro-development planning, is crucial for enhancing agricultural productivity. The traditional method of delineating crop field boundaries is laborious and time-consuming as it involves on-site visits and manual mapping in conjunction with remote sensing images, which fails to reflect land changes due to urban development (Prishchepov et al. 2018; Xia et al. 2016).

Contemporarily, a broader practice for crop field plot extraction involves the use of image segmentation based on edge detection, region segmentation, and machine learning. Traditional methods for detecting edges rely on low-level local cues to identify changes in curvature, noise, or color, such as the Canny algorithm (Canny 1986), which uses the calculus of variations to optimize a given functional. However, shallow features aim at areas with evident changes primarily and do not consider contextual semantic information, resulting in a large amount of noise and blurred edges. Region segmentation-based methods primarily aim to address the problem of non-closed boundaries in edge detection. Assumedly, adjacent elements inside a region have similar values (Tremeau and Borel 1997). There are two basic methods for region segmentation, including merging and splitting (Fan et al. 2001). The basic steps of region segmentation (Bins et al. 1996) are: a) obtain the initial segmentation result of the image; b) merge similar fragments and divide dissimilar ones; and c) continue splitting until there are no remaining fragments. However, region-based methods are dependent on well-segmented objects. If the segmentation quality of the image object is poor, it leads to reduced accuracy in the extraction of crop-field boundaries.

Convolutional neural networks (CNNs) and other deep learning models have shown tremendous advantages in the field of remote sensing (Volpi and Tuia 2018; Wagner and Oppelt 2020; Waldner and Diakogiannis 2020; Zhang et al. 2021). Currently, three main CNN-based methods are used for crop field detection: semantic segmentation, semantic edge detection, and mixed methods. Semantic segmentation (Ronneberger et al. 2015; Badrinarayanan et al. 2017; Lin et al. 2017; Zhu et al. 2017) recognizes crop fields in remote-sensing images, classifies the pixels into different categories, and assigns a unique identifier to each crop field. For example, Chen et al. (2017) combined deep learning and atrous convolutions to achieve crop field segmentation, which helped researchers further improve (Firdaus-Nawi et al. 2011; Maggiori et al. 2017; Peng et al. 2017; Fu et al. 2019). However, deep convolutional networks are prone to the notorious issues of gradient explosion or vanishing gradients when the depth becomes excessively large. Simultaneously, dilated convolutions may introduce the problem of insufficient resolution. In order to address these challenges, another famous special structure known as U-Net was introduced (Ronneberger et al. 2015). Building upon the basis of FCN, they incorporated an up sampling stage and a feature channel fusion strategy for medical image segmentation. The U-Net structure (Diakogiannis et al. 2020; 2021; Wang et al. 2022) has achieved good results in the delineation of objects. Though semantic segmentation usually yields accurate crop field surface information, the crop field boundaries are not satisfying.

Researchers have employed semantic edge detection methods to acquire precise boundaries (Bertasius et al. 2015; Xie and Tu 2015; Yang et al. 2016; Liu et al. 2017; Zhu et al. 2017; Wang et al. 2019; Su et al. 2021). The semantic edge method generally uses multiple convolutional layers, each of which extracts different combinations of features. Bertasius et al. (2015) employed a branched, fully connected subnetwork in conjunction with five convolutional layers; it employed low-level cues to detect object features and transformed them to high-level cues. But it is very challenging to detect high-quality contour only using convolutional layers. Yang et al. (2016) introduced an encoder-decoder-based FCN for foreground object contour detection while suppressing the background edges and demonstrated superior performance in terms of generalized edge detection across various tasks. The issue that arised with deep convolutional layers, specifically having 6 encoders and 6 decoders, is that the network may struggle to converge when going deeper, Liu et al.(2017) proposed a novel network uses all CNN features from different layers to perform the pixel-wise prediction in an image-to-image fasion and try to obtain accurate prediction in different scale. Researchers also used hierarchical, multi-scale features to improve edge and object detection.

Researchers have shifted their focus to model performance and portability due to the high computational costs associated with the increased number of neurons. In practical applications, fast inference speed is crucial, prompting a need for more efficient models. For instance, Wang et al. (2019) proposed an end-to-end network for detecting occlusion boundaries that performed multiple tasks simultaneously. To address the issue of class imbalance, the network used an attention loss function that assigned different weights to false positives and false negatives. Su et al. (2021) integrated traditional edge detection operations into convolutions using pixel-wise difference convolutions to improve performance. However, for tasks requiring long-range dependencies, it is evidently insufficient. The incorporation of attention mechanisms of the Vision Transformer, especially on larger datasets, outperforms traditional approaches in detecting edges. Pu et al. (2022) used the overall background and detailed local information of images to extract clear object edges. This method effectively solves the edge extraction problem when objects are occluded, and it can handle extreme boundaries and nonboundary class imbalances. These methods show excellent results in segmentation and can identify the accurate edge contours of image objects.

Various techniques combining edge detection and segmentation have effectively improved the accuracy and robustness of edge detection, providing strong support for remote sensing image analysis. To tackle complex and fragmented crop field plot extractions, researchers have exploited combined semantic and edge detection methods. Xia et al. (2018) used a method that includes the U-Net and RCF models to extract hard and soft boundaries and combine them to form complete crop field plots. Author also mentioned that the effectiveness of the method may be unfavorable when confronted with data exhibiting regional heterogeneity. For large-scale single fields extraction, Waldner and Diakogiannis (2020) used a fully connected U-net backbone with conditional inference to recognize the plot range, plot boundaries, and nearest plot boundary distance, and finally synthesized the segmentation outputs of three segmentation models. when the distances between the plots’ edges are close, the model does not effectively differentiate between edges, resulting in predicted edge widths that are smaller than the fact. In order to address the issue, Long et al. (2022) proposed a new multi-task BSiNet that combines the learning of three tasks: 1) crop field recognition, 2) crop plot boundary prediction, 3) obtaining distance features, and employs a spatial grouping enhancement module for crop field extraction. Xu et al. (2023) employed a cascade multi-task network integrated with semantic and edge detection, a fixed edge refinement network featuring local connectivity, and a fusion model to extract crop field boundary information, which has greatly aided in the recovery of small plots and local spatial topology information.

In edge extraction tasks, the edge strength map obtained through semantic edge detection acts as a confidence measure. When the confidence level dips below a certain threshold, the detected edges tend to break, leading to a significant reduction in edge connectivity and overall accuracy. Compared with pixel-based segmentation, graph-based methods have less noise, higher accuracy, and better connectivity. However, overly smooth lines can result in the loss of numerous details. These two methods often counterbalance each other. Graph-based methods are available for object extraction in remote sensing imagery; they use an iterative graph-based approach to gradually add vertices and edges to the graph. In road extraction (Bastani et al. 2018; Chu et al. 2019; He et al. 2020; Tan et al. 2020; Wei et al. 2020), a common method is to output a spatial structure graph directly. Bastani et al. (2018) predicted the starting point by viewing aerial imagery and then iteratively generating a spatial structure graph. However, this iterative process only considers local information and does not capture global information. Moreover, when a starting point deviates significantly from the ground truth, the prediction of the next starting point is biased. Tan et al. (2020) improved the search method for points by making it flexible with a step-length detection technology that can accurately locate each point. A few researchers have combined the advantages of segmented-based and graph-based methods (He et al. 2020; 2020). While these methods excel in road detection, applying them directly to crop field edge detection poses a greater challenge. Unlike roads, crop field edges lack clearly defined key points, and their width varies, making them less tolerant to a fixed approach. Additionally, crop field exhibits stronger regional heterogeneity.

Recently, transformers have achieved considerable success in computer vision, prompting researchers to explore their applications in vision problems, including image classification (Chen et al. 2020; Kolesnikov et al. 2021), object detection (Carion et al. 2020; Zhu et al. 2020), semantic segmentation (Zheng et al. 2021), and image processing (Chen et al. 2021). Carion et al. (2020) proposed an end-to-end transformer-based object detection method called the DETR, which consists of a learnable object sequence and a direct set-based prediction. DETR eliminates tedious object detection pipelines such as NMS in traditional methods and directly predicts objects. Building upon DETR, some improved transformer modules with slow convergence and limited spatial resolution for processing image features have been proposed. Zhu et al.(2020) introduced a novel approach called deformable DETR, which combines the transformer’s relationship modeling ability with sparsity-based spatial sampling of deformable convolution. They developed a deformable attention module that focused on a small set of key sampling points around the reference and implemented an iterative bounding box refinement mechanism to improve detection performance, which proved to be both simple and effective. Relationfomer (Shit et al. 2022) couples object detection and relationship detection between objects from the detection transformer (DETR) and generates a spatial graph structure from the image. The method was unaffected by errors in the current iteration vertex during the generation of the next vertex. It is a clear improvement on global semantic information capture capabilities.

Inspired by the generation of graph structure, it is necessary to consider the structural nature of edges to obtain a complete representation of crop field edges. Representing the turning points in crop field edges with nodes and connecting these nodes with edges to form a complete graph is highly beneficial for vectorization. Non-closed faces are usually excluded during vectorization due to a lack of a few correct edges. While this omission might cause only a minor decrease in the evaluation metric score, it can lead to significant errors in practical applications. Directly outputting the graph from the network in graph structure tasks can avoid noise and complex postprocessing of segmentation, which may lead to errors in interpreting the segmented image. To overcome the limitations of semantic edges, in this study, we incorporated the innate advantages of graphs with semantic edges by introducing graphs into the segmentation results. We utilized the excellent connectivity of graphs guiding imperfect segmentation results and reduced the loss of crop field edges during vectorization.

Our contributions are summarized as follows:

• We designed a CNN model, Densification D-LinkNet (DDLNet), that was tailored to address the complex and dense crop field edge extraction task in high-resolution remote sensing imagery. DDLNet combines contextual semantic information efficiently and achieves feature information fusion across different scales. It also outputs the correct edge information through an edge-guiding module with higher accuracy and lower noise levels.

• We used graphs to guide the imperfect segmentation results of crop field edges in the previous step or to output correct and complete edge structures in cases where the segmentation results were incorrect. Our approach improved the connectivity of crop field edges and had advantages in the vectorization process, as sometimes a complete crop field area only requires filling in a small continuous edge structure.

2. Methodology

Our multi-level framework is divided into three stages: a) preprocessing of training data; b) training of the semantic edge (DDLNet) and spatial graph structure models (Relationformer); and c) fusion (Figure 1). In the first stage, the training data were preprocessed, and the training labels were binarized and converted into corrected binary structure points and edge sets. The second stage is divided into two parts: (1) semantic edge model training, in which we used DDLNet for training and prediction, and the output result was an edge strength map; (2) edge graph structure model training using Relationformer. For fusion, we first performed binarization and refinement based on empirical thresholds on the edge strength map, then transformed the refined edge into a map structure and compared it with the edge patch output by Relationformer, fused discontinuous edge segments, and finally vectorized the output. Sections 2.1.1, and 2.1.2 detail the two models and fusion methods used.

Figure 1. Crop field boundary extraction flowchart.

2.1. Model architecture and calibration

2.1.1. DDLNet model

DDLNet retains the core elements of D-LinkNet (Zhou et al. 2018), which used encoder-decoder structure, dilated convolution, and a pretrained encoder. It has been employed for high-resolution remote-sensing road extraction, achieving competitive performance. However, D-LinkNet only utilized long-range skip connections to fuse features of the same scale at corresponding heights, mitigating information loss introduced by downsampling. It did not consider adequately expressing information from multi-scale features, thus neglecting the intricate interplay between objects and edges in the context of high-resolution remote sensing for crop field extraction. This leaves substantial room for improvement and enhancement. Inspired by DenseNet (Huang et al. 2017) and U-Net3+ (Huang et al. 2020), we added densification full-scale skip connections, which combined all low-level edge details with decoder features to enhance performance. The structure of DDLNet is shown in Figure 2.

Figure 2. Densification D-Link Net architecture diagram. Blue matrix is a multi-channel feature map; the encoder architecture on the left is based on Resnet34; and the decoder architecture on the right uses deconvolution. Each dotted line represents a full-scale skip connection. Each of the convolutional layers uses the RELU activation function except for the output edge prediction result, which is activated by the sigmoid function.

During the down sampling process, some details are lost inevitably due to the decrease in image resolution. Although the network could capture low-level information, high-level details are lost easily. It weakened the model’s ability to handle high-level information and lacked correlation between high- and low-level information, resulting in a loss of contextual semantic information during a few down sampling processes. D-LinkNet solved the issue using long skip connections that connect the same-scale between encoder and decoder to repair the loss of context information. Furthermore, we merged DenseNet and U-Net3+ and created a new multi-scale connection module called the full-scale skip connection module. This module not only performed skip connections within the same scale but also efficiently connected features from different scales to enhance the fusion of features across scales.

We employed deep multi-scale supervision to address the challenge of limited receptive fields in convolutional neural networks, enabling distinct feature information at varying scales to play a pivotal role. This approach entails predicting the feature information for each decoder layer (D1, D2, D3, D4, and D5) and utilizing label supervision to calculate the loss for each decoder layer. This substantially enhanced the network’s ability to learn features across all scales. We performed up sampling on each scale feature, restoring it to its original size for prediction and loss calculation using 1 × 1 convolution blocks to adjust the number of feature channels. The final predicted output was the result of fusing each scale feature ${\widehat{Y}}_{\mathit{\text{final}}},$ with the predicted output being $\widehat{Y}$ and the output weight for each layer being $w_i.$ In this experiment, we selected $w_i$=0.2 for each decoder layer, and the formula for calculation was as follows: (1) $$\begin{array}{c}{\widehat{Y}}_{\mathit{\text{final}}}=\sum _{i=1}^5{w}_i{\widehat{Y}}_i\end{array}$$(1)

Only exploiting low-level features for semantic edge supervision is insufficient as low-level features from shallow layers preserve rich details while containing significant levels of noise and redundant information, leading to low efficiency and insignificant results. The use of high-level semantic information for semantic edge supervision can mitigate these issues effectively. The last layer of the DDLNet decoder, D1, encompasses both low- and full-scale high-level semantic features from other decoders (D2, D3, D4, and D5), allowing effective semantic edge detection through edge label supervision over the features of the D1 layer.

2.1.2. Edge detection based on spatial structure graph

The DETR, proposed by Carion et al. (2020), demonstrates the potential of set-based object detection using an encoder-decoder transformer architecture. It begins with a CNN backbone to extract a compact feature representation and then reshapes the spatial dimensions of the extracted features into continuous vectors. These serialized features were complemented by positional encoding and transmitted to a transformer encoder. The decoder transforms a small number of embeddings, which are learned positional encodings. Finally, each decoder output embedding is passed to a shared feedforward network to predict its class and bounding box, or no-object class.

DETR generates a fixed-size set of N predictions at each pass through the decoder, with N set to significantly exceed the number of detectable objects in the image, following the Hungarian algorithm from prior work (Stewart et al. 2016). The loss function generates optimal bipartite matching between the predicted and ground-truth objects and optimizes the loss for specific objects. This object matching method works similarly to heuristic assignment rules, and DETR only needs to find a one-to-one matching result for the set prediction.

Our work utilized deformable DETR, which features a multi-scale deformable attention module that focuses on a small subset of key sampling points around a reference point rather than on the spatial size of the feature maps. Allocating a small set of fixed key points per query mitigates the issues of convergence and feature spatial resolution, making it an effective attention mechanism for feature maps. As an end-to-end object detector, deformable DETR is efficient and has faster convergence speeds than conventional DETR, thereby opening new possibilities for our study. Given a feature map ${\mathit{f}}_I\in {\mathbb{R}}^{\mathrm{C}\times \mathrm{H}\times \mathrm{W}}$ and q query elements that encode content features and 2-d reference points ${\mathit{x}}_q$ in an index, deformable attention features were obtained using the following formula: (2) $$\begin{array}{c}D\mathit{\text{efAttn}}\left({\mathit{f}}_q,{\mathit{x}}_q,{\mathit{f}}_I\right)=\sum _{\mathrm{m}=1}^{\mathrm{M}}{\mathit{W}}_m\left[\sum _{\mathrm{k}=1}^{\mathrm{K}}{\mathit{A}}_{\mathit{\text{mqk}}}\cdot {\mathit{W}}_m^{\prime }{\mathit{f}}_I\left({\mathit{x}}_q+\mathrm{\Delta }{\mathit{x}}_{\mathit{\text{mqk}}}\right)\right]\end{array}$$(2) where m indexes the attention head, k indexes the sampled keys, and K and M are the total sampled number (K ≪ HW) and attention head number, respectively. $\mathrm{\Delta }{\mathit{x}}_{\mathit{\text{mqk}}}$ and ${\mathit{A}}_{\mathit{\text{mqk}}}\in$ [0, 1] present the sampling offset and attention weight of $k^{\mathit{\text{th}}}$ sampling point in the $m^{\mathit{\text{th}}}$ attention head, respectively. They are obtained by linear projection over the query feature ${\mathit{f}}_q.$ As ${\mathit{x}}_q+\mathrm{\Delta }{\mathit{x}}_{\mathit{\text{mqk}}}$ is fractional, bilinear interpolation (Dai et al. 2017) is used for computing ${\mathit{f}}_I\left({\mathit{x}}_q+\mathrm{\Delta }{\mathit{x}}_{\mathit{\text{mqk}}}\right).$ The computing result goes through a single layer ${\mathit{W}}_m^{\prime }$ followed by a multiplication with the attention weight ${\mathit{A}}_{\mathit{\text{mqk}}}$ and finally another single layer ${\mathit{W}}_m$ merges all the heads.

We described the Relationformer architecture and loss function briefly, as described previously (Shit et al. 2022). The Relationformer consists of four main parts: the CNN backbone, transformer, object-detection head, and relation-detection head. The CNN backbone uses Res-Net101 to extract features according to ${\mathit{f}}_I\in {\mathbb{R}}^{D_f\times \#\mathrm{\text{emb}}},$ where $D_f$ represents the spatial dimension of the feature and #emb represents the embedding dimension. The transformer uses an architecture with a deformable attention module and an encoder-decoder structure that exploits the spatial sparsity of image features to significantly accelerate the convergence of DETR training. This encoder is consistent with a deformable DETR encoder. The decoder uses N + 1 tokens as input, where N represents the first N object tokens. The object-detection head consists of two parts: a fully connected network or an MLP stack that outputs the location of each object, and a single-layer classification module for detecting objects and nonobjects. The input to the relation detection head is a pair of object tokens and a shared relationship token, which is processed by a three-layer fully connected network using formula 3 (where rln and $r$ represent relationship and [rln]-token, respectively; MLP is a three-layer fully connected network headed by layer standardization; the ordering of object token pairs ${\left\{o^i,r,o^j\right\}}_{i\ne j}$ determines the direction i→j). The structure is shown in Figure 3. (3) $$\begin{array}{c}{\stackrel{\prime }{e}}_{\mathrm{\text{rln}}}^{\mathit{\text{ij}}}={\mathrm{\text{MLP}}}_{\mathrm{\text{rln}}}\left({\left\{o^i,r,o^j\right\}}_{i\ne j}\right)\end{array}$$(3)

Figure 3. Overall architecture and details of Relationformer. Using pairs of N objects (obj-token) and an additional (N + 1) token as the interaction and query process of the relationship between objects. Relationship defined as rln-token, and objects, and relationships between objects are combined to form a shared object-relationship diagram.

2.1.3. Fusion

The fusion approach aggregates the semantic and topological information of crop field edges. Specifically, guided by a spatial structural map, it detects discontinuous segments of crop field edges within a certain range and determines their nearest nodes to connect them. The fusion process is illustrated in Figure 4 and Algorithm 1.

Figure 4. Specific implementation process for edge fusion involves converting the refined edges into a graph, and locating the shortest path between the start and end points. If the path is not found, new edges are added to the graph. Black circles refer to vertices; yellow circles refer to start and end points. Blue dot lines refer to the searching range default to 15 pixels (changing with the crop category) and the red line refers to the fixed edge.

Algorithm 1:

Fusion

Input: Testing datasets $x$ with predicted edge strength edge $x_p,$ the binarized edge $x_b,$ the refined edge $x_r,$ the crop of testing datasets $x^{\mathit{\text{ij}}},x_r^{\mathit{\text{ij}}},$ the transformed graph $G_{\mathit{\text{thin}}},$ the predicted spatial structure map $G_{\mathit{\text{pred}}},$ the search range $R,$ the stride $s$ and the threshold $t.$

Initialize:

$x_b$ = binarization($x_p,$ $t$)

$x_r$ = skeletonize($x_b$)

for $x^{\mathit{\text{ij}}},x_r^{\mathit{\text{ij}}}$ in $s,$ width of $x,$ height of $x:$

$G_{\mathit{\text{pred}}}$= relationformer_predict($x^{\mathit{\text{ij}}}$)

$G_{\mathit{\text{thin}}}$= trans_to_graph($x_r^{\mathit{\text{ij}}}$)

for edge in $G_{\mathit{\text{pred}}}:$

$\hspace{0.5em}\hspace{0.5em}\hspace{0.5em}\hspace{0.5em}{\mathrm{n}}_1,$ ${\mathrm{n}}_2$= edge.nodes

$\hspace{0.5em}\hspace{0.5em}\hspace{0.5em}\hspace{0.5em}{m}_1,$ $m_2$= if closet_nodes(${\mathrm{n}}_1,$ ${\mathrm{n}}_2,$ $G_{\mathit{\text{thin}}},$ $R$)

            else build_nodes(${\mathrm{n}}_1,$ ${\mathrm{n}}_2,$ $G_{\mathit{\text{thin}}},$ $R$)

     if not has_path($m_1,$ $m_2,G_{\mathit{\text{thin}}}$)

        connect($m_1,$ $m_2,G_{\mathit{\text{thin}}}$)

$x_{\mathit{\text{output}}}$= trans_to_edge($G_{\mathit{\text{thin}}}$)

return $x_{\mathit{\text{output}}}$

The first step in this process is to refine the crop field edges. Initially, the edge strength map with a size of 1024 × 1024 is converted into a binary image by an empirically set threshold of 50. Subsequently, we used the skeleton extraction algorithm (Zhang and Suen 1984) to extract a refined binary image of the crop-field edge as the basis for further repair.

The second step involves generating spatial structural maps using sliding windows. The high-resolution crop field image and predicted image were 1024 × 1024 pixels and 128 × 128 pixels, respectively, with the stride for rows and columns set to 32 pixels (balanced between precision and speed). Subsequently, 32 images of 128 × 128 crop field plots were input into Relationformer to predict the spatial structural map, which was used as a guide for edge repair.

Third, we check the connectivity of the refined edges. The refined agricultural edge results are transformed into a list of points and edges, denoted as $G_{\mathit{\text{thin}}},$ and the spatial structure map is denoted as $G_{\mathit{\text{pred}}}.$ Following this, we traversed each edge in $G_{\mathit{\text{pred}}}$ and searched for two points in $G_{\mathit{\text{thin}}}$ that were closest to the starting and ending points of the edge, thereby creating new starting and ending points. We determined the connectivity between these new points by finding whether there was a shortest path between them. The edge is considered connected if the shortest path exists; otherwise, it is not.

The final step is to connect the disjointed segments. In the spatial structure map, when the starting and ending points were located in different connected components, we searched for the two nearest nodes in $G_{\mathit{\text{thin}}}$ that belonged to different connected components and added a bridge between them. It connects the starting and ending points to form a unique path, thereby completing the entire merging process.

2.1.4. Loss function

The loss function is composed of two parts: the class-balanced mean squared error (CMSE) loss for semantic edge detection, and the L1 loss and GIOU loss for object detection to detect the true bounding boxes of predictions. Additionally, we used the Cross-entropy Classification Loss (CLS) to detect the predicted and actual categories.

Before we explain our loss function, it is necessary to introduce a widely used loss in typical natural image detection, HED, a class-balanced cross-entropy (CBCE) loss, which used a class-balancing weight $\beta$ on a per-pixel term basis for overcoming the heavy biases between edges (10%) and nonedges (90%). The CBCE loss is calculated by the following formula: (4) $$\begin{array}{c}{\mathit{\text{Loss}}}_{\mathit{\text{cbce}}}=-\beta -\sum \mathrm{\text{log}}\left({\widehat{Y}}_j\in \left|Y-\right|\right)-\left(1-\beta \right)\sum \mathrm{\text{log}}\left({\widehat{Y}}_j\in \left|Y+\right|\right)\end{array}$$(4) where $\beta =|Y+|/\left|Y\right|$ and $1-\beta$=$|Y-|/\left|Y\right|.$ $|Y+|$ and $|Y-|$ denote the edge and non-edge ground truth label, respectively. The edge map predictions ${\widehat{Y}}_j$ (j∈[0,1,…H×W]) are computed using sigmoid function.

Though HED loss is applied in most edge detection networks like RCF and BDCN, blurring and roughness outputs are recurring due to significant noise interference. Hence, we proposed the use of CMSE loss, which incorporates class balance parameters into the foundation of MSE loss, to generate relatively refined edge intensity maps. The CMSE loss represents the sum of the squared absolute differences between the predicted and target values for each pixel. The formula for CMSE loss is as follows: (5) $$\begin{array}{c}{\mathit{\text{Loss}}}_{\mathit{\text{cmse}}}=\frac{1}{m}\sum (\beta {\left(\right({\widehat{Y}}_j\in |Y-|)-Y_j)}^2+\left(1‐\mathrm{\beta }\right)\left({{\widehat{Y}}_j\in |Y+|)-Y_j}^2\right)\end{array}$$(5) where m = H×W, $\beta =|Y+|∕\left|Y\right|.$

We describe two objects as having a valid relationship when they share a connection. Otherwise, the relationship is classified as background. However, these relationships are sparse within all possible permutations and object combinations. Consequently, computing the relationships among all possible object pairs can become exceedingly complex. To mitigate this problem, we randomly sampled three background relationships from the valid relationships, yielding a set R of size M, on which we computed Relationship Proposal Network loss for relationship detection. The overall loss includes ${\ell }_1$ regression loss (${\mathcal{L}}_{\mathit{\text{reg}}}$) and the generalized intersection over union loss (${\mathcal{L}}_{\mathit{gIoU}}$) between the predicted ${\tilde{\mathit{v}}}_{\text{box}}$ and the ground truth ${\mathit{v}}_{\text{box}}$ box coordinates for object detection, the cross-entropy classification loss (${\mathcal{L}}_{\mathit{cls}}$) between the predicted class ${\tilde{v}}_{\mathit{\text{cls}}}$ and the ground truth class $v_{\mathit{\text{cls}}}$ for category prediction, and relationship loss (${\mathcal{L}}_{\mathit{rln}}$) between the predicted relation ${\tilde{e}}_{\mathit{\text{rln}}}$ ($i\to j$) and the ground truth relation $e_{\mathit{\text{rln}}}(i\to j)$ for relationship detection. The simultaneous object-relationship graph generation loss function is defined as follows: (6) $$\begin{array}{c}\begin{array}{cc}{L}_{\mathit{\text{total}}\mathrm{ }}& ={\sum }_{i=1}^N\left[1_{v_{\mathit{\text{cls}}}^i\notin \mathrm{\varnothing }}\left({\lambda }_{\mathit{\text{reg}}}{\mathcal{L}}_{\mathit{\text{reg}}}\left(v_{\mathit{\text{box}}}^i,{\tilde{v}}_{\mathit{\text{box}}}^i\right)+{\lambda }_{\mathit{\text{gIoU}}}{\mathcal{L}}_{\mathit{\text{gIoU}}}\left(v_{\mathit{\text{box}}}^i,{\tilde{v}}_{\mathit{\text{box}}}^i\right)\right)\right]\\ & +{\lambda }_{\mathit{\text{cls}}}{\sum }_{i=1}^N{\mathcal{L}}_{\mathit{\text{cls}}}\left(v_{\mathit{\text{cls}}}^i,{\tilde{v}}_{\mathit{\text{cls}}}^i\right)+{\lambda }_{\mathit{\text{rln}}}{\sum }_{\{i,j\}\in \mathcal{R}}{\mathcal{L}}_{\mathit{\text{rln}}}\left(e_{\mathit{\text{rln}}}^{\mathit{\text{ij}}},{\tilde{e}}_{\mathit{\text{rln}}}^{\mathit{\text{ij}}}\right)\end{array}\end{array}$$(6) where ${\lambda }_{\mathit{\text{reg}}},$ ${\lambda }_{\mathit{\text{gIoU}}},$ ${\lambda }_{\mathit{\text{cls}}}$ and ${\lambda }_{\mathit{\text{rln}}}$ are the loss functions specific weights.

2.2. Experiments

2.2.1. Datasets

The first dataset came from drones surveying tobacco planting regions in Chenzhou City, Hunan Province, covering 24 different villages. The second dataset featured high-resolution remote sensing images of crop fields from Google Earth in Hangzhou, Huzhou, Jiaxing, Ningbo, Shaoxing, and Zhoushan, all located in Zhejiang Province (Figure 5). The tobacco-planting region dataset comprised 371 high-resolution images, each having a resolution of 1024 × 1024 pixels with a 1 m spatial resolution. The crop field dataset contains 150 high-resolution images, each having a resolution of 1024 × 1024 pixels and a spatial resolution of 0.955 m (Table 1).

Figure 5. Datasets. Chehzhou dataset covers 24 villages and Zhejiang dataset includes 6 cities (Hangzhou, Huzhou, Ningbo, Jiaxing, Zhoushan, Shaoxing, comprising 55 clips). Image provides a representative sample of the dataset’s appearance.

Table 1. Datasets. IS and SR refer to image source and spatial resolution, respectively.

Dataset	IS	SR	Dates	Area
Chenzhou	Dajiang	1m	2020/02	19387${\mathit{\text{km}}}^2$
Zhejiang	Google Earth	0.955m	2022/09	64623${\mathit{\text{km}}}^2$

Ground-truth crop images were obtained through manual delineation in the study area. To facilitate model training, we cropped the images to 512 × 512 pixels for DDLNet edge detection, whereas the labels were cropped to 512 × 512 pixels. For the former training data, we cropped the images into 128 × 128 pixel channels and converted the labels into binary spatial structure datasets consisting of point and edge sets (Figure 6).

Figure 6. Ground truth of two study regions. They are all manually drawn images from our laboratory.

2.2.2. Training and implementation details

Two separate models were built using the PyTorch framework and trained on a single NVIDIA 3090 GPU with 24 GB of memory. The DDLNet model used an initial learning rate of 2×${10}^{-4},$ which was updated every 100 iterations. The models were trained for 400 epochs on both the Hunan and Zhejiang datasets, with 80% of the data reserved for training and 20% reserved for testing. The Relationformer model used a lower initial learning rate of 2×${10}^{-5},$ and was trained for 100 epochs on both datasets.

2.2.3. Evaluation metrics

To evaluate the accuracy of crop field edge detection, we employed the Optimal Dataset Scale (ODS) and Optimal Image Scale (OIS) proposed by HED (Xie and Tu 2015). ODS and OIS employ a standard non-maximum suppression technique to obtain finer edges. We used a uniform threshold value for all images and selected the average F1 score (harmonic mean of precision and recall) of the results as ODS. Additionally, the optimal threshold was selected for each image to maximize the F1 score, and the average F1 score was used as the OIS.

Further, we validated the accuracy of our postprocessing methods for crop field surface extraction using joint intersection over union (IoU). IoU stipulates the degree of overlap between the predicted and actual regions of an image, with higher values indicating more accurate predictions. When calculating the polygon IoU, each crop field plot served as a label to reference, and we selected the plots with the largest intersecting area as the correct prediction, accumulating the IoU of each plot using the formula (7) $$\begin{array}{c}\mathit{\text{IoU}}=\frac{A\cap B}{A\cup B}\end{array}$$(7) where A presents the predicted area while B represents the ground-truth.

However, IoU only reflects the accuracy of crop field plot area extraction and cannot reflect the accuracy of crop field edge and internal edge extraction. Therefore, we employed the boundary IoU to evaluate the accuracy of crop field edge detection. The crop field edges were dilated by setting the kernel size to 5 and using the dilation function (Exp) with the kernel size (ks). The calculation formula is as follows: (8) $$\begin{array}{c}\mathit{\text{Boundary IoU}}=\frac{\mathrm{\text{Exp}}\left(\mathrm{A},\mathrm{\text{ks}}\right)\cap \mathrm{\text{Exp}}\left(\mathrm{B},\mathrm{\text{ks}}\right)}{\mathrm{\text{Exp}}\left(\mathrm{A},\mathrm{\text{ks}}\right)\cup \mathrm{\text{Exp}}\left(\mathrm{B},\mathrm{\text{ks}}\right)}\end{array}$$(8)

3. Results

3.1. Edge

We compared several of the most outstanding edge-detection methods in recent years, including BDCN (He et al. 2019), DexiNed (Poma, Riba, and Sappa 2020), PiDiNet (Su et al. 2021), and EDTER (Pu et al. 2022). All methods were trained with the authors proposed parameters to achieve optimal prediction results. As shown in Table 2, BDCN achieved the highest scores among all the methods in ODS (67.38%) and OIS (67.43%) for the dataset in Chenzhou City, Hunan Province. Our DDLNet method obtained second-place scores in ODS (67.38%) and OIS (67.43%), with a marginal difference of 0.48% from the top-performing method for both ODS and OIS and 9.8% and 6.64% from the third-place PiDiNet.

Table 2. Accuracy comparison of different edge detections for crop fields in Chenzhou city.

Dataset	Methods	OIS	ODS
Chenzhou	BDCN	67.86	67.91
	DexiNed	20.07	20.26
	PiDiNet	57.58	60.79
	EDTER	37.84	46.08
	DDLNet(Ours)	67.38	67.43

Both BDCN and DDLNet exhibited better extraction of edges that were unaffected by non-crop fields, resulting in closer OIS and ODS scores. However, the edge strength map produced by BDCN had a larger pixel width, which made it difficult to clearly distinguish between the two edges. In contrast, our method utilizes a smaller pixel width, which better distinguishes between the edges, handling this case more effectively. The scores for PiDiNet (OIS: 57.58%, ODS: 60.79%) were weaker than those for our method because PiDiNet generated more noise and did not distinguish clearly between soil ridges and field edges. EDTER (OIS:37.84%, ODS:46.08%) exhibited a lower extraction ability and a large amount of noise at the edges. After repeatedly adjusting the parameters and experimenting, we found that the edge extraction ability of DexiNed was insufficient, and its performance on the Chenzhou data was not acceptable. Figure 7 presents a comparison of the different methods.

Figure 7. Comparison of different methods used on the test dataset in Chenzhou city.

Table 3 shows that EDTER achieved the best OIS (70.14%) and ODS (68.88%) scores out of all methods in the cultivated land dataset in Northern Zhejiang. In contrast, DDLNet attained inferior OIS (66.52%) and ODS (66.51%) scores, which were 3.62% and 2.37% lower than EDTER, respectively.

Table 3. Accuracy comparison of different edge detection methods for crop fields in Northern Zhejiang.

Dataset	Methods	OIS	ODS
Zhejiang	BDCN	61.25	61.18
	DexiNed	53.49	53.44
	PiDiNet	67.27	66.26
	EDTER	70.14	68.88
	DDLNet(Ours)	66.52	66.51

EDTER stood out in crop field extraction tasks, accurately extracting crop field edges despite its tendency to produce non-cropland and soil ridges. PiDiNet and BDCN are other methods that can extract edges but suffered from high noise, broken edges, and incorrectly extracted edges. DexiNed extracted a significant number of unnecessary internal edges, which decreased the OIS and ODS scores significantly. DDLNet produced the visually cleanest edges with low noise but suffer from limited feature extraction ability in some images and exhibited more pixel points beneath the fixed threshold (selected by OIS and ODS in the calculation process) than EDTER, resulting in lower OIS and ODS scores. Figure 8 compares the edges from different methods.

Figure 8. Comparison of different methods on the Zhejiang test dataset in crop fields.

Edge-detection methods produce non-binary edge-strength maps. Therefore, ODS and OIS select appropriate thresholds for the binarization and computation of the optimal threshold. In essence, ODS sets a uniform threshold for all images, maximizing the F1 score across the entire dataset, while OIS chooses distinct thresholds for individual images, optimizing their F1 score. However, these methods have limitations. They only consider the edge detection capability and do not reflect the performance of crop field extraction or edge-breaking situations. Selecting thresholds that are too high or too low can also result in false or missed detections, affecting the accuracy of performance metrics. These factors are crucial for practical applications because they can affect the actual performance of the model. Therefore, appropriate postprocessing is necessary to demonstrate the capability of the model in practical applications.

3.2. Post-processing

Vectorization is the process of converting the edge information in an image into a vector form. In this process, the pixels in the image are transformed into a series of vectors or curves with geometrical characteristics such as coordinates, length, and width. It is commonly used in map-making, image recognition, and other fields. Accurate edge strength maps are crucial for improving the accuracy and efficiency of vectorization. As shown in Table 4, the vectorization achieved with the DDLNet&Fusion method obtained the best scores for boundary IOU (32.01%) and polygon IOU (61.24%) in comparison to other methods in the dataset in Chenzhou City, demonstrating excellent performance in vectorization tasks. Compared to other methods, DDLNet&Fusion can extract accurate edge information and convert it into a high-quality vector form, thereby improving the accuracy and efficiency of vectorization.

Table 4. Accuracy comparison of post-treatment methods for crop fields in Chenzhou City.

Dataset	Methods	Boundary IOU	Polygon IOU
Chenzhou	BDCN&Fusion	26.00	56.47
	DexiNed&Fusion	2.89	16.04
	PiDiNet&Fusion	20.90	59.66
	EDTER&Fusion	10.69	44.13
	DDLNet&Fusion	32.01	61.24

The completeness of edges had the greatest influence on the boundary and polygon IOU scores. Despite post-processing, DexiNed could only detect a limited number of edges, resulting in low boundary IOU (2.89%) and polygon IOU (16.04%) scores. Furthermore, the holes generated during the binarization process significantly affected the line IOU scores. EDTER is influenced by the edge strength map, which generates many holes during the binarization process, leading to low boundary IOU scores (10.69%). However, polygon IOU (44.13%) was not affected by the accuracy of edges, and our postprocessing successfully extracted the most cultivated plots, resulting in higher scores. Moreover, because of the pixel widths of edges, two adjacent parallel lines are sometimes processed as a single line. The DDLNet edge intensity map had the narrowest pixel width among all models, making it more effective than BDCN in dealing with the issue of two parallel lines. Consequently, DDLNet achieved a higher boundary IOU score (32.01%) than the BDCN boundary IOU (26.00%). Figure 9 shows a post-processing comparison of different methods.

Figure 9. Comparison of the fusion results of different methods on the crop field dataset in Chenzhou City.

Table 5 shows that using DDLNet&Fusion achieved the best boundary IOU (47.33%) and polygon IOU (64.88%) among all methods for vectorization of the Zhejiang crop fields. Our method outperformed the second-place EDTER for Boundary IOU and Polygon IOU scores by 14.54% and 3.59%, respectively.

Table 5. Accuracy of post-processing methods for edge detection in Zhejiang crop field.

Dataset	Methods	Boundary IOU	Polygon IOU
Zhejiang	BDCN&Fusion	27.54	52.35
	DexiNed&Fusion	30.41	50.28
	PiDiNet&Fusion	32.12	58.83
	EDTER&Fusion	32.79	61.29
	DDLNet&Fusion	47.33	64.88

For dense crop field extraction, our post-processing method performed well in various edge detection models. It overcame the limitations stemming from low edge intensity, which leads to binarization loss. In addition, it yields edges that precisely correspond to actual crop field situations. The widths of edge strength maps generated by BDCN and DexiNed are greater than those generated by our method. Moreover, the edge integrity in densely cropped field regions was suboptimal in both the boundary IOU and polygon IOU scores when compared with our DDLNet. Noise has a significant effect on boundary extraction, and reduced noise is more beneficial for the complete extraction of crop fields. Compared to DDLNet, the polygon IOU scores of PiDiNet and EDTER were marginally lower, having high edge integrity. However, the output edge strength map contained unclosed edge pixels that disrupted the fusion process, resulting in a boundary accuracy gap of 15.21% (PiDiNet) and 14.54% (EDTER). The fusion outcomes of different techniques are presented in Figure 10.

Figure 10. Comparison of the fusion results of different methods in Zhejiang dataset.

3.3. Ablation experiment

To further demonstrate the enhanced performance of our method in semantic edge extraction through the fusion of spatial structure maps, we conducted a comparison with an eight-neighbor tracking algorithm. The underlying principle of the eight-neighbor tracking algorithm involves scanning an image to identify non-zero-pixel points. Starting from each non-zero point, we computed the pixel values in the binarized image and the edge-strength image for the eight surrounding pixels. If the pixel value in one direction of the binarized image exceeded zero, we examined the edge strength pixel values of the three opposite-direction pixels. The pixel with the highest value among the three was selected as the next starting point, and this process was iterated. In our post-processing method, the edge probability map was first binarized using a default threshold of 50. The threshold value was determined based on empirical experience, aiming to strike a balance between broken lines and the desired edge width. Subsequently, the skeleton of the binarized image was extracted, followed by vectorization.

As shown in Table 6, using fusion reduces the broken-line problems during the vectorization of crop fields effectively, achieving the best scores and closest visual effects to reality among all methods. In the tobacco dataset, the fused boundary IOU (32.01%) and polygon IOU (61.24%) achieved the highest scores. In the Zhejiang dataset, the fusion method obtained the highest scores for the boundary IOU (47.33%) and polygon IOU (64.88%).

Table 6. Accuracy comparison of different post-processing methods.

Dataset	Methods (Threshold = 50)		Boundary IOU	Polygon IOU
	Fusion	Trace
Chenzhou	ⅹ	ⅹ	30.10	54.27
	ⅹ	√	31.46	58.87
	√	ⅹ	32.01	61.24
Zhejiang	ⅹ	ⅹ	46.19	56.89
	ⅹ	√	46.46	59.95
	√	ⅹ	47.33	64.88

As shown in Figure 11, when Fusion or Trace is not employed, the default post-processing method easily produces broken lines during binarization. As we discarded broken lines that could not form an area, the resulting boundary and polygon IOU scores were lower than those of Fusion and Trace. Trace relies on the edge-strength image, which means that tracking cannot continue when the edge probability is zero. Moreover, edges with insufficient probability can affect the tracking performance because edges detected with low probabilities may not be accurate or may be composed of multiple cluttered lines. Therefore, the boundary IOU and polygon IOU scores were higher than those of the default methods but lower than those of fusion. Our method is guided by the correct edge spatial structure map, which decreases the probability of incorrect connections and results in the highest scores.

Figure 11. Comparison of the effects of different post-processing methods on the Zhejiang dataset.

4. Discussion

Convolutional neural networks utilize multi-scale feature fusion to retain more details, but this can lead to limitations in edge refinement capabilities and result in blurred and disconnected edges. The experimental results demonstrated that our method outperformed other semantic edge models in crop field extraction, though the scores of OID and ODS were not the best. There were two key reasons why we proposed our model. We enhanced the fusion of different scale features and designed the CMSE loss function, which effectively reduced the occurrence of broken and blurred edges. Compared with the most advanced models (BDCN, EDTER, PiDiNet, DexiNed), they all concentrated on edges from natural images and worked efficiently, but they ignored the connectivity of edges and the degree of thickness, which are essential in crop field extraction. For instance, BDCN used 5 blocks, each followed by a pooling layer to progressively enlarge the receptive in the next blocks, with the Scale Enhancement Module (SEM) to achieve a better trade-off between efficiency and accuracy. The results showed that large feature scales were easy to learn, but SEM did not appear to work well in extracting the small crop field plot; it was either blurred or too thick. We also tested the transformer’s ability to extract crop field edges with EDTER. Different from the CNNs, EDTER was designed to capture the long-range global context of the whole image and extract the short-range local cues for fine-grained edges in two stages. It was clear that EDTER compensated for the loss of the CNNs in learning long-range global context, and the experiments showed excellent performance with the highest OIS and ODS scores in the Zhejiang dataset. However, the results in the Chenzhou dataset did not reveal the same enhanced performance. Ongoing improvements to transformers in edge detection are still necessary. In conclusion, it is essential to highlight that despite the effectiveness of our edge detection model, disconnected edges persisted in the results of binary classification, particularly when handling blurred images or images with unclosed edges. This challenge remains a significant concern for semantic edge models and has yet to be fully addressed.

The diversity of the crop field sample had two crucial impacts on our experimental results. Firstly, it posed challenges in detecting and distinguishing different edge features during model training. Secondly, selecting the right threshold value during postprocessing was crucial. In Chenzhou, Hunan Province, where crop field plots had distinct edges but were similar to non-crop fields, setting a high threshold led to more disconnected boundaries. In high-density crop fields in Northern Zhejiang, where plots were closely situated and exhibited diverse shapes, opting for a low threshold resulted in thicker edges and overlapping boundaries. Striking the right balance in threshold selection was vital to accurately capturing boundaries in various crop field samples.

In practical applications, given that the edge of a crop field cannot be used due to broken lines, we removed the edges of crop fields that could not be polygonized during postprocessing. Removing messy lines can optimize the crop field extraction results; however, given that sometimes only a small part of a whole piece of crop field is missing and the entire edge is removed, it also impairs the accuracy of the post-processing edge compared to that of the semantic edge. Additionally, the eight-neighbor tracking algorithm is limited by the edge intensity map. If the edge intensity map is blurred, jagged edges may be generated and the edge transition is not sufficiently smooth; if the edge probability is low, tracking cannot be continued and edge disconnection cannot be avoided.

To enhance post-processing extraction, we introduced Relationformer, an effective and highly connected graph structure detection method whose robust connectivity solved the issue of broken lines after semantic edge thresholding. In long-distance edge detection, using clear line segments proved superior to binarization and eight-neighborhood tracking, although not without challenges. The accuracy of output nodes significantly influenced extraction outcomes, especially in areas with subtle changes, leading to potential positional shifts. Dense edges, irregular shapes, and short-distance segments posed difficulties in reflecting edge structure changes. Moreover, combining two edge detection methods sometimes resulted in misaligned edges, mainly due to neglecting edge structure diagram directionality. This approach led to significant errors in certain cases compared to ground-truth data.

It is certain that the setting of hyperparameters also have an impact on the experimental results. For example, the prediction results of DDLNet typically exhibited the cleanest edges when the number of training epochs was set to 400. Reducing training epochs resulted in more noise and thicker edges, and if the epochs exceeded 400, the generated edges were often incomplete. Opting for fewer training epochs can help retain more features, but it complicates the evaluation of negative samples during post-processing, and our post-processing algorithm needed further improvement. We opted for HIDDEN_DIM = 256 and DIM_FEEDFORWARD = 128 to accelerate training and prevent model overfitting when training relationshipformer. The size of OBJ_TOKEN can impact both the accuracy of the prediction results and the training speed across different datasets. After conducting multiple experimental comparisons, we set OBJ_TOKEN = 250 for the Chenzhou dataset to achieve better predictions. Given the substantial large categories and complex terrain variations in the Zhejiang dataset, we set OBJ_TOKEN = 600 to enhance prediction accuracy. However, it was important to note that the decision comed with a trade-off, as it reduces the model training speed and increases GPU memory usage. The specific choice should be based on the complexity of your training data.

The extraction of complete tillage edges is a complex task. The current approach focuses on solving the connectivity problem of tillage edge extraction using semantic edges and graphs. However, this approach has limitations. In subsequent studies, we plan to incorporate the direction of graph structures into our approach and consider the similarity of polygons to avoid outcomes that do not match the visual effects.

5. Conclusion

This paper introduced a multi-level framework that utilizes a combination of CNN and transformers for extracting crop fields from aerial images. It consists of two stages, each targeting specific challenges. In the first stage, we address the issue of discontinuity in detecting semantic crop field edges by employing a novel CNN model called Densification D-LinkNet. This model incorporates full-scale skip connections, deep multi-scale supervision, and semantic edge supervision, thereby enhancing edge-detection capabilities. In the second stage, we leverage both semantic and topological information about the edges. The topological information is utilized to correct any discontinuous edges during the vectorization process, ultimately producing complete and usable crop field vector plots.

Our method outperformed the previous approaches on two different experimental datasets by addressing the issue of edges that cannot be counted as partial missing, which resulted from inaccurate detection of semantic edges. Additionally, we introduced topological information into the edge extraction process for the first time, resulting in advanced edge extraction effectiveness.

Acknowledgments

Our experiment was based on two research datasets, both of which are proprietary to our laboratory. All authors express their gratitude to the reviewers and editors for their helpful comments and suggestions.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The dataset used in the article can be downloaded at https://pan.baidu.com/s/1JWivVLK2RVURm1MgGcQOyg?pwd=dgt8 –dgt8 and https://drive.google.com/drive/folders/1xHla26EzwOC_KiS2FkHcLkJSd7sQbhnk?usp=sharing; Some network-related codes are publicly available at https://github.com/649064287/DDLNet-main.git.

Additional information

Funding

This work was supported in part by the National Key Research and Development Program of China under Grant 2018YFB0505300 and in part by the National Natural Science Foundation of China under Grant 41701472 and 41971375.