Predicting adolescent violence in Wartegg-ZeichenTest drawing images based on deep learning

Abstract

This thesis deals with the problem of negative behaviour due to changes in mental and physical stress in adolescence. In particular, it is a study to solve the health care problem of students exposed to violence. Among the problematic behaviours, students exposed to violence, especially, have many problems with healthcare. A projective test using pictures can elicit information from adolescents through direct experiences represented by pictures to which the subject unconsciously reacts. Few methods analyse images drawn by adolescents as image data. This study analyses data from 134 violent students who received fifth-degree punishment for violent behaviour and 134 nonviolent students. We use the convolutional neural network (CNN)(softmax), CNN (support vector machine (SVM)), with the style transfer generative adversarial network, and ensemble techniques to analyse drawn images using WZT and predict violence through deep learning. We predict violence from pictures with an accuracy of 93%–98%. This study is the first to automatically analyze and predict violence with a deep learning model in images drawn by adolescents on WZT. It also features WZT to proactively conduct violence investigations to improve health care for students. Advances in deep learning for image feature extraction are expected to provide more research opportunities.

KEYWORDS:

• Wartegg–ZeichenTest
• deep learning-based violence prediction
• convolutional neural network (CNN) (softmax)
• CNN (SVM)
• healthcare

SUBJECT CLASSIFICATION CODES:

• Artificial intelligence
• deep learning
• healthcare

Introduction

The changes in stress experienced during adolescence can lead to negative behaviours, such as maladaptation and depression, in school and daily life. If not appropriately managed, stress can result in problem behaviours, such as mental, physical, psychological, and social deviance (Wilde & Kirnhorst, 1998). Violence is a social stressor arising from competition (Basile et al., 2020). If excessive stress experienced by adolescents is not managed effectively, it can negatively affect health, academic achievement, and they may exhibit more problem behaviours or distortions, such as delinquency and violence (Herrenkohl et al., 2000; Piquero & Sealock, 2004).

According to data from the first survey on school violence conducted by the Ministry of Education for elementary, middle, and high school students in 2022, language violence (41.8%), physical violence (14.6%), group bullying (13.3%), and cyber violence (9.6%) were reported in that order. Over the past ten years, from 2012 to 2022, the proportion of physical violence has increased (Ministry of Education, 2021). The cause of the increase in violence is that teenagers violently express their anxiety and restlessness because they do not know how to deal with it. However, there is a drawback to using self-reported questionnaires to understand the inner world of teenagers because obtaining accurate information due to their defensive or untruthful responses is difficult (Briggs Myers et al., 2013).

A projection test is selected to predict teenage violence using pictures to address these drawbacks and prevent youth violence. The projection test represents direct experiences that the subject reacts to unconsciously through pictures. Projection tests have been developed in various ways depending on the subject (Mattlar et al., 1991).

The Wartegg-ZeichenTest is selected to predict youth violence using projection methods. Specific patterns emerge in the pictures represented in the WZT depending on the tendencies of the teenager (Reardon MacLellan, 2011). The WZT can be applied to various subjects, from children to adults, and presents eight pictures drawn in a 4 × 4 cm square box with a black border. The WZT includes play elements and is a well-known projection method that can reduce stress for teenagers taking the test and minimise their awareness of the testing process (Myers et al., 1998). This study analyzes images directly drawn by teenagers in the WZT to predict violence.

This study used deep learning and convolutional neural network (CNN) models to predict violence in the images adolescents drew in the WZT. The CNN extracts visual features from digital images (Bibi et al., 2021; Ning et al., 2020). Using the CNN(softmax), CNN (SVM), style transfer generative adversarial network (GAN), and ensemble techniques improved the prediction of violence. This study is characterised by conducting an investigation into violence in advance with WZT so that students’ healthcare can be provided. And discovering the unique characteristics of individuals in the pictures students have created offers many opportunities to predict violence using deep learning.

Paper preparation

During adolescence, when many physical and psychological changes occur, stress can significantly affect individuals. Excessive pressure on an adolescent can manifest in distorted forms if not adequately managed, resulting in maladaptive behaviours in daily life and school and negative health outcomes, such as depression. Teenagers who cannot cope with excessive mental stress or stressful situations may exhibit problem behaviours (Wilde & Kirnhorst, 1998), such as violence.

Addressing adolescent violence has long been emphasised in the context of schools. Examining the factors influencing adolescent violence includes many variables (Lösel & Farrington, 2012). The concept of violence and its normalisation starts at a young age, and exposure to adolescent violence becomes dangerous for future victims and perpetrators of violence (Ezell et al., 2022). Continuous exposure to violence can have adverse developmental and adaptive outcomes and harmful effects in the long term (Pedras et al., 2021). Furthermore, violence can worsen adolescent academic achievement, and those more exposed to disadvantages are at a greater risk of not meeting academic standards (Pinchak & Swisher, 2022).

In a study of 717 adolescents in Nigeria, the violent incidence was 87%. Domestic violence accounted for 44.2% of the cases, whereas violence by relatives accounted for 49.2%. Half of the incidents of violence occurred within six months (Ughasoro et al., 2022).

Teachers can be crucial as excellent observers and potential interveners in school violence (Ezell et al., 2022). With sufficient information about a student’s potential for violence, teachers can tailor approaches to the student, providing emotional support and counselling before or immediately after the violence, healing from violence, and preparation for future incidents (Kim et al., 2021). This study aims to predict adolescent violence through deep learning by analysing WZT images drawn directly by teenagers.

Research method

The research subjects expressed parental consent and willingness to participate, and data on 134 youths who were subject to the 5th violent act were collected as violence data from September 2020 to December 2022. There were 134 adolescents without violence, and data on non-violence were collected from April 2012 to March 2013. We used deep learning to extract violence-related features from WZT images drawn by teenagers. The focus was on accurately identifying violent features in the pictures drawn by teenagers. We applied deep learning to digital images to extract complex visual features, and CNN models perform well in improving the classification of violent features in visual tasks (Song et al., 2020; Szegedy et al., 2017). To predict violence from WZT drawings created by teenagers, we employed various methods, such as the CNN (softmax), CNN (SVM), style transfer GAN, and ensembles (Park et al., 2022). Classification performance may vary depending on the trained drawing images, so we used the style transfer GAN to enhance drawing images while maintaining the image features to prevent performance degradation (Lee et al., 2020).

This study employs image data from 134 violent and 134 nonviolent adolescents in WZT drawing images, which produced 1,904 drawing images, with eight drawing images per person. If five of these drawing images in a WZT image drawn by a person are predicted to indicate violence, the person is determined to be violent.

The testing data consisted of 27 violent and 27 nonviolent data. In addition, 75 people were used in the training data, and the remaining 32 were used in the validation data. The Google TensorFlow deep learning framework was used, and the experiment was conducted on a MacBook M1 chip graphics processing unit. A binary classification prediction model was created using the softmax activation function and the sparse categorical cross-entropy loss function in the CNN, and the SVM loss function was also applied to check the performance change. Additionally, the style transfer GAN was used to synthesise margins in the WZT drawing images, and the SVM binary classification model was trained using the synthesised data to determine its influence on performance. Finally, the three prediction models were ensembled, and the difference in performance from the original prediction model was confirmed.

Wartegg-ZeichenTest

The WZT is a projective test developed by Ehrig Wartegg in 1939 and has been used in various fields, such as psychological counselling, therapy, educational counselling, and criminal justice. The WZT consists of eight 4 × 4 cm stimulus drawings with a black border, and various types of pictures are used in the test (Soilevuo Grønnerød & Grønnerød, 2012). Each of the eight stimulus drawings has a different meaning and interpretation, allowing for an understanding of the nature of the drawings Figure 1.

Figure 1. Wartegg-ZeichenTest.

Stimulus drawing images 1, 2, 7, and 8 of WZT are curves, making it easy to represent living things. Stimulus drawing images 3, 4, 5, and 6 of WZT are straight lines primarily representing concrete objects. The first stimulus image is a central point, indicating the centre and self-experience. The second stimulus image is unique, with “movement” that floats in the air. This image is placed in the upper left corner of the space to stimulate the imagination and emphasise the movement characteristic. The third stimulus image means “ascension” and starts low on the left, rising higher to the right, which indicates strong goal orientation. The fourth stimulus image is a small square with a feeling of “heaviness” that effectively inspires counselling sessions. It makes one feel mental stress and difficulty. The fifth stimulus image has two lines that explain the mind, and this contrastive form represents tension, antagonism, and aggressiveness, implying the overcoming of tension by the extension of the line. When in an introverted psychological state, the impulse is suppressed by space with nothing drawn in the upper right corner. This paper concludes that violent drawing images of adolescents appear in the expression of tension and impulse suppression. The sixth stimulus image has horizontal and vertical lines that unconsciously demand their combination. The seventh stimulus image with a dotted semicircle drawn in the lower right corner represents delicacy. It appeals to sensitivity. Last, the arch-shaped curve in the eighth stimulus image represents safety (Avé-Lallemant, 2000). The characteristics and meaning of the theme of each stimulus picture are shown in Table 1.

Table 1. Characteristics and meaning of the subject of each stimulus picture.

Stimulus picture No	Characteristics	Meaning
1st stimulus picture	Centre point	Centre, self-experience
2nd stimulus picture	Soft curve at the top left of the space	Stimulation of emotions and imagination
3rd stimulus picture	Three bars that become higher	Elevating rise, effort
4th stimulus picture	Rectangle	Heaviness, mental burden, trouble
5th stimulus picture	Intersecting slanted lines in the lower left corner	Suggestion of overcoming tension, hostility, inherent aggression
6th stimulus picture	Horizontal and vertical lines	Integration, completeness, unconsciousness
7th stimulus picture	Semicircle drawn with dots	Delicacy, sensitive sensitivity, empathy
8th stimulus picture	Arched curve, bow-shaped	Harmony, relief, a sense of security

The characteristics of WZT allow for a clear understanding of an individual’s latent conscious attitude and unique relationship with the world. Adolescents draw open-ended pictures, expressing themselves naturally without defensiveness, and these drawings contain specific patterns that reflect individual personality traits (Roivainen & Ruuska, 2005).

In psychological typology, Jung classified individuals based on similar and homogeneous characteristics, even if their attitudes and mental functions differ, because certain patterns appear in specific parts of adolescent drawings that are formally similar. However, each unique type does have a certain degree of universality (Jung, 2014).

Violence prediction using CNN (softmax)

The CNN is an architecture used in deep learning to determine image features and is widely used in visual processing (Ding et al., 2022; Liu et al., 2021). The CNN is applied in various industrial fields, such as e-commerce product image searches and medical X-ray image analyses (Heidari et al., 2020; Huang & Liao, 2022). The layers in the CNN architecture consist of convolutional, activation, pooling, and fully connected layers. Ensembling multiple CNNs performs better than using a single CNN.

The activation functions used in the CNN layer are rectified linear units and softmax, and the optimiser is Adam, with sparse categorical cross-entropy as the loss function. The maximum number of epochs is set to 30, and if no improvement in validation accuracy occurs three or more times, the training is designed to stop.

For the softmax output, the interpretation of the result can employ probability values for N (nonviolence) and Y (violence), so the result [0.2, 0.8] can be interpreted as a 20% probability for nonviolence and an 80% probability for violence. The highest probability is selected as the final result. The detailed assumptions of the CNN are presented in Figure 2. All training processes were stored as pipelines to be automated and reused, and they were reused with minimal modifications in the following two sections on violence prediction.

Figure 2. Convolutional neural network (CNN) technique.

Violence prediction using the CNN (SVM)

The SVM is used for prediction and can predict nonlinear regression problems and time series data. The SVM performs well in solving binary classification problems (Niu & Suen, 2012; Senan et al., 2021). The activation function in the last layer of the CNN should be changed from softmax to linear, and the loss function should be changed from sparse categorical cross-entropy to hinge to use the SVM as the final classification method in the CNN algorithm.

The interpretation of the SVM result is that if the value is negative for 0, it is classified as nonviolence (N). If it is positive, it is classified as violence (Y), and the highest value is selected as the final result. The detailed assumptions of the CNN(SVM) are provided in Figure 3.

Figure 3. Convolutional neural network with the support vector machine (CNN(SVM)) technique.

Violence prediction using the style transfer GAN

Hand-drawn images have considerable white space, resulting in lower accuracy when analysing them with the CNN. Therefore, we generated new data using the style transfer technique to combine the WZT drawings with backgrounds, creating 268 new drawing images by applying this technique to the 134 violent and 134 nonviolent drawing images. The images used for style transfer were public domain photographs blended with content images to the extent that the original drawings were not damaged. Style transfer is a technique that combines a content image with a style image to create a new image. The sample images used in the experiment are illustrated in Figure 4. Insert caption here.

Figure 4. Style transfer GAN Technique

We modified the CNN and SVM loss function structure used in the experiment to train the models on the synthesised images. The original, unsynthesised data were used as the testing data. The reason for using different training and testing data is to compare the performance of the models using the same testing data in all experiments.

Violence prediction using ensembles

We combined the predictive models created in the previous sections to observe changes in predictive performance through ensemble methods. The ensembles used two types of voting techniques: hard and soft voting. In hard voting, the most common output among the results from each model used in the ensemble is chosen as the final output. Figure 5. shows 2–3 more application photos of the proposed research.

Figure 5. A collection of WZT 5 stimulation pictures for violent students

Evaluation criteria

The F-measure was employed as an evaluation criterion for the four algorithms used in deep learning techniques. Accuracy measures the proportion of predictions made by the algorithm that match the actual values in the overall dataset. Precision measures the ratio of correctly predicted answers (true positive or TP) out of the cases predicted to be correct (TP and false positive (FP)). Recall measures the ratio of correctly predicted answers (TP) out of all actual positive cases (TP and false negative (FN)) or the proportion of TP cases among all positive cases predicted by the algorithm. Precision and recall can be biased towards either positive or negative cases when an imbalance exists in the dataset. The F-measure uses the harmonic mean of the precision and recall to evaluate model performance. (1) $\begin{array}{rl}Accuracy& =\frac{\sum TP+\sum TN}{\sum Totalpopulation}\end{array}$(1) (2) $\begin{array}{rl}Precision& =\frac{\sum TP}{\sum TP+\sum FP}\end{array}$(2) (3) $\begin{array}{rl}Recall& =\frac{\sum TP}{\sum TP+\sum FN}\end{array}$(3) (4) $\begin{array}{rl}F-measure& =2\times \frac{Precision\times Recall}{Precision+Recall}\end{array}$(4)

Results

This study directly applied and analysed image data expressed on the WZT by adolescents to predict adolescent violence. The study aimed to predict violence using data from 134 violent students who received 5th-degree disciplinary action for violent behaviour and 134 nonviolent students. Among the eight WZT images drawn directly by one person, an image with aggressive tendencies was classified as an attack image, and its predictive performance was presented. Figure 6. below shows the results of 27 violent data used as test data, and figure 7. shows the result of 27 non-violent data using deep learning techniques.

Figure 6. Test dataset of WZT 5 stimulus drawing images of students who showed violence

Figure 7. Test dataset of WZT 5 stimulus drawing images of students who showed non-violence

As presented in Table 2, the CNN (softmax) technique failed to predict four violent pictures, and the combined CNN (SVM) and style transfer GAN techniques failed to predict three violent pictures. In addition, the CNN (SVM) method failed to predict one violent picture, and the ensemble method displayed the same results. Additionally, the CNN (softmax) technique failed to predict four violent drawing images. The combined CNN (SVM) and style transfer GAN techniques incorrectly predicted three violent drawing images. In addition, the SVM technique failed to predict one violent image, and the ensemble technique produced the same results.

Table 2. Confusion matrix results.

	TP	FN	FP	TN
CNN (softmax)	23	4	0	27
CNN (SVM)	26	1	0	27
CNN (SVM) + GAN	24	3	0	27
Ensemble	26	1	0	27

Notes: TP: true positive, FN: false negative, FP: false positive, TN: true negative, CNN: convolutional neural network, SVM: support vector machine, GAN: generative adversarial network

Table 3 demonstrates the accuracy of predicting violence using the fifth stimulus drawing from the WZT images, with a prediction accuracy of 92.5% to 98.1%.

Table 3. Algorithm evaluation criterion value.

	Accuracy	Precision	Recall	F1
CNN (softmax)	0.92592	1	0.85185	0.92
CNN (SVM)	0.98148	1	0.96296	0.98
CNN (SVM) + GAN	0.94444	1	0.88888	0.94
Ensemble	0.98148	1	0.96296	0.98

Notes: CNN: convolutional neural network, SVM: support vector machine, GAN: generative adversarial network

Discussion

Predicting adolescent violence is challenging. However, despite this difficulty, deep learning can predict adolescent violence. Recognising that more data are needed for higher prediction accuracy is essential. Deep learning technology has been advancing rapidly, and its results can sufficiently predict reliable characteristics. In this study, we predicted the features of adolescent violence in drawing images through WZT drawing images created by students. Moreover, research has attempted to develop models that can automatically apply sufficient data to increase prediction accuracy in complex patterns of drawing images that can use projection methods.

No one can fully explain the complexity of human beings, even if violence is justified. However, understanding all characteristics of violence is valuable. Predicting the elements of violence is a very cautious task. For students exposed to violence and experiencing difficulties adapting to school, suggesting educational methods for coping with violence is necessary. In addition, efforts should be made to effectively plan and implement learning and teaching methods to mitigate potential violence. Deep learning through the WZT drawing images by adolescents seems very helpful in predicting violence.

Conclusion

This study is the first to predict various personality types expressed in the Wartegg-Zeichen Test in adolescents based on a deep learning model and automatically analyse drawing images. We employed a dataset of image classes based on the style transfer GAN. Then, we used various methods to improve the prediction accuracy of violence in image data. The initial average prediction value of the CNN was 92.1%. The average prediction value of the ensemble increased to 98.1%, the same as the result of the SVM. The final average prediction value using SVM and style transfer GAN was 94.4%, suggesting that violence prediction can be automatically accurately predicted using SVM or ensemble methods. The fifth picture that best represents violence out of the eight drawing images in WZT is the one that can be used to predict violence automatically. Currently, as a follow-up to this thesis, more data are secured to improve the prediction rate, develop an improved model, and a follow-up thesis is being prepared.

Adolescence can be stressful with its physical, psychological, and social changes. This vulnerable and unstable phase can lead to various maladaptive issues including health. Therefore, it is crucial to increase awareness of stress during adolescence, and it is expected that deep learning can create many opportunities in multiple fields of violence prevention.

Acknowledgement

Acknowledgements, avoiding identifying any of the authors prior to peer review

Disclosure statement

The authors of this paper declare that they have no competing interests, financial or non-financial, to report.

Additional information

Funding

This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2023-2020-0-01789) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation) and this work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) under the Artificial Intelligence Convergence Innovation Human Resources Development (IITP-2023-RS-2023-00254592) grant funded by the Korea government (MSIT).