What’s Creative About Sentences? A Computational Approach to Assessing Creativity in a Sentence Generation Task

Figures & data

Table 1. Overview of objective text metrics including a short description of their aim, implementation, output scale, and whether lemmatization was applied.

Metric		What	How	Scale	Lemmatization
1)	General word infrequency	Infrequency of words in a sentence	Mean score based on inverted frequency values provided by a German online dictionary (DWDS).	Low to high infrequency (0–6)	Yes
2)	Word combination infrequency	Infrequency of word combinations in a sentence	Total number of search results returned by Google, mapped on an (inverted) logarithmic scale, averaged across all possible combinations per sentence (=  6).	Low to high infrequency (log scale of base 10)	No
3)	Context-specific word uniqueness	Uniqueness of word choice per trial	Mean normalized relative occurrence of words used in a participant’s response compared to other responses in a trial.	Low to high uniqueness (0–1)	Yes
4)	Syntax uniqueness	Uniqueness of syntax, i.e., sentence structure	Mean normalized relative occurrence of a particular sentence’s order of part-of-speech tags compared to all sentences in the entire dataset.	Low to high uniqueness (0–1)	No
5)	Rhyme	Presence of words that rhyme	Check if sounds in word endings are identical and count total number of rhymes. Words are converted to numerical sound representations according to Cologne Phonetics.	0: no rhyme 1: 1 rhyme pair 2: 3 rhyming words 3: 2 rhyme pairs 4: all 4 words rhyme	No
6)	Phonetic similarity	Presence of similar sounds	Find recurring similar sounds based the word’s numerical sound representation according to Cologne Phonetics, based on mean of results from normalized Levenshtein distance and longest common subsequence (LCS).	No to very similar sounds (0–1)	No
7)	Sequence similarity	Similarity between cue word and words in the sentence in terms of spelling	Maximum similarity ratio between cue word and each word in a sentence (“difflib SequenceMatcher.ratio()” function).	Maximum similarity ratio per sentence (0–1)	No
8)	Semantic similarity	Similarity between cue word and sentence in terms of semantic meaning	Mean semantic similarity values (absolute cosine distance) between cue word and each word in the sentence. Implemented with SpaCy module (Explosion AI GmbH, 2022) using the available NLP pre-processing pipeline and a pre-trained model for German (“de_core_news_lg”).	No to high similarity (0–1)	Yes (SpaCy)

Table 2. Descriptive statistics, and intercorrelations of all objective measures.

	M	SD	1	2	3	4	5	6	7	8
1. Creativity rating	1.12	0.52
2. General word infrequency	2.35	0.73	.20
			[.13, .25]
3. Word combination infrequency	3.13	1.20	.24	.66
			[.18, .29]	[.63, .70]
4. Context-specific word uniqueness	0.81	0.13	.30	.25	.36
			[.24, .36]	[.19, .30]	[.30, .41]
5. Syntax uniqueness	0.83	0.24	.15	−.15	−.16	−.08
			[.08, .21]	[−.21, −.09]	[−.22, −.10]	[−.14, −.02]
6. Rhyme	0.19	0.42	.15	.09	.04	−.01	.10
			[.09, .21]	[.03, .15]	[−.02, .11]	[−.07, .06]	[.04, .16]
7. Phonetic similarity	0.29	0.10	.10	.02	.07	.01	.03	.17
			[.04, .16]	[−.04, .08]	[.01, .13]	[−.06, .07]	[−.04, .09]	[.11, .23]
8. Sequence similarity	0.53	0.16	−.05	.03	−.06	−.18	.01	−.04	.15
			[−.12, .01]	[−.03, .09]	[−.12, .01]	[−.24, −.12]	[−.06, .07]	[−.11, .02]	[.09, .21]
9. Semantic similarity	0.09	0.05	.07	.01	−.06	−.11	.02	−.01	.13	.30
			[−.00, .13]	[−.06, .07]	[−.12, .01]	[−.17, −.04]	[−.04, .09]	[−.07, .06]	[.07, .20]	[.24, .36]

N =  989. Values in square brackets indicate the 95% confidence interval for each correlation. |r| > .062: p < .05; |r| > .082: p < .01; |r| > .104: p < .001.

Figure 1. Distribution of text metric scores.

Table 3. Multiple linear regression analysis (enter method; bootstrapped with 2000 samples) predicting human ratings by variables showing significant zero-order correlations with the criterion.

Predictor	ß	ß 95% CI	p	sr^2	sr^2 95% CI	r (p)
(Intercept)			< .001				R^2=  .167**, 95% CI [.13, .22]
General word infrequency	.07	[−.00, .15]	.084	.00	[.00, .01]	.20 (< .001)
Word combination infrequency	.11	[.03, .19]	.001	.01	[.00, .02]	.24 (< .001)
Context-specific word uniqueness	.25	[.19, .32]	< .001	.06	[.03, .08]	.30 (< .001)
Syntax uniqueness	.18	[.14, .23]	< .001	.03	[.02, .05]	.15 (< .001)
Rhyme	.11	[.04, .17]	< .001	.01	[.00, .03]	.15 (< .001)
Phonetic similarity	.07	[.01, .13]	.015	.00	[.00, .02]	.10 (< .01)

CI, 95% percentile confidence interval based on 2000 bootstrapping samples; r, zero-order correlation; sr², squared semi-partial correlation; ß, standardized regression weight.

Explosion AI GmbH. (2022). SpaCy industrial-strength natural language processing in python. https://spacy.io/

 Google Scholar