Beyond the Screen With DanceSculpt: A 3D Dancer Reconstruction and Tracking System for Learning Dance

Figures & data

Figure 1. Characteristics of commonly used dance practice videos (2D) (a) example scene in a video (https://www.youtube.com/watch?v=wU2siJ2c5TA&ab_channel=NewJeans) where the view shifts to highlight a specific body part of a dancer; (b) Another example scene in a video (https://www.youtube.com/watch?v=qsHvdRA0a6o&ab_channel=NewJeans) where the view is fixed to show the entire body of the dancer (Note that the two videos are different videos of the same dance).

Figure 2. The overall architecture of the DanceSculpt system (a) dancer viewer; (b) dance feedback tool.

Figure 3. UI examples of the DS system (a) dancer viewer; (b) dance feedback tool.

Figure 4. A summary of the experimental process over participants.

Table 1. Behavioral response variables and signals.

Behavioral variables		Signals
Mastery of dance elements (Pose/Trajectory/Formation/Timing)	Question	Verbal and physical interaction through questions (e.g., “Which way should I turn my hand?”). This typically occurs when users are not familiar with the dance elements.
	Confirmation	Physical interactions such as verbal responses and demonstrations when users have mastered the dance elements correctly (e.g., “I should go forward like this, and you should go backward”).
Emotion	Discontent	Expressions of difficulty or annoyance (e.g., sighing, grumbling, showing frustration).
	Confidence	Expressions of self-assurance or mastery (e.g., laughter, applause).
	Passion	Statements of motivation to learn more (e.g., “Let’s take a closer look”).

Table 2. The average frequency of behavioral responses observed in each stage (standard deviations in parentheses).

Behavioral response variables			Stage a		Stage b		Stage c		Total avg.
			CP	DS	CP	DS	CP	DS	CP	DS
Mastery of dance elements	Question	Pose	37.25 (18.41)	47.13 (19.37)	24.75 (11.18)	30.00 (22.08)	12.88 (5.99)	16.50 (4.24)	24.96 (15.98)	31.21 (20.78)
		Trajectory	6.25 (2.55)	11.88 (7.10)	16.38 (6.63)	12.00 (3.02)	20.00 (10.35)	22.88 (11.74)	14.21 (9.13)	15.58 (9.37)
		Formation	4.25 (2.82)	11.50 (6.70)	20.13 (5.57)	20.13 (9.46)	17.00 (9.25)	15.50 (7.75)	13.79 (9.15)	15.71 (8.49)
		Timing	2.75 (1.75)	2.25 (1.16)	1.50 (1.31)	11.63 (6.32)	17.75 (9.25)	19.38 (4.53)	7.33 (9.19)	11.08 (8.37)
		Sum of Question	50.50	72.75	62.75	73.75	67.63	74.25	60.29	73.58
	Confirmation	Pose	11.88 (8.89)	19.13 (8.08)	11.50 (6.52)	14.88 (3.18)	10.00 (2.51)	5.75 (4.20)	11.13 (6.29)	13.25 (7.80)
		Trajectory	6.50 (5.21)	18.38 (8.86)	4.50 (2.67)	18.50 (7.62)	3.13 (1.96)	15.38 (5.18)	4.71 (3.69)	17.42 (7.20)
		Formation	4.75 (2.38)	12.00 (6.41)	5.25 (4.65)	17.25 (8.43)	3.88 (2.30)	15.00 (7.17)	4.63 (3.20)	14.75 (7.39)
		Timing	0.88 (0.35)	2.13 (0.83)	1.50 (1.07)	3.13 (1.64)	3.63 (2.50)	4.00 (1.85)	2.00 (1.93)	3.08 (1.64)
		Sum of Confirmation	24.00	51.63	22.75	53.75	20.63	40.13	22.46	48.50
Emotion		Discontent	34.50 (26.12)	34.63 (23.93)	6.88 (5.57)	13.88 (5.82)	11.50 (4.90)	4.38 (1.60)	17.63 (19.41)	17.63 (18.76)
		Satisfaction	12.00 (10.61)	5.50 (4.41)	7.50 (5.50)	11.75 (14.68)	5.75 (1.91)	11.13 (4.36)	8.42 (7.20)	9.46 (9.25)
		Passion	2.88 (1.81)	6.75 (2.55)	6.00 (3.51)	10.38 (7.37)	1.25 (0.89)	5.13 (2.59)	3.38 (3.00)	7.42 (5.06)

Figure 5. Comparison of participants’ dance performance by learning methods.

Figure 6. Comparison of user experience of dance learning approaches.