Visualizing Multiple Regression

Figures & data

Figure 1. (a) Uncorrelated Variables. (b) Correlated Variables With Redundant Information in Salary Example. The area of an oval denotes the regression sum of squares for the variable.

Figure 2. (a) Venn Diagrams of SS of Two Variables. Darker and lighter shades, respectively, correspond to SS(x₁) and SS(x₂). (b) SS(x₂ | x₁) is Indicated by Shaded Area.

Figure 3. Venn Diagram Showing Partial Correlations With Two Variables (x₁, x₂) Partialed Out.

Figure 4. Type I SS for x₂ (Shaded Region) When the Order is (a) x₁, x₂, x₃; (b) x₁, x₃, x₂.

Figure 5. Type II SS for x₂ (Shaded Area). It is equivalent to the Type I SS when the variable is the last predictor entered.

Table 1. SS of Partitions in the Venn Diagram in Figure 6

Variable	SS
family only	0.8
peer group only	8.3
school only	0.4
family and peer group only	0.7
family and school only	4.2
school and peer group only	3.3
family, school, and peer group	10.7
Total SSR	28.4

Figure 6. Venn Diagram Showing SS in Achievement Example.

Figure 7. Venn Diagram Showing SS in Average Stepwise Regression.

Table 2. Example of Suppressor Variable (Kendall and Stuart 1973)

x1	x2	y
2.23	9.66	12.37
2.57	8.94	12.66
3.87	4.40	12.00
3.10	6.64	11.93
3.39	4.91	11.06
2.83	8.52	13.03
3.02	8.04	13.13
2.14	9.05	11.44
3.04	7.71	12.86
3.26	5.11	10.84
3.39	5.05	11.20
2.35	8.51	11.56
2.76	6.59	10.83
3.90	4.90	12.63
3.16	6.96	12.46

Figure 8. Partition of Areas When There Are Three Variables.

Table 3. Summary of Two-Sample t-test (Two-Sided) for Treatment and Comparison Groups

	Comparison Group	Treatment Group
Average score	2.496	3.000
Standard deviation	1.67	1.72
Sample size	133	97
t-statistic	t = 2.22

Kendall, M., and Stuart, A. (1973), Advanced Theory of Statistics (Vol. 2; 3rd ed.), NY: Hafner.

 Google Scholar