Testing Ten Theories

Abstract

Using the most comprehensive data set now available, this investigation tests the precision of all exchange theories that now contend. Beyond precision, the investigation focuses on broad issues of effectiveness including consistency, parsimony, and whether the theories can be applied to structures larger than normally studied in the lab. Seeking greater parsimony, this investigation introduces a new model by combining parts of two contending theories. We find that all ten theories have scientific merit for all can predict with some effectiveness for the exchange structures experimentally investigated. Nevertheless, the ten vary in precision. Elementary Theory is the most precise. The new Expected-value Resistance model ranks second in precision and is the simplest. Both apply to large networks as well as the best of the other theories.

Keywords:

• experiment
• explanation
• network exchange
• power
• precision
• prediction
• science
• scope
• social structure
• theory testing

Research reported here was supported by grants SBR-9423231, IIS-9817518 and SBR-9811323 to the first author from the National Science Foundation.

Notes

¹As Lucas (2003) points out, the “external validity” problem, so often attributed to the experimental method, is resolved when experiments test theory that applies outside the lab, as do the theories tested here.

²Nevertheless, falsifications have occurred in network exchange. Friedkin demonstrated that Markovsky et al.'s (1988) graph theoretic power index gave multiple contradictory predictions for some networks (personal communication). That demonstration and the struggle to fix the index are recounted in Lovaglia et al. (1996). Earlier, Willer (1986) showed that Power-Dependence's vulnerability procedure offered logically and mathematically impossible predictions. It might be thought that if a theory is logically consistent, it cannot produce inconsistent predictions, but graph theoretic and vulnerability procedures show that the suggestion is wrong. Both are internally consistent, but both produce contradictory predictions.

³Also see Lakatos (1970), Popper (1994), and Kuhn (1970). For Kuhn, the evolutionary image employed in his 1969 Postscript (found in Kuhn, 1970) is particularly to the point.

⁴The four listed criteria are prominent in the works of philosophers of science and sociologists already referenced and in Fararo and Kosaka (2003). We adopt the following meaning for precision: “Theories are precise to the degree that they generate accurate and detailed statements about phenomena.” (Markovsky, 1996, p. 34). Effectively the same meaning is also found in Wagner and Berger (1985) and Wagner (1984, 1994). By range of application we mean the variety of phenomena to which a theory can be successfully applied. That meaning is drawn from Walker and Cohen (1985) and agrees with meanings given in the previously cited papers.

⁵To our knowledge no one has suggested that a lack of simplicity is sufficient to reject a theory; nevertheless, parsimony is a desirable quality. For a theory to become rapidly more complex is a clue to the theorist that a new start would be fruitful. For example, shortcomings in the graph theoretic power index found by Friedkin (see footnote 2) were solved only by making that index massively more complex (Lovaglia et al., 1999) such that it has been displaced by the much simpler Girard and Borch method given later in this paper. As will be seen, some of the ten theories have become more complex as they were developed.

⁶Some of the theories tested here have no scope of application but that investigated in the experiments; however, others are broader. Comparing the broader theories, beyond the scope tested here, there is little or no scope overlap. This lack of overlap introduces incommensurability blocking judgments of relative breadth of scope. For example, Expected Value Theory also applies to influence structures, but Elementary Theory does not. By contrast, Elementary Theory recognizes seven power conditions, but Expected Value Theory recognizes only exclusion, the one investigated here. Which scope is broader? Incommensurability blocks any answer to that question.

⁷Van Assen (2001) has shown that payoffs from resource pool divisions are similar to some but not all exchanges.

⁸Homomorphically equivalent positions cannot be distinguished once arbitrary labels are removed. Since positions are indistinguishable, it is logically and empirically impossible to assign different payoffs across positions.

a. Eight sessions were run each consisting of four periods containing four rounds. At the end of each period, subjects were rotated to a new position. Data points were calculated for each period by taking the average value obtained in the last three rounds. Round one was never used.

b. Eleven sessions were run each consisting of four periods containing four rounds. At the end of each period, subjects were rotated to a new position. Data points were calculated for each period by taking the average value obtained in the last three rounds. Round one was never used.

c. Seven sessions were run each consisting of four periods containing four rounds. At the end of each period, subjects were rotated to a new position. Data points were calculated for each period by taking the average value obtained in the last three rounds. Round one was never used. In two periods of one of the sessions, no agreement was reached.

d. Seven sessions were run each containing six periods consisting of four rounds each. At the end of each period, subjects were rotated to a new position. Data points were calculated for each period by taking the average value obtained in the last three rounds. Round one was never used. In two periods agreements were not reached.

e. Seven sessions were run each containing six periods consisting of four rounds. At the end of each period, subjects were rotated to a new position. Data points were calculated for each period by taking the average value obtained in the last three rounds. Round one was never used.

f. Eight sessions were run each containing six periods consisting of four rounds. At the end of each period, subjects were rotated to a new position. Data points were calculated for each period by taking the average value obtained in the last three rounds. Round one was never used. In one period agreement was not reached in two of the last four rounds and the datum point was not used.

g. Five sessions were run each containing four periods consisting of ten rounds each. At the end of each period subjects were rotated to a new position. Data points were calculated for each period by taking the average value obtained in the last six rounds. Rounds one through four were never used.

Note: All predictions without a star are not significantly different from the observed means.

∗Significant at < .1.

∗∗Significant at < .01.

∗∗∗Significant at < .001

⁹Beyond strong, equal and weak, there are also compound networks that contain breaks resolving them into strong, weak and/or equal power component parts. See discussion of Figure 2.

¹⁰One further network, the Kite also was listed in that special edition, but power differences predicted for it by all theories are too small. Thus, by criterion 1, it is not included here.

¹¹This program and, with noted exceptions, others mentioned below are available from the authors of this paper.

¹²The consistency problem is not due to a glitch in Skvoretz's program: the inconsistent predictions are given by Cook and Yamagishi (1992).

¹³As can be seen by examination of equation 1, Power-Dependence predictions for agreed upon exchange ratios will shift as subject utilities differ. Predictions for experiments just referenced by Power-Dependence theorists are not qualified by those utility differences nor are ours. Power-Dependence theorists have not tested this part of their theory nor do we, a scope limit of this study. Still, even if differing utilities do shift agreements, random assignment and systematically changing subject pairings in exchange as in the reported experiments should wash out the effects.

¹⁴The simplest network for which contradictions are produced is A–B–C–B–A. Beginning l calculations at either B gives distinct and different values to the two Bs and also to the two As. Giving different values to the Bs is contradictory because the two are identical and similarly for the two As.

¹⁵In some of the exchange literature, simulation results have been presented as data supporting theory. Simulations like X-Net are theories and its results are predictions.

¹⁶But in L4 it is not clear to us that both Bs must gain 12 or more. For example, if one B gained P_B = 18, by the expressions above the other could gain as little as P_B = 6. Thus it seems to us that both Bs need not be favored. Nevertheless, for testing this and other networks we follow Bienenstock and Bonacich.

¹⁷The calculation of w _ij values allows the 1995 formulation to be applied to networks in which resource pools are not equal in size. See Bonacich and Friedkin (1998).

¹⁸We date the Skvoretz-Fararo version of Coleman's theory to 1995 for it was then that predictions from it were first tested (Lovaglia et al., 1995).

¹⁹Yamaguchi (2000) identifies complementarity with Power-Dependence Theory's positive connection. Since that condition is not within the contended scope it is not discussed here.

²⁰Details of the experimental design used here had been published prior to Yamaguchi's 1996 paper. In that paper he suggests that, in these experiments, relations are not perfectly substitutable because time constraints introduce transaction costs when switching partners, but that suggestion is not correct. In these experiments, negotiations do not go on first in one relation and then switch to become ongoing in another. To the contrary, subjects negotiate in all of their relations simultaneously and normally have standing offers in all prior to selecting their best deal. Pretests allow time constraints to be set so that subjects can optimize across their alternatives.

²¹It is not unusual for theories to be applied outside their scope. The classical gas laws are for an “ideal gas” that, because its atoms are dimensionless, cannot exist. Thus all applications of gas laws are formally outside their scope, yet useful predictions result.

²²Whereas the Core is the simplest of the theories, it does not make point predictions for weak power networks.

²³Interested scholars can replicate this research on ExNet 3.0 that is located at weblab.ship.edu. An earlier system, ExNet 1.0 was used to collect some of the data for L4, Stem and Box-Stem. Though the screen was not active, the network was displayed at each subject station as were all ongoing offers and exchanges.

²⁴Rotating subjects through positions, a procedure that has been used for more than 20 years to control for individual differences, allows stronger inferences from structure to exchange ratios. By generating new subject pairings, each session gives more than one datum point. Subject rotation may also reduce equity concerns and reactions to injustice that might affect power and exchange dynamics (Cook and Emerson, 1978; Molm and Cook, 1995; Hegtvedt and Killian, 1999). Lovaglia et al. (1995) find that mean payoffs by position are not significantly different when, under more limited information conditions, subjects are held in a single position.

²⁵That those data are the same is masked by the different exchange ratios reported in the two papers. Exchange ratios differ, at least in part, because the two papers use two different modes of calculation. Skvoretz and Willer (1993) estimate exchange ratios by a constrained regression whereas Lovaglia et al. (1995) take the mean.

²⁶Previous analyses suggest that means reported in Table 1 represent equilibrium values. Using the data set of this research, Emanuelson (2005:158) found no significant differences between the lowest observed mean of the first two experiment periods and the highest observed mean of the last two experiment periods for all networks studied here. By Emanuelson's analysis, exchange ratios of weak power networks, unlike those of strong power networks, do not increase over time. Following that analysis, means reported here are taken as equilibrium values.

^a The Deviation Score is the average deviation between predictions and observed means weighted by the number of expected exchanges as calculated using Expected Value (1993) likelihoods.

^b Deviation Score and rank by number of Supported Predictions not at equilibrium. See text.

²⁷Here the use of t-tests is unconventional for it is a test of H₁ not H₀. The predicted values are tested against the observed. When the two are close, the test is not significant. More importantly, the purpose is not evaluate a single theory, a use where their liability for type one error might be considered to be a shortcoming. Instead they are used only to rank order the relative precision of the ten theories. Standing in the rank order is determined by the number of predictions not significantly different from observed values. In principle, that standing can be affected by the cut off point chosen. Fortunately, as the reader can determine, this rank ordering is relatively immune to the chosen cut off point. There are no reversals when < .1 or < .01 are counted as supported predictions.

²⁸Bonacich objects to the testing of point predictions asserting that it has been shown that the results of all social science experiments are culture bound. Therefore, all point prediction tests are meaningless (Personal Communication). We offer two comments on this objection. First, dispensing with point predictions is undesirable for it dispenses with precision, a crucial and well established criterion by which scientific theories have long been compared. Second, it has not been shown that all social science experiments are culture bound. Some certainly are. Cross cultural studies of ultimatum, public goods and dictator games have shown very substantial variation (Henrich et al., forthcoming). Others have not. Experiments on an array of coercive and exchange structures conducted in the United States and in Communist Bloc Poland failed to show different outcomes (Willer and Szmatka, 1993). Since those experimental structures are included in the scope of the theories studied here, they are the ones from which possible cross-cultural variations should be inferred.

²⁹Above it was noted that, in equal power networks, all positions are identically connected such that, but for their labels, they cannot be distinguished. The same rule applies when subnetworks, like the J, K, L, M box, when broken from a larger network.

³⁰As already mentioned, the program used here for Power-Dependence was not developed by the authors of the theory but by Skvoretz who was interested in experimental tests on relatively small networks. (See Skvoretz and Willer, 1993.) Until more capable programs are developed, the Skvoretz program defines the range over which the theory can be applied.

³¹See results for null connected networks in Willer and Skvoretz (1997).

³²By contrast, at X-Net's default of 25 rounds, standard deviations are all nonzero and many are substantially larger indicating that equilibrium is not attained.

³³It has been suggested that a “theory” that predicts equal 12 - 12 divisions will score better than any of the theories evaluated here. That suggestion is easily checked. The 12 - 12 “theory,” if it be such, has a deviation score of 1.28 and two supported predictions. Thus it qualifies just under mid-pack of the theories. By contrast, the best theory has a deviation score less than 1/4 as large and has more than four times the number of supported predictions. Nor does the 12 - 12 “theory” stack up well against even the least precise theories. Unlike the 12 - 12 “theory,” both Power-Dependence and Expected Value correctly rank many power differences across networks. For example, Stem > L4 > DBox.

³⁴Here we follow Hempel (1952, 1965) and Toulmin (1963). Prediction and explanation differ only by whether the derivation came before or after the fact, respectively.