Limits of medical research – some considerations

Quantitative medical research relies heavily on the use of statistics. Outcomes are statistically illustrated, and we make inferences from our research data to defined populations based on the statistical attributes of our results. We may look for indicators for correlations, or for causes and effects in our results, and try to extrapolate the findings to the realm of a more general population.

When we are thinking of causes and effects our ideal may be pictured as a simplistic mechanical model, for example a game of billiards. At the start, balls are lying on the green felt in random order. The player applies a force on one ball using the cue, and mechanistic laws decide how the balls roll, in theory. The cause is the force and direction of the cue and the effect is the movement of the balls. The cause is moving the balls from a random position to a deliberate position. An observer may interpret the pattern as improbable under the supposed random pattern, and may search for the cause of the ball position.

In reality, however, even billiard balls do not behave idealistically in accordance with mechanistic laws or statistical models. They do not consist of a point with a certain mass, nor is the surface idealistically hard or elastic. The green felt will be slightly uneven. Thus, even in this simple model, reality is much more complex than the model.

If we increase the number of balls the picture becomes more complicated. In order to understand the interaction between the balls, we may study a sample of a few balls. Then from the results of the sample we may use statistics to learn more of how the multiple balls that we have put on the billiard board may roll and interact. Statistics is the science whereby inferences are made about specific random phenomena on the basis of relatively limited sample material [1]. This goes directly for the RCT and also has consequences for epidemiological samples even if these are seldom “relatively limited”.

When we compare some variables of a given subset of data, we use statistics to get a p-value. We rejoice in getting a p-value below 0.05 or even 0.01, because we may then state that the probability that the difference we observe between the subgroups may be caused by chance in only 5% (1%) of possible outcome cases. That something may vary by chance is a characteristic of random phenomena. Here, in my opinion, we are at the crucial point. A random phenomenon is a mathematical concept. In reality few systems do vary at random. The case coming closest to a random phenomenon is the movement of molecules in a gas. Which phenomena in medicine vary at random? Do cell processes or organ processes in the human body really vary at random, that is by chance? To my knowledge, even at the cellular level, processes are highly regulated. From the viewpoint of family practice, all bodily processes are thoroughly regulated. A person is in addition directed by will, feelings, and social interaction. To a certain degree, the expression of the personal is a social construct. From this social construct we draw data for analysis as if the parameters were varying at random. Is, then, the use of statistics feasible? Quite often, using statistics in medical research, we may do so inappropriately, stretching the process beyond the limits of statistical science.

The above reasoning does not implicitly discredit any use of statistics, but rather points to the need for a broader research strategy. Going quantitative or qualitative, we tend to stay atheoretical in our own conception, relying on our research methodology to reveal some truth. The risk we then run is not only addressing too limited areas in the field we research, but we may also draw conclusions and point at extrapolations unwarrantedly. By relying too exclusively on statistics, we may fail to address properly the underlying conceptual frameworks and background theories that inevitably form our data interpretations and our conclusions. Different methodological approaches deliberately include and exclude different types of data, thereby supporting probably invisible competing knowledge philosophies, which have practical influence for the findings of studies and their interpretations [2]. Research communities in general practice may profit by critically considering the nature of the knowledge claims of their discipline [3]. Further, which epistemological discussions are needed concerning the nature of the knowledge in general practice, and which research strategies are needed to create valid information?

All too often empirical studies in general practice journals continue to focus overwhelmingly on the techniques of data collection and analysis [4]. Both quantitative and traditionally qualitative research methods standing alone may be inadequate tools for studying the complex, unstable, and non-linear conditions characteristic of many research areas in general practice, and for the interpretation of research results.

Research strategies must be formed by the research communities of general practice themselves. My attention is drawn to some generative (rather than hypothesis testing) research strategies for addressing complex problems, such as for instance (narrative) case studies or sociological strategies [5,6]. Other research strategies may be available, or should be created [7]!