A 30-Year Citation Analysis of Bibliometric Trends at the Archives of Environmental Health, 1975–2004

ABSTRACT

This article describes a 30-year citation analysis of the Archives of Environmental Health (AEH), from the earliest available data in 1975, to 2004, when it became the Archives of Environmental & Occupational Health (AEOH). Longitudinal trends were examined regarding the number of items published, the number of citations received, the immediacy index, and the journal's impact factor. A list of the 5 most highly cited articles was also established, including citation frequency and citation lag times. Overall, this study demonstrates that citation analysis can provide an interesting look at the development of a journal over time. The examination of what articles, themes, and topics were being published, cited, or ignored also offers a unique insight into the direction of not only a particular journal, but also the discipline within which it exists.

KEYWORDS:

• bibliometrics
• citation analysis
• citation trends
• impact factor
• occupational health
• scientific journal

Citation analysis can be described as an examination of the frequency and pattern of citations in articles and books. ¹ It represents the most common method in a broader discipline known as bibliometrics; the use of mathematical techniques to investigate publishing and communication patterns in the distribution of literature. ² Modern citation analysis can trace its origins back to 1873, when the Frank Shepard Company of Colorado Springs, Colorado, first began publishing a list of American court cases, each followed by a complete history recorded in a simple code. ³ Their product known as Shepard's Citations was widely used to establish which legal cases had since been referred to, other court decisions that had affected a particular case, as well as any other information of potential value to the lawyer. ⁴ The citation database proved to be of great value because court cases, and indeed law itself, is usually based on precedent. ³ By the turn of the 20th century, interest in citation analysis had expanded beyond the legal profession and had begun attracting the attention of various librarians and scholars, although much of their early work involved simply counting references and sorting reference lists. ⁵

One of the first citation analyses ever conducted in science was an examination of literature in the field of comparative anatomy between 1543 and 1860, reported by Cole and Eales in 1917. ⁶ In 1927, Gross and Gross ⁷ from Pomona College in California examined references from a single volume of The Journal of the American Chemical Society. From this, the authors established which journals were being most frequently cited, and thus, which journals would be the most useful for a library to purchase. Their technique of reference counting was based on a few basic assumptions: firstly that the usage of a scientific periodical is reflected in the number of times its articles are cited, and secondly that the journals they chose to examine were actually representative of the field in which they existed. Although it took no account of a journal's relative size or citation density, the “Gross and Gross method” would nevertheless be used across a variety of fields ^{8 , 9 , 10} and, for better or for worse, would go on to influence an entire generation of librarians and scientists. ¹¹

In 1934, Bradford ¹² published an examination of the journals Applied Geophysics and Lubrication, reporting that each reference list contained a few very productive sources, a large number of moderately useful sources, followed by an even larger number of constantly diminishing productivity. Bradford surmised that finding all appropriate articles on a given subject would require the scrutiny of several thousand journals, the bulk of which would yield only occasional references, or none at all. ¹² In 1944, Brodman ¹³ examined methods for choosing physiology journals, and was able to reveal further insight into the world of citation analysis, particularly with regard to its potential shortcomings. Firstly, Brodman demonstrated that no journal is truly representative of an entire field and that journals should not be weighted equally. Most importantly, however, she demonstrated that long-held assumptions of the Gross and Gross method were simply not true. ¹³ In time, various other scholars would also begin to question the previously accepted Gross and Gross method. In 1960, for example, Raisig ¹⁴ reexamined the 5 highest-ranked periodicals in Gross and Gross's original 1927 study, and subsequently proposed a new method for citation analysis and a new ranked list of periodicals.

As the world of scientific literature continued to expand throughout the 20th century, it became increasing difficult for librarians and scholars to simply keep track of all the available material. Indeed, the body of published knowledge had become enormous and was continuing to grow. By the mid-1950s, for example, it had been estimated that Shepard's Citations alone contained at least 50 million citations. ³ Approximately 30,000 annual court cases in the United States added around 1 million extra citations per year to the database. ¹⁵ For these reasons, it had become clear that any citation analysis in the scientific world would be considerably facilitated by the generation of a citation index. A citation index can be described as an ordered list of cited articles, with each accompanied by a list of citing articles. ¹⁶ Perhaps the most well-known of these, the Science Citation Index® (SCI®), was originally developed by Eugene Garfield in 1955 as a means of facilitating the dissemination and retrieval of scientific literature. ³ Citation analysis as a tool for journal evaluation would later be described by him in 1972, ¹⁷ and by 1974, the SCI® was covering around 400,000 articles from approximately 2,500 journals. ¹⁸ By 1975, citation analysis was being used by a variety of science administrators, universities, science foundations, and civil courts. ¹⁹ In 1976, Garfield's company, the Institute for Scientific Information (ISI), now Thomson Reuters, would go on to publish the Journal Citation Reports® (JCR®), ²⁰ and issue what has arguably become its most famous by-product, a list of journals ranked by average citation frequency, known as the journal impact factor. ²¹

With the advent of science citation indexes in the mid-20th century, citation analysis as a discipline continued to evolve, and by the 1960s it had been suggested that more time was now being devoted to the selection of what is to be read, rather than the actual reading itself. ²² The use of these techniques had also expanded to include social sciences, with a 1967 article describing the psychological journal network between 1950 and 1960. ²³ In 1972, the ISI introduced the Social Sciences Citation Index® (SSCI®), ²⁴ from which some of the earliest social science citation analyses were conducted during the 1970s and 1980s. ^{25 , 26 , 27 , 28} Psychology journals appear to have been first ranked in 1973, ²⁶ with citation analysis first being conducted in this field during 1977. ²⁸ Citation analysis was also being undertaken in the field of tropical medicine during the 1970s, ²⁹ a trend that would continue in later years. ^{30 , 31 , 32 , 33} In 1981, Pearson ³⁴ published his citation analysis of drug information services activity between 1961 and 1979. In the same year, citation analysis was conducted in some of the smaller, specialist disciplines. ³⁵ In 1982 an analysis of the Journal of Medical Education was conducted, ³⁶ followed by an analysis of the Journal of the American Dietetic Association (1967–1983), conducted by Lansing and colleagues ³⁷ and published in 1985. The following year would see the publication of citation analyses of the individual journals Fertility and Sterility ³⁸ and Physical Therapy, ³⁹ to name a few. One of the first studies in dermatology, an analysis of the Journal of Investigative Dermatology using the SCI®, was published in 1989, ⁴⁰ followed by a citation analysis of all clinical journals in this field between 1945 and 1990. ⁴¹

By the turn of the 21st century, the evaluation of journal performance had become a hot topic in the world of contemporary academic publishing, with citation analysis having been conducted in almost every discipline. One neglected aspect of citation analysis, however, has traditionally been the environmental and occupational health (EOH) sciences field, where few detailed investigations appear to have been conducted. ⁴² Indeed, it was not until 1992 that the first bibliometric research specifically dedicated to our field was published by McCunney and colleagues, who looked at the journals of occupational medicine ⁴³ and environmental health. ⁴⁴ One thing the authors did find was that the field of EOH appeared to be expanding. In their latter article, ⁴⁴ for example, the authors demonstrated a consistent increase in published articles indexed under the headings of “environmental pollution,” “environmental pollutants,” and “environmental cancer.” Although the journal, the Archives of Environmental Health (AEH) was included as part of their wider investigation, no one has ever conducted a detailed analysis of bibliometric indicators for the AEH over a long period of time. For these reasons, the current study was undertaken as the definitive bibliometric analysis of the journal, and one that would include one of the longest periods of citation analysis ever investigated, the 30-year time period between 1975 and 2004.

METHODS

Data Sources

Historical data for the current study was obtained from 2 sources at Thomson Reuters. Citation counts for individual articles published in the AEH (including number of citations received and citation time lag) were extracted from the Web of Science® (WOS®) database during early 2009 and subsequently ranked. Historical data on journal performance (including citable items, total citations received, impact factors and immediacy indices), was extracted from the JCR® via a custom order during late 2008. Although an early impact factor for the AEH is listed in one of Garfield's books, ⁴⁵ the earliest official data for our journal is from 1975, the first year that the JCR® was issued in conjunction with the SCI®. ²⁰ As such, 1975 was chosen as the start point for the current study, with the final year of the AEH (2004) being used as the termination point for analysis. The current data set therefore contains a complete 30-year bibliometric history of the journal during this time period.

Items Investigated

A variety of items were investigated in the current study, beginning with the category of articles published. Four individual bibliometric indicators were then examined, Citable Items, Citations Received, Immediacy Indices, and journal Impact Factors; similar to previous research conducted in a variety of fields. ^{46 , 47 , 48} The term “Citable Items” refers to articles that are most likely to be cited by others, usually including original manuscripts and reviews. The exact definition of this term is ascertained by Thomson Reuters, although some description on how it is calculated was originally described by Garfield in 1986, ⁴⁹ then founder and Chairman of the ISI. The term “Citations Received” refers to the number of times that articles published in a given journal are cited in the reference lists of any journals in a given year. ⁴⁸ The term “Immediacy Index” represents the average number of times that an article published in a certain journal in a certain year is cited over the course of that same year. ⁵⁰

A journal's “Impact Factor” is calculated by ascertaining the number of times that articles published in a particular journal in a given 2-year period are cited in the following year, divided by the total number of “citable” articles published by that same journal during the same 2-year time period. ⁵¹ A watershed in journal performance occurs when its impact factor exceeds 1, indicating that, on average, its articles are being cited more often than they are being published. ⁵² The current study also sought to identify the most highly cited articles ever published in the AEH, the so-called Citation Classics. The term “Citation Classic” was coined by Garfield in 1977, ⁵³ and can be described as individual articles that have attracted a higher than average number of citations since being published. ⁵⁴ Further information on the exact definitions of bibliometric terms used by Thomson Reuters can be found on the company Web site. ⁵⁵

Analysis Methods

A variety of analysis methods were used in the current study. The first, establishing the category of articles published in the AEH between 1975 and 2004, was achieved by searching the Web of Science® database and then filtering journal articles by category. Trends in the number of citable items and citations received were plotted by year, as too, were trends for the immediacy index and impact factor scores. A considerably more detailed analysis of impact factor performance was then undertaken using standard bibliometric techniques. These included the ascertainment of impact factor trends ⁵⁶ by calculating the Index of Annual Change (IAC) with its associated Probability (P) values, as previously described. ⁵⁷ Average impact factor scores were also established, with the results including the Standard Deviation (SD) of impact factor fluctuations and 95% Confidence Intervals (95%CI), similar to studies undertaken by others. ^{58 , 59 , 60 , 61} Impact factors were analyzed in 5-year blocks (1975–1979, 1980–1984, 1985–1989, 1990–1994, 1995–1999, 2000–2004) and also for the overall time period, 1975–2004. Highly cited articles were identified by using the Web of Science® database search function, similar to previous studies, ^{62 , 63 , 64 , 65} and are current as of early 2009. The proportion of all citations received for each article was then calculated for the first 2, 5, and 10 years following publication. A plot of citation lag time for these highly cited articles was also constructed, similar to other investigations. ⁴⁶

RESULTS

Category of Articles

Each category of article published between 1975 and 2004 and its proportion of the total articles are displayed in Figure 1. A total of 2,054 articles were published in the 30-year period, of which Research Articles comprised the vast majority (84.2%). Editorials and Letters-to-the-Editor were the next most common type, accounting for 4.6% each. This was followed by Corrections or Additions (1.8%), Notes (1.4%), Book Reviews (1.0%), Proceedings (1.0%), Meeting Abstracts (0.8%), Review Articles (0.4%), and Biographical Items (0.2%).

Citable Items and Citations Received

Time trends in the number of citable items published and the number of citations received each year are displayed in Figure 2. As Figure 2 suggests, the journal published a peak number of citable items in its first 2 years (128 and 113, respectively), with an overall average of 60 citable items per year. The number of citations received peaked in 1977 with 2,775, and averaged approximately 2,273 citations received per year over the 30-year period.

Fig. 1 Category of articles published in the Archives of Environmental Health, 1975–2004. (As identified by the Thomson Reuters Web of Science® database.)

Fig. 2 Citable items and citations received by the Archives of Environmental Health, 1975–2004. (Data adapted from the Thomson Reuters Journal Citation Reports®, 1975–2004.)

Fig. 3 Impact factors and immediacy indices at the Archives of Environmental Health, 1975–2004. (Data adapted from the Thomson Reuters Journal Citation Reports®, 1975–2004.)

Immediacy Indices and Impact Factors

Time trends in the impact factor and immediacy index are displayed in Figure 3. The immediacy index peaked for 2 years in 1985 and 1986 at 0.860 and 0.814, respectively, and averaged 0.085 over the entire 30-year period. The journal's impact factor experienced 2 separate peaks, firstly in 1977 when it was 1.780 and again in 1994 when it was 1.831. An in-depth analysis of the impact factor in 5-year increments revealed that the mean score fluctuated from 0.856 between 1985 and 1989, to experience its peak of 1.622 between 1975 and 1979. The overall average impact factor score for the AEH was 1.240, as indicated in Table 1.

Table 1 30 Years of Impact Factor Trends at the Archives of Environmental Health, 1975–2004*

	1975–1979	1980–1984	1985–1989	1990–1994	1995–1999	2000–2004	Overall
Minimum	1.385	0.860	0.814	1.058	1.226	0.853	0.814
Maximum	1.780	1.121	0.950	1.831	1.776	0.163	1.831
Mean ^a	1.622	0.983	0.856	1.406	1.446	1.131	1.240
S.D. ^a	0.145	0.097	0.056	0.313	0.213	0.351	0.342
I.A.C. ^b	−0.061	0.031	0.018	0.192	−0.103	−0.192	−0.003
P value ^b	0.226	0.391	0.382	0.006^#	0.130	0.058	0.649
*Raw data extracted from the Thomson Reuters Journal Citation Reports®.
^a Mean and Standard Deviation (S.D.). ^59
^b Index of Annual Change (I.A.C.) and P value for slope. ^57
^#Statistically significant increase in the impact factor trend.

Highly Cited Articles

A list of the most highly cited articles published in the AEH between 1975 and 2004 is displayed in Table 2. The most highly cited article, written by Pope and colleagues from Brigham Young University in Utah, described a study of daily mortality and PM₁₀ pollution in Utah Valley. ⁶⁶ Around 5% of all citations it has so far attracted were received in the first 2 years following publication, 28.3% had been received within 5 years, and 71.4% within the first 10 years. Of the other 4 highly cited articles, Falck and colleagues ⁶⁷ had received 77.3% of its citations within 10 years, whereas Popes’ second article on the highly cited list ⁶⁸ had received 65.4%. The articles by Lancranjan et al ⁶⁹ and Shamberger et al ⁷⁰ had only received 12.6% and 44.8% of their total citations, respectively, within the first 10 years. Figure 4 displays a plot of citation lag time for the 5 most highly cited articles, suggesting that the majority of citations were received within the first 10 to 15 years following publication, after which time progressively fewer citations were being attracted.

Table 2 Highly Cited Articles From the Archives of Environmental Health, 1975–2004*

	Article Details					Proportion of all Citations**
Rank*	Author(s)	Title	Year	Volume	Pages	2 Years	5 Years	10 Years
1	Pope CA, Schwartz J, Ransom MR. ^66	Daily mortality and PM10 pollution in Utah Valley.	1992	47	211–217	5.1%	28.3%	71.4%
2	Falck F, Ricci A, Wolff MS, et al. ^67	Pesticides and polychlorinated biphenyl residues in human breast lipids and their relation to breast-cancer.	1992	47	143–146	13.5%	41.1%	77.3%
3	Pope CA. ^68	Respiratory hospital admissions associated with PM10 pollution in Utah, Salt-Lake, and Cache Valleys.	1991	46	90–97	9.4%	32.9%	65.4%
4	Lancranjan I, Popescu HI, Gavanescu O, et al. ^69	Reproductive ability of workmen occupationally exposed to lead.	1975	30	396–401	0%	0%	12.6%
5	Shamberger RJ, Tytko SA, Willis CE. ^70	Antioxidants and cancer. Part VI. Selenium and age-adjusted human cancer mortality.	1976	31	231–235	0.5%	12.4%	44.8%
*As identified by the Thomson Reuters Web of Science® database.
**Proportion of all citations received for each article within 2, 5, and 10 years of publication.

Fig. 4 Citation lag time of the 5 most highly cited articles at the Archives of Environmental Health, 1975–2004. (Raw data extracted from the Thomson Reuters Web of Science® database.)

DISCUSSION

A citation-based analysis of the AEH, as provided in the current study, offers an interesting insight into its development over time. The journal was officially launched in July 1960, following an earlier meeting of the Board of Trustees of the American Medical Association (AMA). According to the Director of the AMA's Division of Scientific Publications, this move was in recognition of journal's broader mission to encompass not only occupational medicine, but also preventive and aerospace medicine. ⁷¹ Katharine Boucot (Boucot Sturgis after 1962) (1903–1987) was appointed as foundation Editor-in-Chief, ⁷² a position she would eventually occupy for 11 years. ⁷³ Although the journal's chronological history has been described elsewhere, ⁷⁴ citation analysis reveals an additional dimension of its development over time, by examining which articles and themes were attracting the most attention within the scientific community.

From an overall perspective, it has been suggested that no specialty in medicine is any better than its published literature. ⁷⁵ The AEH would eventually become a leading journal for the publication of occupational hygiene and air pollution studies in the 20th century. ⁷⁶ Publication of results from the Community Health and Environmental Surveillance Studies (CHESS) during the 1970s was a significant contribution for US environmental standards, particularly in relation to air quality. ⁷⁷ The AEH would also become a major outlet for crucial air pollution studies during the early years of the new US Environmental Protection Agency (EPA). The 1973 article by Shy and colleagues, ⁷⁸ for example, provided important information regarding air pollution and its effects on US schoolchildren. Another series of studies published by Winkelstein and colleagues ^{79 , 80 , 81 , 82} also offered significant insight into particulate levels and their potential health effects among adult men. Evidence that air pollution was one of the AEH's niche topics can also be found in its citation counts, whereby the most highly cited article published between 1975 and 2004 described a study of daily mortality and PM₁₀ pollution in Utah Valley. ⁶⁶ Similarly, Pope's earlier (1991) article from the AEH's highly cited list also focused on air pollution. ⁶⁸

Regardless of the topic, citation lag times for the 5 highly cited articles clearly suggested that most citations were being received within the first 10 to 15 years, after which time progressively less citations were being attracted. This finding is consistent with previous research conducted in some other fields where it has been shown that the most-cited articles appear to reach their zenith 15 years after publication. ⁸³ Not all disciplines have such long lag time, however, with a study of ergonomics journals by Lee and colleagues ⁴⁶ finding that the peak citation rate occurred 4 years after publication. Either way, delays in receiving citations will always occur at any journal given the nature of the system itself, that is, it takes time after an article is published for its references to noticed, cited, tracked, and then recorded. ⁸⁴

The time taken for an article to receive half its total citations is generally known as the Cited Half Life, a concept originally based on radioactive decay. ⁸⁵ Cited Half Lives were not examined in the current study. The citation lag time of the AEH's most highly cited article between 1975 and 2004 (Pope et al ⁶⁶ ) was also interesting to consider, with around 5% of all its citations being received in the first 2 years following publication, slightly less than 30% being received within 5 years and around 70% after 10 years. Pope and colleagues’ 1992 article ⁶⁶ was not the journal's most highly cited paper of all time, however. The AEH's number one, all time, citation classic is in fact a 1966 article by Kasl and Cobb describing health, illness, and sick role behavior. ⁸⁶ By early 2009 this article had received over 350 citations.

Eugene Garfield coined the term “Citation Classic” in 1977 ⁵³ to denote articles that had attracted a higher than usual number of citations. ⁸⁷ Not all types of articles are cited at the same rate, however. ⁸⁸ Editorials tend to be less highly cited than the average source item, ⁸⁹ whereas laboratory science and methodological papers generally perform much better. ⁸⁷ The most highly cited article in science, a 1951 laboratory methods paper by Oliver Lowry, ⁹⁰ had been cited almost 300,000 times by 2005. ⁹¹ On the other hand, the discipline of EOH tends to attract considerably fewer citations. Gehanno and colleagues, for example, ⁶⁵ previously found that of the 15,000+ articles published in 5 major EOH journals since 1949, only 85 had been cited more than 11 times. Further investigation of the smaller journals in our field failed to find even a single article that had been cited over 100 times. ^{54 , 92 , 93} Citation rate is not necessarily related to research quality and not all articles with groundbreaking research are destined to become highly cited. Sabin's 1960 paper describing an oral polio vaccine, for example, ⁹⁴ despite saving at least half-a-million lives each year, ⁹⁵ had been cited fewer than 100 times by the late 1980s. ⁹⁶

Due to ongoing changes in the world of scientific literature, mainly that it is expanding, the definition of a citation “classic” has also changed over time as the average number of citations received per article increases. In 1965, for example, Price ⁹⁷ had suggested that an article cited 4 or more times per year might be considered a “classic.” Similarly, Garfield noted that the criteria for a citation classic had risen from around 250 citations in 1955 to over 1000 by 1995. ⁹⁸ Aside from article types, citations are not evenly distributed across all journals, and they tend to be clustered around a few core titles. A previous study in veterinary science, for example, ⁹⁹ demonstrated that less than 10% of the periodicals accounted for 66% of the total citations.

As the type of article published can often influence the number of citations that a journal will receive, an examination of this information is often useful in citation analysis. Between 1975 and 2004, the AEH published many articles of which Original Research comprised the vast majority, followed by Editorials, Letters-to-the-Editor, Corrections or Additions, Notes, Book Reviews, Proceedings, Meeting Abstracts, Review Articles, and Biographical Items. Interestingly, although review articles are generally known to attract a proportionately higher number of citations, ¹⁰⁰ none of the AEH's highly cited articles were reviews. An examination of the number of citable items published and the number of citations received each year suggested that the journal published a peak number of citable items in its first 2 years. It is difficult to know why this situation occurred, although it is possible that there was a healthy backlog of articles already waiting to be published as soon as the AEH was inaugurated.

An interesting finding in the current study was the lack of clear trends in immediacy indices over time, even though 2 definite peaks were detected in 1985 and 1986. These 2 peaks resulted from the fact that at least 6 articles ^{101 , 102 , 103 , 104 , 105 , 106} published in 1985 and 1986 were actually cited in the same year, a fairly rare occurrence. The citation patterns of these 6 articles were also interesting in themselves, given that none were destined to become citation classics. Of the 2 articles published back-to-back by Ujeno, for example, ^{106 , 107} only Part 1 ¹⁰⁶ was cited in the same year as it was published. On the other hand, at least 2 articles ^{102 , 104} attracted citations not only in the year they were first published, but also for many years after that. Immediacy indices represent the proportion of article published in a given year that are cited in that same year, ⁵⁰ and can be useful for indicating how “current” a journal's published research is. The lack of clear trends identified in the current study was not surprising, however, as other investigations have also failed to demonstrate clear longitudinal patterns in this regard. ⁴⁸ The Adjusted Age Distribution previously described by Asai ¹⁰⁸ may offer a means for modifying the immediacy index, in order to make the data it lists more usable.

An analysis of the AEH's impact factor, both in the short and long term, provided some interesting results. Firstly, the journal appeared to experience 2 separate peaks, one in 1977 and another in 1994. This result is not surprising as impact factors are not usually stable over time, even though in many scientific fields they appear to be steadily rising. ⁵⁹ Contemporary scholars are now devoting an increasing amount of research effort to investigate this phenomenon. Longitudinal trends have been documented in both individual journals and the disciplines they occupy, including public health, ¹⁰⁹ occupational health, ⁴⁸ occupational hygiene, ⁵⁹ anesthesia and critical care, ¹¹⁰ respiratory care, ¹¹¹ geriatrics, ¹¹² general medicine, ⁵⁶ and many others. Differences in impact factor scores are also known to exist even between allied fields. The impact factors of internal medicine journals, for example, appear to be increasing at a greater rate than those in public health. ¹⁰⁹ Overall, increases have also been seen at a larger scale. Between 1984 and 2004, for example, the maximum impact factor of scientific journals increased from 29.4 to 52.4. ¹¹³

Aside from rises, impact factors can also fall or even disappear altogether, ^{114 , 115 , 116} particularly when a journal changes its name and the old and new titles are not unified. ¹¹⁷ A longitudinal study of core journals in occupational health, for example, found that periodicals that changed their name often lost their impact factor for a few years. ⁴⁸ Given that journal impact factors also vary by country and language, ¹¹⁸ by scientific discipline, ¹¹⁹ and even between the journals of a particular field, ¹²⁰ the examination of average impact factors does offer an interesting insight into a journal's performance. An in-depth analysis of the AEH's average impact factor in the current study revealed that the mean score fluctuated from 0.856 between 1985 and 1989, to experience its peak of 1.622 between 1975 and 1979. The overall average impact factor score for the AEH was 1.240. Impact factor fluctuations of this nature are known to occur within the scientific literature, and increasing number of studies have begun to quantify the progression of this particular bibliometric measure.

Although it is certainly tempting to take bibliometric data at face value, any examination of journal impact factors should still be undertaken with caution. Indeed, Garfield originally stated that, like nuclear energy, the impact factor itself can be somewhat of a mixed blessing. ¹²¹ This measure has risen from relatively obscure beginnings to being an obsession for some journal editors, ¹²² to the point where citations are now seen as the “currency” of modern science. ¹¹⁶ High scores are presumably attractive for potential authors and thus, good for a journal's business, thereby leading publishers to use impact factors for marketing. ¹²³ Attracting a continuous supply of high-quality articles remains challenging for most periodicals, particularly those in the smaller medical subdisciplines, ¹²⁴ as authors often prefer to send their articles to more prestigious titles. ¹²⁵ Indeed, studies have shown that many researchers will continue to submit their work to the general medical journals, ¹²⁶ with higher impact factors. In occupational health, for example, McCunney and Harzbecker found that occupational medicine journals were 50 times more likely to cite the general medicine literature, than vice versa. ⁴³ This makes it increasingly difficult for smaller journals with relatively lower scores to attract a continuous supply of quality material. It has also been suggested that the citation indexing system is often biased in favor of “popular authors,” ¹²⁷ and that a journal's impact factor may be reduced if it does not conform to current “customs, fads, and fashions.” ¹²⁸ This may relate to the so-called Cumulative Advantage Distribution, whereby success breeds success. ¹²⁹

As a result, many scholars have begun to question the relevance of journal impact factors, and also the reliability of the calculations upon which they are based. This is particularly concerning because the impact factor exhibits certain design flaws that permit a certain degree of artificial inflation. ¹³⁰ One of the first such instances of attempted artificial inflation was reported in the late 1990s, ¹³¹ and by the end of the 20th century, various techniques commonly used for doing so had been recognized. ¹³² One common method is to carefully control the number and category of articles being published. Review articles, for example, are known to influence journal impact factors, given that they are more likely to be cited than original research. A previous study of pathology journals, for example, ¹³³ found that the inclusion of reviews increased the impact factor by 0.61. On the other hand, the 2-year citation counting window tends to favor “fast-moving” and basic biomedical disciplines. ¹⁰⁹ This means that faster-moving disciplines will be naturally advantaged, and so too, will articles that are more likely to be cited quickly. As such, although the impact factor has definite advantages in that it is somewhat tangible and reproducible, it is still not perfect and should not be taken at face value. ¹³⁴

Concerns regarding the methodology have led to various modifications for both the impact factor specifically, and citation indexing, generally. In 1999, for example, Takahashi and colleagues suggested the concept of a “topic-based” impact factor, ¹³⁵ a proposal which attracted the attention of Eugene Garfield himself. ¹³⁶ The idea would later be examined in more detail by others. ¹³⁷ In 2003, Huth ¹³⁸ proposed the idea of a “scope-adjusted” impact factor, whereby a journal's current impact factor would be divided by the number of journals citing its papers and then multiplied by 1000. Perhaps the most famous contemporary alternative is the H-Index, an index to quantify an individual's scientific research output first proposed by Hirsch in 2005. ¹³⁹ In Hirsch's proposal, a scientist would have an index of h, if h of his or her articles had received at least h number of citations each. Various other authors have since investigated relative merits of the H-Index, ^{140 , 141 , 142} and an H-type index has also been recently proposed for journals. ¹⁴³

At a broader level, various studies have looked at methods for improving information retrieval from the large bibliographic databases such as MEDLINE. ^{144 , 145 , 146 , 147 , 148 , 149 , 150 , 151} Citation ranking has been compared to peer evaluation, ¹⁵² whereas methods for ranking research productivity have also been described. ¹⁵³ The potential use of alternative measures of journal utility have been suggested, including Internet-based journal sessions and article download counts. ¹⁵⁴ Methods for refining the analysis in bibliometrics have also been attracting an increasing amount of attention among bibliometricians in recent years. Solari and Magri, ¹⁵⁵ for example, have proposed a new approach to evaluating scientific journals by using a box-plot of the frequency distributions. It is the future development and refinement of bibliometric techniques such as these that will enable scientists to more clearly establish accurate trends and patterns within the literature of their chosen field.

CONCLUSION

Overall, as this study has shown, citation analysis can provide an interesting look at the development of a journal over time. The examination of what articles, themes, and topics were being published, cited, or ignored also offers unique insight into the direction of not only a particular journal, but also the discipline in which it exists. In the current investigation, citation analysis and the identification of highly cited articles confirmed the importance of air pollution studies published by the AEH. It is well known that the journal formed an important part of knowledge dissemination in this particular area, and would be influential in the development of many EPA and air pollution standards in the US. High citation counts for air pollution articles well into the 1990s, also suggests that this topic continued to form an important part of the AEH's publication legacy, long after the original CHESS studies were released.

Acknowledgments

Derek R. Smith is Professor of Environmental and Occupational Health, and Director of the WorkCover NSW Research Centre of Excellence, University of Newcastle, Ourimbah, New South Wales, Australia.