Doctoral Dissertations in Chemistry and Physics: A Longitudinal Study

ABSTRACT

A bibliometric analysis was conducted over the bibliographies of chemistry and physics doctoral dissertations between 1970 and 2020 to ascertain what types of documents are cited, average ages of cited references and overall changes of dissertation references that have occurred with the advent of electronic resources. The study reviewed 50 dissertations for each discipline with 10 dissertations from each decade (n = 100). Results measured 12,065 total references and found that the median age of journals increased in both chemistry and physics, doubling from 2010 to 2020. Only a few dissertation studies have been done on a longitudinal base to evaluate changes in publication trends. Similarities between the two disciplines with increases in the percentages of journal articles and declines in the number of books, other dissertations, and material types are calculated.

KEYWORDS:

• Chemistry
• physics
• dissertations
• citation analysis
• bibliometrics

Introduction

Line and Sandison (1974) distinguished synchronous studies that analyze the reference ages from source literature and diachronous studies that analyze citing literature. Median is often used to measure age in synchronous studies such as in this review (Line and Sandison 1974; Ortega 2008). Bibliometric techniques often use citation analysis, which involves reviewing the type of cited material, age, and calculated “obsolesce” of citations. Quantitative metrics such as number of ties to special journals, publishers, number of authors, countries, or other affiliations can also be measured.

Smith (1981) made distinctions between the terms “references” and “citations” for the purposes of citation analysis as “a reference is the acknowledgment that one document gives to another; a citation is the acknowledgment that one document receives from another.” By this definition, this study reviewed references and that terminology is used to refer to the data used, and citation analysis is used to describe the analysis process (Ortega 2008; Smith 1981).

Historically one of the first citation analyses was conducted by Gross and Gross (1927) when they examined the references in articles published in the 1927 Journal of the American Chemical Society in order to determine prominent science journals to determine best practices in core science library collections.

One of the first dissertation citation analysis was done in 1981, when McCain and Bobick first reviewed doctoral dissertations and preliminary dissertations to assess 1975–1977 biology library journal usage for library collection and assessment purposes. Doctoral dissertations are single-authored monographs generally held to be a significant penultimate document of a Ph.D. program. Most are written over a 3-to-4-year period. Often chapters or parts of a dissertation become a part of a peer reviewed journal article and, ironically, the articles may be published with coauthors including the faculty advisor and/or others working in a research group (Evans et al. 2018).

The University of Illinois at Urbana-Champaign (UIUC) is a Carnegie R1 university with over 34,000 undergraduates, over 17,000 graduate students, and 1,900 tenure-system faculty, offering degrees in over 150 programs. In 2019, UIUC awarded almost 14,000 degrees, including 874 PhDs. The UIUC Library supports this wide variety of instructional and research programs with comprehensive journal subscriptions from all major commercial and professional society publishers. The department of Physics is one of the top ranked physics programs in the United States with leading programs in Condensed Matter Physics and High Energy Physics (HEP). Physics 55 faculties; 33 post docs; 120 graduate students (teaching and research assistants) each year; chemistry 30 tenure track faculty; 28 post docs and around 50 research and graduate assistants each year. Chemistry has adjunct faculty and lecturers teaching lower-level undergraduate courses, whereas physics often has teaching assistants for courses.

Likewise, the School of Chemical Sciences at UIUC is in the top 10 chemistry programs in the United States, according to the US News and World Reports Ranking (https://www.usnews.com/best-graduate-schools/top-science-schools/chemistry-rankings?_sort=rank-asc).

There have been many bibliometric studies determining core journals from dissertation analysis in most science disciplines. Identifying these core journals has been an important aspect of collection development. While there has been a surge in new journal titles, both in traditional subscription basis and new open access publishing models, the highest impact and most cited journals remain on core journal lists. Since many studies duplicate those results, this study focuses on a few different objectives:

• What types of resources are being used in addition to journals and books?

• What are the ages of materials being cited and if there are different ages for various document types?

• Is the age of references increasing or decreasing particularly with more and older online content being made available?

• Determining how online references can be better identified as library usage.

Literature review

Many analyses on theses and dissertation bibliographies have been performed, primarily as a collection development tool in academic libraries. The ideology is that what graduate students are referencing in theses and dissertations is a good indicator of the current research areas at academic institutions and that it is useful to determine whether libraries are purchasing the books, journals, and other materials that support those dissertation areas. Some studies focus on particular material types such as books (Flaxbart 2018; Franks and Dotson 2017; Phillips 2018), government documents (Barnes 2006), and open access journals (Wani and Wani 2019). Many focus on specific disciplines within certain time periods (Flynn 2020; Kaczor 2014; Rose-Wiles 2022; Vallmitjana and Sabaté 2008; Zhang 2013). Many of them look across disciplines or areas of study (Kayongo and Helm 2012; Kushkowski et al. 2003; Zipp 1996).

Barnes (2006) examined doctoral dissertations at Mississippi State University for references to federal government publications (n = 707) between 2000 and 2004. Their College of Arts and Sciences had the highest number of references (3.3%) and how the transition from print documents to online formats increased in use but indicated a decline of references to government documents. Barnes suggested that some students may have reviewed government technical reports but did not cite them.

Sinn (2005) reviewed 67 math and statistics dissertations between 1980 and 2002 with 2145 references. Journals resulted in 65.9% of the total references with 27% book references. The average ages for the document types were journals at 16.6 years and books at 14.5 years. Technical reports were referenced 57 times and preprints had 55 references, measuring just over 2% each.

A later study at the University of Albany examined a math and statistics dissertation citation analysis from 2009 to 2019 in order to find resource types that graduate students are using, variation of journal titles as compared with other STEM disciplines, references to advisors’ publications, age of references, and publishers with highest reference counts. This article found similar metrics to previous analysis and revealed the importance that the arXiv preprint server has had for mathematics, physics, and other topics covered (Flynn 2020).

Zhang compared chemistry and chemical engineering dissertations between 2002 and 2011 at Mississippi State and found 87.1% journal article references and 7.1 books from 3,886 chemistry references. Chemical engineering was a bit surprising at 71.4% articles and 13.2% book from a total 2768 references.

Zipp (1996) conducted an early study that looked at some previous citation analysis in biology at Temple University (McCain and Bobick 1981) and geology at both UCLA and Stanford (Noga et al. 1994). Both studies also reviewed journal and book references in faculty publications in order to compare to dissertations references to identify highly cited library materials. Zipp utilized the Kendal τ test as opposed to Pearson r-index due to sample size greater than 30 and since the Pearson value can fall farther from zero. Zipp found that the top 40 journal titles cited by graduate students in their dissertations predicted the same top 40 titles cited by faculty publications.

Cole et al. (2018) looked at dissertations written in 2016 from nine engineering disciplines, using ProQuest Dissertation and Theses Abstract and Index (PDTAI) dissertations that had bibliographies that could be downloaded separately from the main text and using R they extracted 20 references from each decade from 1950 to 2017. They then searched 1260 references to determine how discoverable each resource was in Google Scholar, Scopus, and Compendex indexing services. References were categorized as findable in full record with abstract, reference only, or not easily discovered. All subject areas had a high number of journal articles, followed by books ranked second until 1980, when conference proceedings began to outnumber books. This was consistent with this study.

Flaxbart’ s (2018) analysis of books cited in chemistry dissertations found that references to books had decreased over his study between 1988 and 2015. He found that the earlier 1988 dissertations had the highest references (10.4%) to books. Overall, 5.4% of the book references with a median age of 11 years. The cumulative percentage of the UT-Austin books referenced by age resulted in a Pareto distribution of 21 years or less.

Eckel (2009) compared engineering master’s theses and doctoral dissertation references, and found greater exhaustiveness in the number of references, chronological age, and document type of references in dissertations. His study cited fewer scholarly resources such as academic journals than other dissertation studies of 44.3%. Eckel found that the references in engineering master’s theses tended to be more focused on web resources, gray literature, trade journals, and other resources easily retrieved through search engines. Masters’ students did cite monographs marginally more than doctoral references and were attributed to a need for more foundational understanding.

Becker and Chiware also found the same results as Eckel in that more engineering master’s theses were citing books than journals when compared to doctoral dissertations. Their study was over 100 dissertations published between 2005 and 2014 with a total of 7947 references (Becker and Chiware 2015).

Atmospheric science dissertations published between 2000 and 2010 were reviewed by Kaczor in 2014. Her results were in line with many other physical science areas of 85.1% of journal articles and 7.6% books from a total of 3298 references.

Dotson and Franks (2015) looked at research references in civil engineering, computer science, math, and physics dissertations over a 15-year period (1998–2012). For the 133 physics dissertations, articles were ranked top at 75.3%. Interestingly, technical reports and books fluctuated in second place over the course of their study. Overall, books were 7.8% and technical reports at 4.4% of the evaluated references. Surprisingly, preprints (such as arXiv) were ranked eighth during most years of their study (Dotson and Franks 2015).

Franks and Dotson (2017) followed up their 2015 citation analysis to specifically evaluate book references and the type of publishers (commercial, academic, society, government, etc.) Dotson and Franks 2017 examined books cited in 2003 to 2012 dissertations at Ohio State University and resulted in 73% of books cited were from the major scientific publishers, irrespective of format or age. Commercial publishers ranked at the top 73.17% with various commercial publishers ranked at the top by the four disciplines (civil engineering, computer science, math, and physics). The study did not account for age or format of the books. At the time of their article, they noted that e-book packages were mainly older content, which many libraries would already own books in print format. Theirconclusion that 50% or more dissertation book references were from seven main publishers: Wiley, Elsevier, Springer, Cambridge University Press, Taylor & Francis, AMS, and Pearson, and many libraries could assert that these e-book platforms would be sufficient for their respective user groups.

The Kushkowski et al.’s (2003) study had a longitudinal period to evaluate changes in publications and also analyzed both Master’s and Doctoral theses written between 1973 and 1992 at Iowa State over many disciplines in order to compare references between the graduate-level work and determine whether or not the library owned the cited resources. A total of 629 theses were selected from the print collection and every fifth reference from the respective bibliography with a total of 9102 references. The subject areas included Arts and Humanities, Social Sciences, Biological Sciences, Engineering, and the Physical Sciences.

Kushkowski (2005) followed with a second study of web references in 2005 specifically looking at print and electronic dissertations in economics and whether there were differences in references to web content in the two formats. Utilizing a traditional print collection at Iowa State and the early established electronic theses and dissertation (ETD) collection at Virginia Technology University, he was able to ascertain that web references were low in both groups and that the increased use and accessibility of electronic dissertations had not increased their usage, and consequently, referencing in other dissertations.

Gooden’s (2001) analysis of 30 chemistry dissertations between 1996 and 2000 also indicated the highest usage of journals (85.5%) to monographs (8.4%). Her analysis uncovered that only 12 journal titles could cover 50% of journal references.

Anaehobi and Ofoedu (2021) carried out a chemistry dissertation analysis on 85 documents published between 2010 and 2019 with a total of 4730 references. Their findings revealed 2001 journal references resulting in 42.3% and 1365 book references at 28.86% of the total.

Rose-Wiles (2022) reviewed 34 chemistry dissertation references over a 10-year period (2008–2018) to compare to an earlier study of faculty references in American Chemical Society journals. Statistics on the average number of references cited, source types, and various age metrics including median years and percentages for references ≤5 years, ≤10 years, between 10 and 19 years and over 20 years were included. She found the median age of references have increased slightly from 11.9 years in 2008 to 13.4 years in 2018. A comparison to library holdings and Interlibrary loan (ILL) requests were also measured.

Appendix 1 provides a comparative table of dissertation studies in chemistry and/or physics with number of documents studied, citation totals, percentages of articles and books, and median or average ages.

Methodology

ProQuest Dissertation and Theses Abstracts and Index (PDTAI) was searched using the exact school’s name and department name and limited by each decade (e.g. 1970–1979, 1980–1989, etc.). For example, the search string was “sch.Exact(‘UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN’) AND dep(PHYSICS) AND pd(20100101–20201231)” where publication date (pd) is edited for each decade and dep is edited for department. Results were saved to the MyResearch option in ProQuest in separate discipline folders.

A random number generator was used to select 10 dissertations from each discipline for each decade (i.e., 1970s, 1980s, 1990s, etc.). The bibliographies were printed from the full-text format. In some cases, references were listed at the end of a chapter but in most cases, there was a complete bibliography at the end of the dissertation. Earlier dissertations from the 1970s often used the Chicago Manual of Style with multiple resources listed in a single numbered reference. All individual references were counted, deduped, and adjustments were made in the total reference count when Chicago style was used. Starting around 2008, many ProQuest dissertations offered separate Reference files, which enabled the references to be extracted into Excel or other software. Other bibliographic studies have extracted these files into Excel or other software for their citation analysis (Flynn 2020; Rose-Wiles 2022). For consistency with the review of earlier dissertation bibliographies, the option to download reference lists was not used. Bibliographies continued to be reviewed in their original format.

A Microsoft Excel worksheet was designed with author name, year of dissertation, total number of pages, and total number of references. Library staff reviewed each bibliography and counted each reference by year of publication of the journal articles, books, conference paper or proceeding, technical report, or “other” category. “Other” was counted for personal communications, catalogs, preprints, or software, and a brief note was made about which type. A “web resource” was added for dissertations for the decades beginning with the 1990–1999 decade. Notes on the organization affiliated with technical reports were made such as the federal agency: (Department of Energy, National Science Foundation), corporation, or academic institution. Some foreign agencies were sometimes cited, such as CERN for High Energy Physics reports.

An aggregate Excel worksheet to compile each decade of 10 dissertations by discipline was created to summarize counts and ages for journal articles, books, dissertations or theses, and conference documents (proceedings or papers). “Other” type references and technical reports were also compiled with short identifying notes. The oldest reference and material type were also indicated. Statistics on the total number, ages, average, median year, top 25% and 80% were generated for each worksheet.

Results

The total number and percentages of various material types of articles, books, dissertations, conference papers, technical reports, and miscellaneous “other” references by discipline are outlined in Table 1. After reviewing the summaries and finding the total number of conference proceedings, technical reports, and “other” references were minimal, most often less than 2%, it was decided to concentrate further analysis on articles, books, and dissertations.

Table 1. Chemistry and physics dissertation reference totals by material type.

	Chemistry		Physics		Combined Totals
Document Type	Number	Percentage	Number	Percentage
Articles	6518	89%	3806	80%	10,324
Books	588	8%	582	12%	1170
Dissertations	74	1%	117	2%	191
Conference Proc.	21	0%	53	1%	74
Technical Reports	18	0%	65	1%	83
Other*	87	1%	105	2%	192
Totals	7306	100%	4728	100%	12,034

*Personal communications; data, patents, processes, preprints (see text).

Percentages of journal articles, books, dissertations, and conference proceedings fluctuated over time. For example, chemistry journal percentages were at or above 90% during the 1970, 2000, and 2010-decade analysis. The lower 86% (1980) and 89% (1990) percentages lowered the overall chemistry journal percentage to 89%.

Similarly, physics has the lowest percentage, 75%, of journal references during the 1970 decade, but it steadily increased over time to over 91% during the 2010-decade to the overall 80% average.

The total number of pages, references, average number of references per page, and the standard deviation were varied between the two disciplines and often fluctuated over the five-decade period. Table 2 contains the chemistry dissertations which totaled 8772 pages and 7180 citations for an average of .819 references per page. Chemistry references ranged from 12 in 1980 to 398 in 1978. Page count ranged from 78 in 1970 to 519 also in 1978. The overall standard deviation for chemistry references is 15.63.

Table 2. Chemistry reference and pages by decade.

All Chem	Average References/Dissertation	Number of Reference Range	Standard Deviation	Average Page Count	Average References/Page
1970	154	28–409	102.33	191	0.81
1980	132	12–320	82.31	184	0.72
1990	125	40–293	70.47	172	0.73
2000	152	83–223	42.81	157	0.97
2010	168	94–316	58.75	172	0.98

Physics dissertations, shown in Table 3, totaled 7076 pages and 4505 references for an average of 1.569 references per page. Physics had a minimum of 19 references in a 1987 dissertation to a maximum of 285 references in 2015. Individual dissertation total page counts range of 63 pages in 2008 to 324 pages in 1974.

Table 3. Physics references and pages by decade.

All Physics	Average References/Dissertation	Number of Reference Range	Standard Deviation	Average Page Count	Average References/Page
1970	73	28–153	42.1	160	2.19
1980	78	19–116	29.2	143	1.84
1990	75	21–117	31.0	127	1.07
2000	87	36–143	28.6	128	1.42
2010	163	80–285	54.7	146	0.889

There is a slight increase in the average number of references per dissertation, despite a near consistent number of references during the 1980s and 1990s. However, the number of references nearly doubled from the 2000-decade dissertations to the 2010 dissertations in physics. Since the dissertations were randomly selected, it may be happenstance that those selected had more references. The overall standard deviation for physics average references was 34.18.

While many longer dissertations had a correspondingly high number of references, this was not always the case. Surprisingly, there were a few dissertations that had very few article references, and a majority were to monographs or monographic series.

Figure 1 reflects the percentages of the two top-tier journal articles and book references for chemistry in addition to dissertation percentages for all five decades. Chemistry journal article references have always been at least 86% and most decades were over 90%. The decades with dissertations that have a higher percentage of book references were in the 1980s and 1990sdecades. Other chemistry dissertations have always been at less than .02% of the total dissertations and 1970s physics dissertations in Figures 1 and 2.

Figure 1. Chemistry reference percentages for articles, books, and dissertations.

Figure 2. Physics reference percentages for articles, books, and dissertations.

Figure 2 is a similar graph of the same 3-tiered reference percentages, with journal articles steadily increasing from 75% to 91%; whereas book references have steadily decreased over time from 19% to 8% over the five-decade period. Other dissertations referenced have also declined over time from .06 to .01.

Journal article references

As in all the other dissertation reference analyses reviewed, journal references are most numerous. The percentages of journal articles from various chemistry dissertation studies are between 79% and 95% (see Appendix 1). There are fewer physics studies indicating between 74% and 80% journal references, overall.

The Kushkowski et al. (2003) study of Physical Science dissertations indicated journal usage of over 80% during all four quartiles between 1973 and 1992. This was the same percentage of their study for the biological sciences. Other disciplines in Kushkowski’s review indicated ranges of 65% to 75% in engineering during the quartiles, social sciences 55% journal proportion, and a significant increase in the arts and humanities references to journals from just under 20% in 1973–1977 to 50% in the 1988–1992 quartile.

Other disciplines with dissertation analyses reflected the following percentiles for journal references:

Zhang’s (2013) chemical engineering: 71.4%; Kayongo and Helm (2012) Arts & Humanities: 23%, Engineering: 78%; Social Sciences: 46%. Eckel (2009) reviewed a variety of engineering doctoral dissertations resulting in 44.3% journal reference percentage. This was significantly more than the 29.3% found in the same study for Master’s theses journal references. Dotson and Franks (2015) results included civil engineering with 61.0%; computer science: 34.1% (second ranking after conferences at 41.8%) and mathematics with 58.1% journal references. Flynn’s Math and statistics review had 57% of journal references. And Sinn (2005), also, in math and statistics was 65.9%.

Monograph references

Phillips (2018) examined 916 science dissertations in seven disciplines over a ten-year period (2008–2017) at City University of New York. His focus was on commercial publishers and their marketing influence in dissertation research and references. Of the 144 chemistry dissertations, there were an average of 10.28 book references per dissertation. Physics averaged 8.59 books referenced per the 129 dissertations. The percentage of commercial publishers 83.6% in chemistry and 71% in physics.

Cole, et al. measured small numbers of book references in several engineering disciplines including no books in computer engineering, 12 references (mostly in the 1960s) in civil engineering; Electrical engineering cited books, including textbooks, but more frequently in earlier decades of their 1950–2017 study, and less than 20% for references to books in industrial engineering (Cole et al. 2018).

Flynn (2020) citation analysis of math and statistical dissertation between 2009 and 2019 indicated a 25.3% rate of books and 5.2% of book chapters. This coincided with other math dissertation citation analysis studies of 34% (Kayongo and Helm 2012), 27.1% (Dotson and Franks 2015), and math and statistics combined of 27% (Sinn 2005).

Franks and Dotson (2017) analysis of dissertation book references also did not indicate print or electronic access due to citation formats, i.e., no print-format references and/or Digital Object Identifiers (DOIs) were included. The inclusion of DOIs in book and/or book chapter references would help libraries be better able to scrutinize electronic usage statistics.

Flaxbart noted some barriers to books being cited in dissertation including discovery where books and book chapters were not generally indexed as well as journal articles. His review of chemistry dissertation book references at UT Austin had a downward trend. This has changed with Scopus and Google Books and publishers indexing chapters more robustly. As more libraries have implemented electronic first collection mandates, perhaps the references to books and chapters will increase over time. Flaxbart also noted that students may be reading books for the background information, but not citing them, or perhaps faculty advisors are not as supportive of book references in dissertations as journal articles (Flaxbart 2018).

This study’s citation analysis uncovered 49 out of 50 dissertations in both physics and chemistry, which had at least 1 book reference. There was a slight increase in chemistry book references during the 1980s and 1990s decades but decreased to an overall 6% during the 2010s (8% average over all decades).

Physics during the 1970s had the highest percentile of book references, 19%, but has decreased over the following decades to a low of 8% in 2010 with the median age of 13 years. UIUC started to license and purchase all Wiley, Springer, and Elsevier e-books (except textbooks) content around 2015. This default arrangement has enabled the library to provide these major commercial publishers’ content effortlessly and efficiently, so it is unfortunate that the book citation numbers have not increased or remained stable.

Dissertations and theses references

The use of other Master’s theses or Ph.D. dissertations have varied over the years. Overall references to dissertations were very minimal with 1% in chemistry and 3% in physics over the five decades. Often the dissertations were from other graduate students from UIUC. However, other times, they were to dissertations from other US academic institutions and occasionally from foreign schools. With accessible electronic access to dissertations through ProQuest Theses and Dissertation Abstract and Index (PTDAI) and the Networked Digital Library to Theses and Dissertations (https://ndltd.org/), and more schools putting digitized copies of their theses and dissertations into institutional repositories, the discovery and access has become much easier. However, it was the earlier decades that had more references to national and international dissertations.

Dotson and Franks (2015) also measured a decline in references to thesis and dissertations in their citation analysis, despite increases in electronic access to ETDs, ProQuest, and NDLTD. They suggested that often faculty advisors were aware of the research of the cited dissertation and provided access to dissertations and this would be a factor for earlier decades when finding theses and dissertations would have been more difficult. However, they also concluded that the decline could be the viewpoint that other graduate student work is less significant and/or faculty advisor influence or embargoes that deter access to the most current research (Dotson and Franks 2015).

Gooden’s (2001) chemistry dissertation citation analysis as the “other” category was the references to other dissertations or theses, but often had deficient information.

Conference papers or proceedings references

Many other citation analysis studies categorized conference papers in the “other” category. (Becker and Chiware 2015; Gooden 2001) Of course, other studies that looked at computer science dissertation references indicate higher percentages of conference proceeding references over journals and books (Dotson and Franks 2015; Kayongo and Helm 2012). Kayongo and Helm, whose citation analysis covered many disciplines, noted that conference proceedings are also cited heavily in Electrical Engineering and Economics (Kayongo and Helm 2012).

Zhang’s analysis found conference proceeding references in 2% of the chemistry dissertations with a total of 78 references over 10 years (Zhang 2013).

A 2014 atmospheric science study found a higher number of references to conference papers, along with book references, when compared to atmospheric science faculty publications. Kaczor attributed this to doctoral students’ foundational research (Kaczor 2014).

This study found higher references to conference proceedings during the 1970s for physics yet declining throughout the decades. For physics, there were a total of 53 conference references with the maximum age being 50-year difference between the dissertation publication date and the year of the proceeding. The average age of conference papers was 9.61 years, median age was 5 years, top 25% cited were at 3 years, and 80% were 13 years.

Chemistry dissertations in this analysis had a result of 21 conference references. The maximum age was 22 years and the average age was 2.75 years with a median of 3 years. The top 25% were published within 1 year of the dissertation and 80% were by 5.2 years.

Technical reports references

As Barnes noted in the citation analysis of the US Government documents, the migration of technical and government reports to online environment from print may have resulted in students utilizing the online resources but not citing them in their dissertations (Barnes 2006).

Dotson and Franks (2015) review of physics dissertations included technical reports as the 2^nd or 3^rd tier category in eight of their 15-year study with an overall 4.4%. This percentage combined with peer-reviewed articles (75.3%) and books (7.8%) came to 87.5%, leaving 12.5% with items, such as conferences, preprints, thesis, and web pages. Their average item count graph is very similar to the physics percentage graph for this study in Table 5.

Table 4. Aging for articles-books-dissertation chemistry references.

Document Type	Average Age	Median Age	25% Age	80% Age	Oldest Year
Articles	19.2	15.2	5	34.4	1783
Books	12.6	10	3	23	1883
Dissertations	6.4	4	1	9	1873

Table 5. Aging for articles-books-dissertation physics references.

Document Type	Average Age	Median Age	25% Age	80% Age	Oldest Year
Articles	15.4	13	3.4	31	1866
Books	11	13	2	21.6	1909
Dissertations	18	7	3.8	35.8	1892

Technical report usage was higher in the earlier decades of this study but declined steadily with reference averages over the five decades at 1.93 for physics and .24 for chemistry. Most reports cited in physics dissertations were from the National Bureau of Standards (NBS), Rand Corporation, and labs associated with the US Department of Energy (DoE). A few were from academic institutions such as Stanford and MIT, and two were from CERN. A total of 65 technical reports were referenced in physics, with the oldest being from 1957. The average age in comparison to the published dissertation year was 9.67 with a median age of 6 years, the top 25% of technical reports cited in physics was within 3 years and 80% at 14 years.

In the chemistry dissertations in this study, reports were much less frequent with total of 18 technical report references. With an average age of 5.44-year difference from the dissertation date, the oldest from 1962. The median age was 8.5 years, top 25% within 1 year, and 80% at 10.8 years. Chemistry technical reports were mainly industry-related such as Aldrich Chemical and IBM, but a few from DoE National laboratories.

“Other” category references

Many citation analysis studies include an “other” category but reference and/or material types are variable. For example, Gooden grouped many various cited material types including technical reports, conference papers, personal communications, and patents together as “other,” which resulted in a large percentile falling into that category.

This study categorized personal communications, websites, manuals, software, equipment, processes (mainly in chemistry), patents, preprints, and websites. There were some “one-offs” including a reference to the 1922 documentary film “Nanook of the North” from a 2010-decade physics dissertation. A drug-related press release was also cited in a 1990s chemistry dissertation. The most numerous “other” references in physics dissertations were 40 personal communications, 19 arXiv preprints, 10 web references (including Wikipedia entries), and 8 types of equipment utilized.

Chemistry dissertations cited 29 personal communications, 18 processes, 15 patents, 10 softwares, and 4 data resources. In both disciplines, the references to personal communications were more numerous in the early decades. None of them noted that the communications were in e-mail or written format.

Unlike other dissertation citation analyses, this study found no references to technical or industry standards.

Other notable findings included:

References to Wikipedia or other non-peer reviewed resources. Although only three times in only one dissertation, it was surprising – and disappointing – to see this instead of citation to reference resources such as a scientific dictionary, handbook, or encyclopedia. Reference books would certainly have been better cited than the Wikipedia references found in some other dissertation bibliographies as peer reviewed general sources.

Reference age

Eckel's review of engineering doctoral dissertations ranged in number of years for various material types. Conference proceedings’ average age of 9.2 years and 42% being less than 5 years are comparable to this study, albeit in engineering disciplines. Likewise, scholarly journals were 16.6 years, with 22% being less than 5 years.

Kushkowski et al. (2003) study indicated a small increase (approximately 7 years to 12 years) in the average age of physical science references over the two decades between 1973 and 1992. Their study divided the years into 5-year increments and indicated a decrease in average reference age for engineering and social science dissertations.

Kaczor's atmospheric science dissertations discovered an average of 9-year journal age for 52.6% of the references, and 31 years for 90%. This result was slightly “younger” than the physics ages in this study.

Zhang’s (2013) study in chemistry had consistent findings with 53.4% of articles, books, and conference proceeding references being 10 years or less. That study found book references that were over 20 years comprising 28%, whereas journal articles (31.4%) and conference proceedings (30.8%) less than 5 years were the highest percentage. These results are like Ortega’s analysis of reference age in chemistry journals in which 25% of article references were less than 5 years and 90% were 26 years or less.

Rose-Wiles (2022) study of chemistry dissertation study over a 10-year period and had a median age of references range from 9.0 (2011) to 14.8 (2009) with a resulting median age of 11.8 years and a slightly older 13.2 years for books.

This study found that 46 journal articles and 4 books were the oldest references in chemistry across all decades with 1783 journal articles from Proceedings of the Royal Society referenced in a 2010 dissertation.

Surprisingly, there was one 1994 physics dissertation with the oldest reference to both a journal article and book published in 1953. Otherwise, the totals were similar with 40 oldest references to journal articles and 8 to books.

Figures 3 and Figures4 present the various ages (minimum, 25%, median, 80%, and maximum) of article references in chemistry and physics dissertations.

Figure 3. Chemistry journal minimum, 25%, median, 80% and maximum reference ages.

Figure 4. Physics journal minimum, 25%, median, 80% and maximum reference ages.

Almost every dissertation has at least one article reference that had been published in the same or within the prior year. This has increased to 2 years (Physics in 2000 and Chemistry in 2010). However, the top 25% references have also increased to 6 years in both disciplines.

In both disciplines, the median age had increased significantly for journal articles and increased from approximately 10 years to 20 over the five-decade period. Both disciplines have shown similar increases in the 80% and maximum age growth. In the 2010 decade, both disciplines had maximum age outliers with chemistry at 97 years and the physics outlier of 85 years.

Tables 4 and 5 reflect the age averages for articles, books, and dissertations. Despite the two different disciplines, it is interesting to see the similarities of ages of journal article and book references. The various ages of referenced books differed with chemistry increasing slightly.

In physics, the average and 80% ages to other dissertations were surprisingly much older than expected, with an average of 18 years, but the top 25% were less than 4 years. Both disciplines, however, referenced dissertations from the late nineteenth century.

The arXiv preprint references in this study were an average of 3 years. This is less than the 8.4-year average found in Flynn’s math and statistics dissertation arXiv references. The oldest arXiv preprint cited in Flynn’s study was 27 years, and this study was 4 years. As Flynn reflects, since many arXiv preprints are often published in peer-reviewed journals within 2.5 years, it is surprising to see older arXiv preprints referenced in lieu of the journal articles. Flynn indicates that it may be attributed to the faculty advisor provided the arXiv version or that it was not eventually published in a journal (Flynn 2020).

Conclusions and future study

This local dissertation study has shown that chemistry and physics graduate students reference a wide range of document types with a reliance on the most recent materials. Both disciplines primarily reference journal articles but also rely on foundational monographic literature. For both chemistry and physics dissertations, the overall numbers of references have significantly increased during the most recent decade and the age of references has also doubled. This has increased the median age of referenced journal articles in physics from 10 to 20 years and chemistry from 11 to 20 years as well.

For the most part, digital object identifiers (DOIs) are not utilized in dissertations written after 2000. This could be because they are not required by departments or graduate colleges for dissertation submission. It would be good if this changed since many federal funding agencies have started to require them for grant applications. They would also be useful to help librarians determine when online resources were used for collection development, bibliometric studies, and other usage statistics. Several articles noted that references did not indicate electronic formatted materials (Gooden 2001; Kushkowski 2005). Reviewing more current, additional dissertations in the local ETD system found that some dissertations did include DOIs; however, many of them were inconsistent in that not all references included the DOIs, especially for older references. This could be that the students are finding the references through other bibliographies and DOIs were not included and that often locating retrospective DOIs to references is often difficult to locate. Gooden also noted that users prefer online access to materials, yet less than 1% of the references she reviewed referred to the online version. She attributes this to students being unaware of electronic formats in citing style guides. In Kushkowski et al. (2003) study, they commented that electronic content may be shrinking usage reliance on printed materials, which has undeniably happened, but it would be advantageous for libraries to measure electronic library usage via student dissertation bibliographies for both collection development and awareness of research focuses. Kushkowski also questions the increase in electronic content and how changes in research methodologies could be affecting reference inclusion.

This study confirms that doctoral students are often citing surprisingly older materials and sometimes unexpected content. The outlying maximum age of recently referenced journal articles and books published in the nineteenth century are still being cited and should indicate that collections are best evaluated by both obsolesce age and circulation and usage review. However, the number of references to monographs and dissertations decrease with median age decreasing in both disciplines.

Acknowledgments

The author would like to thank Jeff Loftiss, staff member in the Chemistry Library for his work on this study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Appendix 1:

Comparison of other Chemistry and Physics Dissertation Reference Studies

Table

Author/Year	Period/Type	Discipline	# Diss	# Cites	% Articles	% Books	Age (M) Median (A) Average
Flaxbart (2018)	1988–2015 Books	Chemistry	50	Books 474 Mono Ser 123	n/a	5.4% 6.8%	11 years (M)
Phillips (2018)	2008–2017 Books	Chemistry Physics	144 129	1481 1108	n/a	Ave cites 10.28 8.59	no metrics
Kushkowski et al. (2003)	1973–1992	Physical Sciences	66	*1250	*85%	*17%	All Disc. 12 years (A) 8 years (M)
Gooden (2001)	1996–2000	Chemistry	30	3704	85.8%	8.4%	no metrics
Vallmitjana and Sabaté (2008)	1995–2003	Chemistry	46	4203	79%	12%	Art 14 years (A) 9 years (M)
Zhang (2013)	2002–2011	Chemistry	24	3886	87.1%	7.1%	no metrics
Kayongo and Helm (2012)	2005–2007	Chemistry Physics	11 22	2015 2125	95% 74%	3% 13%	10.2 (A) 16.1 (A)
Rose-Wiles (2022)	2008–2018	Chemistry	34	4637	87%	9%	Art 11.84 (M) Books 13.2 (M)
Schlembach (2023) (This Study)	1970–2020	Chemistry Physics	50 50	7306 4728	89% 80%	8% 12%	Chem Art. 15.2 Chem Bks 10.6 Phys Art & Bks 13