Should low-quality evidence dominate high-level evidence? A systematic review and meta-analysis of systematic reviews of musculoskeletal physical therapy interventions

ABSTRACT

Objectives

To determine the: 1) quality of articles cited in systematic reviews (SRs); 2) methodological quality of the SRs; and 3) impact of quality on level 1A evidence.

Methods

SRs related to musculoskeletal physical therapy interventions were identified. The methodological quality of the SRs and articles cited by the SRs were assessed by two blinded reviewers. Data analysis was performed by a third blinded researcher. Additional comparisons were made based on the Journal Impact Factor, spin, financial bias, and conflict of interest.

Results

Twenty-four SRs were identified; 21/24 SRs had ‘critically low’ quality on the AMSTAR 2. Thirty-four percent of included studies were ‘low quality,’ and 58% of SRs included studies that had unreported external validity. One-half of the SRs represented ‘spin,’ and one-third of the SRs generated conclusions based on low-quality clinical trials.

Discussion

The ‘critically low’ SRs methodological quality was exacerbated by low-quality research inclusion. Most SRs failed to follow best practices, including prospective registration and integration of professional librarians in the search process. Based on the high proportion of SRs that include low-quality trials and overall low methodological quality, further discussion regarding practice recommendations on level vs. quality of evidence is warranted.

Level of Evidence

1a

KEYWORDS:

• Journal impact factor
• physical therapists
• publications
• randomized controlled trials as topic
• bias

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

No funding was provided for the development of this manuscript. I certify that neither I nor the co-investigators on this research project have any conflicts of interest related to this work.

Notes on contributors

Sean Riley

Sean P. Riley, PT, DPT, ScD is an Assistant Professor in the Physical Therapy program at Sacred Heart University in Fairfield, CT, USA and faculty in the Sacred Heart University Orthopedic Physical Therapy Residency program. He graduated from the UConn Physical Therapy Program in 2004 and completed his Doctorate in Physical Therapy at Simmons College in 2006. He completed his Doctor of Science from Texas Tech University Health Science Center in 2013. Sean became board certified in Orthopedics through the American Board of Physical Therapy Specialties in 2007 and completed his sports certification in 2009. He completed his Fellowship in Manual Therapy from Regis University in 2018.

Brian T. Swanson

Brian T. Swanson, PT, DSc is an Assistant Professor of Physical Therapy in the Department of Rehabilitation Sciences at The University of Hartford in West Hartford, CT, USA and faculty in the Select Medical Orthopedic Physical Therapy Residency program. Dr. Swanson graduated from Quinnipiac University with a Bachelor of Science degree in Physical Therapy in 1996 and earned a Doctor of Science in Physical Therapy from Andrews University in 2012. Brian became board certified in Orthopedics through the American Board of Physical Therapy Specialties in 2005. He completed a two-year residency and Fellowship in the Kaltenborn-Evjenth Norwegian Approach to Orthopaedic Manual Physical Therapy at the Institute of Orthopaedic Manual Therapy, Woburn, MA in 2006.

Steven F. Sawyer

Steven F. Sawyer, PT, PhD, is professor and chair of the Department of Rehabilitation Sciences, Associate Dean for Faculty Development, and director of the Clinical Postural Control Research Laboratory in the Center for Rehabilitation Research, all within the School of Health Professions at Texas Tech University Health Sciences Center. His PhD in Neuroscience was earned from the University of California at San Diego in 1989. He graduated from PT school at the Texas Tech University Health Sciences Center in 1997 and has been on the faculty since then. He received certification in Vestibular Rehabilitation from Emory University in 2001.

Jean-Michel Brismée

Jean-Michel Brismée, PT, ScD is a professor in the Doctor of Science Program in Physical Therapy at Texas Tech University Health Sciences Center in Lubbock, Texas. Dr. Brismée graduated from the Catholic University of Louvain-la-Neuve in Belgium with Bachelor of Science degrees in Physical Education and Physical Therapy in 1985. He graduated from Texas Tech University in 1996 with a Master of Sciences in Sports Health and earned a Doctor of Science degree in Physical Therapy in 2003. He is the Chair of the Research Committee and Director of the Fellowship program in Orthopaedic Manual Therapy of the IAOM-US, Editor-in-Chief of the Journal of Manual and Manipulative Therapy and maintains clinical practical at University Medical Center in Lubbock, Texas.

Geoffrey Staysniak

Geoffrey Staysniak is Health Sciences Librarian at Sacred Heart University. He has a BA from Fairfield University, an MLIS from Simmons College.