BioSocial Health J. 1(3):115-126.
doi: 10.34172/bshj.12
Review Article
Quality of the systematic reviews in cochrane multiple sclerosis related articles
Masoud Zeynalzadeh Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Project administration, Writing – original draft, 1
Nasim Mahdavi Investigation, Methodology, Validation, Visualization, Writing – original draft, 1
Morteza Atayi Investigation, Methodology, Writing – original draft, 1
Hanieh Salehi-Pourmehr Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Resources, Software, Supervision, Validation, Writing – review & editing, 1, 2, *
Sakineh Hajebrahimi Conceptualization, Project administration, Resources, Supervision, Writing – review & editing, 1, 3
Author information:
1Research Center for Evidence-Based Medicine, Iranian EBM Centre: A JBI Centre of Excellence, Faculty of Medicine, Tabriz University of Medical Sciences, Tabriz, Iran
2Medical Philosophy and History Research Center, Tabriz University of Medical Sciences, Tabriz, Iran
3Department of Urology, Tabriz University of Medical Sciences, Tabriz, Iran
Abstract
Introduction:
To enhance the assessment of the systematic reviews and meta-analyses performed by the Cochrane Multiple Sclerosis (MS) Group.
Methods:
Our study was conducted on 57 systematic reviews and meta-analyses related to MS, published by the Cochrane database until July 2023.
Results:
We found that the most encountered risk of bias was the low-risk domain, associated with Selective Reporting (data reporting), and followed by an unclear outcome for Allocation Concealment (selection bias). In contrast, Blinding of Participants and Personnel (performance bias) showed the highest risk of bias. Also, we concluded that up to 2015, the most prevalent risk of bias was ‘low outcome’ for Selective Reporting (data reporting). However, from 2016 till 2023, the most common risk of bias shifted to ‘low outcome’ for Random Sequence Generation (selection bias).
Conclusion:
Despite significant enhancements in improving the quality of studies, there is still a far way to achieve the ideal quality.
Keywords: Randomized controlled trials as topic, Bias, Systematic reviews as topic, Multiple sclerosis
Copyright and License Information
© 2024 The Author(s).
This is an open access article distributed under the terms of the Creative Commons Attribution License (
http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Funding Statement
This study was supported by the Research Center for Evidence-based Medicine and the Research Vice-chancellor of Tabriz University of Medical Sciences (Grant No 68272).
Introduction
In recent decades, the remarkable proliferation of journals and articles, considering the advancements in medical science, has brought the structure of articles and research methodology into sharper focus.1,2 It is clear that the quality of articles directly influences the quality of results; therefore, it is vital to adhere to the research principles. Substandard research can negatively impact healthcare quality, influencing public health policies and treatments in detrimental ways.1 Systematic reviews and meta-analyses, as the most reliable sources of information, play a pivotal role in synthesizing data from available evidence. The quality of these reviews is paramount, as they often guide clinical practice and policy.3 The methodological quality of randomized controlled trials (RCTs) included in systematic reviews can vary, considering the reliability of the review’s conclusions. The Cochrane Database of Systematic Reviews is a collection of high-quality, independent evidence, designed to inform healthcare decision-making. This database, which contains systematic reviews and meta-analyses of healthcare interventions, is widely recognized as a reliable source of current information on the effectiveness of healthcare treatments. Each review within the database, undergoes a rigorous editorial process to ensure its quality and relevance, making it an invaluable resource for healthcare professionals, researchers, and policymakers. This database comprises 53 review groups, each concentrating on a specific topic, including the Cochrane Multiple Sclerosis (MS) group.
MS, the most common non-traumatic disability in young adults, is not confined by geographical boundaries, and its prevalence is increasing in both developed and developing countries.4,5 This disease occurs more frequently in the age range of 20 to 45 years and is twice as common in women as in men.6-8 Factors such as genetics and environmental influences, including exposure to sunlight for vitamin D, ultraviolet radiation, the Epstein-Barr virus, obesity, and smoking, have a significant impact on patients with MS.9
For MS patients, where treatment decisions can profoundly affect the quality of life, the stakes are particularly high. Assessing the quality of studies and interventions becomes not just a matter of academic rigor but a necessity for ensuring patient safety and optimal outcomes. The “risk of bias”, “sample size”, and “blinding” are among the critical factors that determine the quality of a study. In this context, the article aims to explore the methodologies employed in assessing RCT quality within systematic reviews, with a focus on those of MS. It will delve into the challenges faced in this endeavor and propose strategies to overcome them, ultimately aiming to contribute to the enhancement of healthcare quality for MS patients.
Methods
This study was conducted on 57 articles published by the Cochrane Neurological Condition Group until July 2023. We searched the Cochrane database in July 2023 and included all systematic reviews and meta-analyses published up to that date. Studies involving animals and those that did not assess bias as per the Cochrane risk of bias tool were excluded. The Cochrane Library comprises databases that contain a wealth of high-quality, independent evidence. The Cochrane Neurology Group covers a range of topics including stroke, dementia and cognitive disorders, epilepsy, peripheral neuropathies, movement disorders, headache and migraine, cancers, motor neuron disease, neurodevelopmental disorders, neuromuscular junction disorders, spinal cord disorders, sleep disorders, and MS. As of the search date, it consisted of 1001 Cochrane reviews and 215 protocols. We accessed the Cochrane Library using a subscription managed by our organization and selected reviews on MS. Initially, we extracted general information from all studies, including topics, year of publication, author names, and other required information such as interventions, outcomes, and results.
We applied the Joanna Briggs Institute (JBI) Critical Appraisal tool, which contains 11 questions, to each Cochrane review to assess the risk of bias. The validity of the reviews was evaluated by two reviewers using standardized critical appraisal instruments from the JBI (JBI-MAStARI). Any disagreements were resolved through discussion, and if consensus could not be reached, a third assessor was consulted. The JBI is an international research organization specializing in evidence-based healthcare. It is renowned for promoting the synthesis, transfer, and utilization of evidence in healthcare. The Institute provides resources to help healthcare professionals integrate the best available evidence into their practice. The JBI critical appraisal tools used in this study are designed to help users assess the methodological quality of research studies, thereby determining the availability and reliability of the study results. These tools are particularly useful for researchers conducting systematic reviews or evidence synthesis. Each tool provides a checklist of specific criteria to be considered when evaluating a study, such as the appropriateness of the study design, the methods used for data collection and analysis, potential biases, and the relevance of the results. Responses to these criteria are “yes”, “no”, “unclear”, or “not applicable”. The PRISMA statement is a widely recognized set of guidelines for reporting systematic reviews and meta-analysis in health research. It helps authors improve the reporting of their results, thereby facilitating critical appraisal and interpretation.
We first assessed the included systematic reviews and meta-analyses through critical appraisal. Then, we extracted a collection of biases from all understudied RCTs in these systematic reviews, which were appraised by the authors of the systematic reviews using the Cochrane standard risk of bias tool. Finally, we extracted the results of the risk of bias assessment in each Cochrane review. The Cochrane Risk of bias tool is a checklist used to assess the risk of bias in clinical trials. It aids reviewers in evaluating the validity of included studies and is widely used in systematic reviews and meta-analyses. This tool includes several key domains: Selection, Performance, Detection, Attrition, Reporting, and other sources of bias. Each domain is evaluated to determine the potential risk of bias within the study. Reviewers assign a judgment of “Low risk”, “Unclear risk”, or “High risk” for each domain based on the information provided in the study. Descriptive statistics was used to analyze the data using SPSS software versions 16.
Results
A total of 57 systematic review articles and meta-analyses, encompassing a subset of 509 clinical trials, were studied and evaluated. The analysis of clinical trials included in systematic reviews related to MS Cochrane yielded the following results: Our primary objective was to assess the appropriateness of research questions in these trials. The analysis revealed that all systematic review studies from the Cochrane MS Group posed appropriate research questions. Table 1 provides details related to the objectives of these included studies.
Table 1.
Objectives and clinical questions of Cochrane systematic review studies
Study
|
Aim
|
Garegnani 202010 |
Comparing the effectiveness and adverse effects of common and complex shunt devices for CSF diversion in people with hydrocephalus |
Parks 202011 |
Evaluating the effects of dietary interventions (including dietary programs with recommendations for whole foods, coarse nutrients, and healthy natural products) compared to placebo or other interventions on health outcomes (including outcomes related to MS and serious side effects) in people with MS |
Hayes 201912 |
Evaluating the effectiveness of interventions designed to reduce falls in people with MS |
Latorraca 201913 |
To evaluate the effects (benefits and disadvantages) of palliative care interventions compared to usual care for people with any type of MS |
Jagannath 201914 |
Evaluation of the profit and safety of venous PTA in individuals with MS and CCSVI |
Amatya 201815 |
Investigating the effectiveness and safety of non-pharmacological treatments for managing chronic pain in MS |
Köpke 201816 |
Evaluation of the effectiveness of information provision interventions for people with MS, aimed at promoting informed choice and improving patient-related outcomes |
Jagannath 201817 |
Evaluation of the benefits and safety of Vitamin D supplement for reducing disease activity in people with MS |
Rietberg 201718 |
Investigating the effects of respiratory muscle training versus any other type of exercise or no exercise on respiratory muscle function, lung function, and clinical outcomes in people with MS |
Zhang 201719 |
To compare the effectiveness, tolerance, and safety of Alemtuzumab versus Interferon Beta-1a in treating people with RRMS to prevent disease activity |
Filippini 201720 |
1. Estimating the benefits and safety of disease-modifying drugs that have been evaluated in all studies (random or non-random) for the treatment of the first clinical attack indicative of MS compared to placebo or no treatment.
2. To evaluate the relative effectiveness and safety of disease-modifying drugs considering their benefits and safety.
3. Estimation of the benefits and safety of disease-modifying drugs that have been evaluated in all studies (random or non-random) for treatment initiated after the first attack ("primary treatment") compared to treatment initiated after the second attack or at another later time point ("delayed treatment"). |
La Mantia 201621 |
To evaluate whether Beta-IFNs and GA are different in terms of safety and effectiveness in treating people with Relapsing-Remitting MS (RRMS) or not. |
La Mantia 201622 |
To evaluate the safety and benefit of Fingolimod versus placebo, or other Disease-Modifying Drugs (DMDs), in reducing disease activity in people with Relapsing-Remitting Multiple Sclerosis (RRMS). |
He 201623 |
To evaluate the absolute and comparative effectiveness and safety of Teriflunomide as a monotherapy or combination therapy compared to placebo or other Disease-Modifying Drugs (DMDs) (Interferon Beta (IFNβ), Glatiramer Acetate, Natalizumab, Mitoxantrone, Fingolimod, Dimethyl Fumarate, Alemtuzumab) in the disease process of people with MS. |
Yang 201524 |
To evaluation of the efficacy and safety of sodium channel blockers for neuroprotection in individuals with multiple sclerosis (MS) to prevent disability occurrence and reduce disease burden. |
Tramacere 201525 |
To compare the benefits and acceptability of Interferon beta-b1, Interferon beta-a1, Glatiramer acetate, Natalizumab, Mitoxantrone, Fingolimod, Teriflunomide, Dimethyl Fumarate, Alemtuzumab, Pegylated Beta-interferon a1, Immunoglobulins for the treatment of people with RRMS and providing a ranking of these treatments according to the benefits and their acceptability as the proportion of participants who withdrew due to any adverse event. |
Heine 201526 |
To determine the effectiveness and safety of therapeutic exercise compared to control conditions without exercise or other interventions on fatigue, measured by self-reported questionnaires, in people with MS. |
Xu 201527 |
To evaluate the benefits and safety of Dimethyl Fumarate as monotherapy or combination therapy compared to placebo or other approved disease-modifying drugs (Interferon Beta, Glatiramer Acetate, Natalizumab, Mitoxantrone, Fingolimod, Teriflunomide, Alemtuzumab) for patients with MS. |
Khan 201528 |
Investigating the effectiveness and safety of remote rehabilitation intervention in MS for improving patient outcomes. |
Rosti-Otajärvi 201429 |
Evaluation of the effects of neuro-psychological rehabilitation on health-related factors, such as cognitive performance and emotional well-being in patients with MS. |
Xiao 201430 |
To evaluate the efficacy and safety of MMF for preventing disease activity in patients with RRMS. |
Liu 201231 |
To evaluate the safety of Daclizumab and its effectiveness in preventing clinical worsening in patients with RRMS. |
He 201332 |
To evaluate the absolute and comparative effectiveness, tolerability, and safety of pharmacological treatments for memory impairment in adults with MS. |
He 201333 |
The safety and efficacy of Rituximab, as monotherapy or combination therapy, were evaluated against placebo or approved disease-modifying drugs (DMDs) (interferon β-IFN, Glatiramer Acetate, Natalizumab, Mitoxantrone, Fingolimod, Teriflunomide, Dimethyl Fumarate, Alemtuzumab) for reducing disease activity in people with RRMS. |
He 201334 |
To evaluate the effectiveness and safety characteristics of Laquinimod as a monotherapy or combination therapy against placebo or approved DMDs (interferon beta, glatiramer acetate, natalizumab, mitoxantrone, fingolimod, teriflunomide, dimethyl fumarate) for modifying the course of disease in patients with MS. |
Filipini 201335 |
To estimate the relative effectiveness and acceptability of Interferon (b-1IFNß (b-1ß Betaseron), interferon (a-1IFNß (a-1ß Rebif and Avonex), Glatiramer Acetate, Natalizumab, Mitoxantrone, Methotrexate, Cyclophosphamide, Intrazavens, Avonex Immunoglobulins and long-term Corticosteroids against placebo or other active agent in participants with MS and provide a ranking of treatments based on effectiveness and risk-benefit balance. |
Martinelli Boneschi 201336 |
To evaluate the effectiveness and safety of MX compared to the control group in participants with Relapsing-Remitting MS (RRMS), Progressive-Relapsing MS (PRMS), and Secondary Progressive MS (SPMS). |
Amatya 201337 |
To evaluate the effectiveness of various non-pharmacological interventions for the treatment of spasticity in adults with MS. |
Burton 201238 |
Comparison of the effectiveness of oral and intravenous steroids in promoting disability recovery in MS relapses in six weeks or less. |
Tejani 201239 |
To evaluate whether the supplement Carnitine (oral or intravenous) can improve quality of life and reduce fatigue symptoms in patients suffering from MS-induced fatigue, and to identify any side effects of Carnitine when used for this purpose. |
Xiao 201240 |
To evaluate the effectiveness and safety of Sildenafil Citrate for ED in patients with MS. |
Sitjà Rabert 201241 |
To investigate the effectiveness of WBV (Whole Body Vibration) for improving functional performance with regard to daily basic life activities (ADL) in neurological diseases. |
La Mantia 201242 |
To investigate whether IFN therapy in secondary progressive multiple sclerosis (SPMS) is more effective than placebo in reducing the number of patients experiencing disability progression. |
Wang 201143 |
To evaluate the effectiveness and safety of statins that are prescribed either alone or as a complement to approved treatments for MS. |
Pucci 201144 |
To evaluate the effectiveness, tolerance, and safety of NTZ in treating patients with RRMS. |
Koch 201145 |
To investigate the effectiveness and tolerance of pharmacological treatments for depression in patients with MS. |
La Mantia 201046 |
To investigate the clinical effectiveness of Glatiramer Acetate in treating MS patients with relapsing-remitting (RR) and progressive (P) Multiple Sclerosis. |
Rose 201047 |
To evaluate the impact of interventions to reduce or eliminate ankle equinus in people with neuromuscular disease. |
Rojas 201048 |
To identify and summarize evidence of the usefulness and safety of Beta Interferon in patients with PPMS. |
Khan 200949 |
To evaluate the effectiveness of virtual reality programs compared to alternative programs or usual care in returning to work, efficiency, and employment in pwMS for evaluating the cost-effectiveness of these programs. |
Ciccone 200850 |
To determine the effectiveness and safety of long-term use of Corticosteroids in MS. |
Clerico 200851 |
To evaluate the effects of immunomodulatory drugs compared to placebo in adults to prevent the conversion of CIS to CDMS, which means preventing a second attack. |
Casetta 200752 |
Comparison of Azathioprine with placebo to determine the effect of Azathioprine on primary clinical outcomes, namely disability progression and recurrence in patients with MS. |
Khan 200753 |
To evaluate the effectiveness of structured MD rehabilitation in adults with MS. To discover effective rehabilitation approaches in different environments and the outcomes that are influenced. |
La Mantia 200754 |
To determine whether CFX slows the progression of MS or not. |
Pucci 200755 |
To determine the effectiveness and safety of Amantadine in treating fatigue in people with MS. |
Mills 200756 |
To evaluate of the effectiveness and tolerance of drug and non-drug treatments for ataxia in patients with MS. |
Thomas 200657 |
To evaluate the effectiveness of psychological interventions for people with MS. |
Gray 200458 |
To identify and summarize evidence that Methotrexate is beneficial and safe for people with MS. |
Urciuoli 200459 |
To evaluate and summarize the effectiveness and safety of PGE1 in the treatment of erectile dysfunction. |
Bennett 200460 |
To evaluate the effectiveness and safety evaluation of HBOT in the treatment of MS. |
Shakespear 200361 |
Evaluation of the effectiveness and absolute tolerance and comparative study of anti-spasticity agents in MS patients. |
Gray 200362 |
To identify and summarize the evidence which indicates that intravenous immunoglobulins are safe and beneficial for individuals with MS. |
Steultjens 200363 |
To determine whether occupational therapy interventions in MS patients improve functional ability, social participation, and/or health-related quality of life. |
Solari 200264 |
To determine the effectiveness and safety of Amino-pyridines for neurological deficits in adults with MS." |
Rice 200165 |
The purpose of this review was to evaluate the effects of recombinant Interferons in adults with RRMS. |
Filippini 200066 |
The primary objectives were to determine the effects of Corticosteroids and ACTH for the treatment of MS patients with acute exacerbations in terms of improving disability. Reducing the risk of new exacerbations during follow-up and preventing the progression of disability in long-term follow-up. Secondary objectives included the frequency and severity of adverse effects and their acceptability in light of the benefits. The different effects of Corticosteroids with respect to doses and drugs, routes of administration, duration of treatment, and the time interval between the onset of symptoms and randomization, based on indirect comparisons; different therapeutic effects based on the course of the disease and the effect of Corticosteroids or ACTH on magnetic resonance imaging as an alternative indicator of disease activity. |
Following this, we assessed the quality of each Cochrane systematic review study using the JBI checklist (Table 2).
Table 2.
Assessing the quality of the studies using the JBI checklist
Author – year
|
Q1
|
Q2
|
Q3
|
Q4
|
Q5
|
Q6
|
Q7
|
Q8
|
Q9
|
Q10
|
Q11
|
Garegnani 202010 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Parks 201911 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Hayes 201912 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Latorraca 201913 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Jagannath 201914 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Amatya 201815 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Köpke 201816 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Yes |
Jagannath 201817 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Rietberg 201718 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Zhang 201719 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Filippini 201720 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
La Mantia 201621 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
La Mantia 201622 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
He 201623 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Yes |
Yang 201524 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
Tramacere 201525 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Heine 201526 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Xu 201527 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Khan 201528 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Rosti-Otajärvi 201429 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Xiao 201430 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
Liu 201331 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
He 201332 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
He 201333 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
He 201334 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
Filippini 201335 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Martinelli Boneschi 201336 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Amatya 201337 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Burton 201238 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Tejani 201239 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Xiao 201240 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Sitjà Rabert 201241 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
La Mantia 201242 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Wang 201143 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Pucci 201144 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Koch 201145 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
La Mantia 201046 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Rose 201047 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Rojas 201048 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Yes |
Khan 200949 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Ciccone 200850 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Clerico 200851 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Casetta 200752 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Khan 200753 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
La Mantia 200754 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Pucci 200755 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
No |
Yes |
Yes |
Mills 200756 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Thomas 200657 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
Gray 200458 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
Urciuoli 200459 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Bennett 200460 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Shakespere 200361 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
NA |
Yes |
Yes |
Gray 200362 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Steultjens 200363 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
No |
Yes |
Yes |
Solari 200264 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA |
Yes |
Yes |
Yes |
Rice 200165 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
Yes |
Yes |
Filippini 200066 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
NA: Not Applicable.
Q1. Is the review question clearly and explicitly stated? Q2. Were the inclusion criteria appropriate for the review question? Q3. was the search strategy appropriate? Q4. Were the sources and resources used to search for studies adequate? Q5. Were the criteria for appraising studies appropriate? Q6. Was critical appraisal conducted by two or more reviewers independently? Q7. Were there methods to minimize errors in data extraction Q8. Were the methods used to combine studies appropriate? Q9. Was the likelihood of publication bias assessed? Q10. Were recommendations for policy and/or practice supported by the reported data? Q11.Were the specific directives for new research appropriate?
The results of this evaluation are presented in Table 3, indicating that all studies met the acceptable quality standards.
In the process of conducting quality reviews of clinical trials under systematic review studies, the most frequently observed risk of bias was a low outcome for Selective Reporting (data reporting), followed by an unclear outcome for allocation concealment (selection bias) (Figures 1 and 2).
Table 3.
The Number of Different Biases in the Articles Included in the Study
Num
|
Study
|
Sample size
|
No of included RCTs
|
Random sequence generation (Selection bias)
|
Allocation concealment (Selection bias)
|
Blinding of participants and personnel (Performance bias)
|
Blinding of outcome assessor (Detection bias)
|
Incomplete outcome data (Attrition bias)
|
Selective reporting (Reporting data)
|
Blinding (Performance and detection bias)
|
Other bias
|
Low
|
High
|
Unclear
|
Low
|
High
|
Unclear
|
Low
|
High
|
Unclear
|
Low
|
High
|
Unclear
|
Low
|
High
|
Unclear
|
Low
|
High
|
Unclear
|
Low
|
High
|
Unclear
|
Low
|
High
|
Unclear
|
1 |
Garegnani 202010 |
962 |
6 |
2 |
2 |
2 |
1 |
- |
5 |
6 |
- |
- |
5 |
1 |
- |
4 |
1 |
1 |
- |
- |
- |
- |
2 |
4 |
6 |
- |
- |
2 |
Parks 202011 |
- |
30 |
12 |
2 |
16 |
11 |
2 |
17 |
17 |
4 |
9 |
12 |
2 |
16 |
3 |
16 |
11 |
- |
- |
- |
6 |
8 |
16 |
12 |
8 |
10 |
3 |
Hayes 201912 |
839 |
13 |
10 |
- |
3 |
4 |
- |
9 |
- |
- |
13 |
9 |
1 |
3 |
9 |
2 |
2 |
- |
- |
- |
1 |
7 |
5 |
4 |
- |
9 |
4 |
Latorraca 201913 |
146 |
3 |
1 |
1 |
1 |
1 |
1 |
1 |
- |
3 |
- |
1 |
2 |
- |
- |
3 |
- |
- |
- |
- |
1 |
- |
2 |
3 |
- |
- |
5 |
Jagannath 201914 |
238 |
3 |
2 |
- |
1 |
3 |
- |
- |
2 |
- |
1 |
3 |
- |
- |
3 |
- |
- |
- |
- |
- |
3 |
- |
- |
3 |
- |
- |
6 |
Amatya 201815 |
565 |
10 |
9 |
- |
1 |
1 |
- |
9 |
6 |
2 |
2 |
6 |
2 |
2 |
8 |
2 |
- |
- |
- |
- |
10 |
- |
- |
8 |
- |
2 |
7 |
Köpke 201816 |
1387 |
11 |
11 |
- |
- |
7 |
1 |
3 |
2 |
8 |
1 |
9 |
1 |
1 |
7 |
3 |
1 |
- |
- |
- |
4 |
- |
7 |
1 |
- |
10 |
8 |
Jagannath 201817 |
933 |
12 |
4 |
2 |
6 |
2 |
4 |
6 |
- |
- |
- |
- |
- |
- |
5 |
5 |
2 |
7 |
3 |
2 |
4 |
- |
8 |
7 |
4 |
1 |
9 |
Rietberg 201718 |
195 |
6 |
3 |
1 |
2 |
2 |
1 |
3 |
- |
5 |
1 |
2 |
1 |
3 |
1 |
1 |
4 |
- |
- |
- |
1 |
2 |
3 |
- |
- |
- |
10 |
Zhang 201719 |
1694 |
3 |
3 |
- |
- |
2 |
- |
1 |
- |
3 |
- |
3 |
- |
- |
- |
1 |
2 |
- |
- |
- |
3 |
- |
- |
- |
- |
3 |
11 |
Filippini 201720 RCTs |
3745 |
10 |
8 |
- |
2 |
4 |
- |
6 |
1 |
7 |
2 |
4 |
1 |
5 |
4 |
4 |
2 |
- |
- |
- |
6 |
3 |
1 |
8 |
- |
2 |
Filippini 201720 OLEs |
1868 |
8 |
- |
8 |
- |
- |
8 |
- |
- |
8 |
- |
- |
8 |
- |
1 |
7 |
- |
- |
- |
- |
2 |
6 |
- |
2 |
- |
6 |
12 |
La Mantia 201621 |
2904 |
6 |
4 |
- |
2 |
1 |
- |
5 |
1 |
4 |
1 |
3 |
1 |
2 |
- |
6 |
- |
- |
- |
- |
3 |
3 |
- |
- |
2 |
4 |
13 |
La Mantia 201622 |
5152 |
6 |
6 |
- |
- |
5 |
- |
1 |
5 |
1 |
- |
5 |
1 |
- |
3 |
3 |
- |
- |
- |
- |
5 |
- |
1 |
2 |
4 |
- |
14 |
He 201623 |
3231 |
5 |
5 |
- |
- |
5 |
- |
- |
2 |
3 |
- |
- |
5 |
- |
- |
3 |
2 |
- |
- |
- |
5 |
- |
- |
- |
5 |
- |
15 |
Yang 201524 |
120 |
1 |
1 |
- |
- |
1 |
- |
- |
1 |
- |
- |
1 |
- |
- |
- |
1 |
- |
- |
- |
- |
1 |
- |
- |
1 |
- |
- |
16 |
Tramacere 201525 |
25113 |
39 |
34 |
- |
5 |
21 |
1 |
17 |
12 |
15 |
12 |
19 |
7 |
13 |
20 |
14 |
5 |
- |
- |
- |
36 |
3 |
- |
3 |
33 |
3 |
17 |
Heine 201526 |
2250 |
45 |
27 |
2 |
16 |
18 |
6 |
21 |
- |
44 |
1 |
- |
44 |
1 |
30 |
11 |
4 |
- |
- |
- |
42 |
2 |
1 |
34 |
5 |
6 |
18 |
Xu 201527 |
2667 |
2 |
- |
- |
2 |
2 |
- |
- |
2 |
- |
- |
2 |
- |
- |
- |
2 |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
2 |
19 |
Khan 201528 |
531 |
9 |
3 |
1 |
5 |
2 |
6 |
1 |
- |
8 |
1 |
1 |
8 |
- |
6 |
1 |
2 |
- |
- |
- |
9 |
- |
- |
1 |
1 |
7 |
20 |
Rosti - Otajärvi 201429 |
986 |
20 |
7 |
13 |
- |
6 |
14 |
- |
4&5
|
14&13 |
2&2 |
14 |
1 |
5 |
16 |
3 |
1 |
- |
- |
- |
16 |
4 |
- |
12 |
6 |
2 |
21 |
Xiao 201430 |
26 |
1 |
1 |
- |
- |
- |
- |
1 |
- |
1 |
- |
- |
- |
1 |
1 |
- |
- |
- |
- |
- |
- |
1 |
- |
- |
1 |
- |
22 |
Liu 201331 |
851 |
2 |
2 |
- |
- |
1 |
- |
1 |
2 |
- |
- |
2 |
- |
- |
2 |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
2 |
23 |
He 201332 |
625 |
7 |
7 |
- |
- |
7 |
- |
- |
- |
- |
- |
- |
- |
- |
4 |
3 |
- |
6 |
- |
1 |
5 |
- |
2 |
- |
- |
7 |
24 |
He 201333 |
104 |
1 |
- |
- |
1 |
- |
- |
1 |
- |
- |
- |
- |
- |
- |
- |
1 |
- |
1 |
- |
- |
1 |
- |
- |
- |
- |
1 |
25 |
He 201334 |
1106 |
1 |
1 |
- |
- |
1 |
- |
- |
1 |
- |
- |
1 |
- |
- |
- |
1 |
- |
- |
- |
- |
1 |
- |
- |
- |
- |
1 |
26 |
Filippini 201335 |
17401 |
44 |
21 |
1 |
22 |
16 |
2 |
26 |
13 |
14 |
17 |
28 |
4 |
12 |
26 |
13 |
5 |
- |
- |
- |
30 |
12 |
2 |
2 |
35 |
7 |
27 |
Martinelli Boneschi 201336 |
221 |
3 |
2 |
- |
1 |
2 |
- |
1 |
- |
- |
- |
- |
- |
- |
2 |
1 |
- |
2 |
1 |
- |
3 |
- |
- |
1 |
2 |
- |
28 |
Amatya 201337 |
341 |
9 |
2 |
1 |
6 |
3 |
2 |
4 |
4 |
4 |
1 |
5 |
2 |
2 |
5 |
2 |
2 |
- |
- |
- |
7 |
- |
2 |
- |
- |
9 |
29 |
Burton 201238 |
215 |
5 |
3 |
- |
2 |
2 |
1 |
2 |
3 |
- |
2 |
2 |
- |
3 |
3 |
- |
2 |
- |
- |
- |
3 |
- |
2 |
1 |
3 |
1 |
30 |
Tejani 201239 |
30 |
1 |
- |
- |
1 |
- |
- |
1 |
- |
- |
- |
- |
- |
- |
- |
- |
1 |
- |
- |
1 |
1 |
- |
- |
- |
1 |
- |
31 |
Xiao 201240 |
420 |
2 |
2 |
- |
- |
2 |
- |
- |
2 |
- |
- |
2 |
- |
- |
- |
2 |
- |
- |
- |
- |
2 |
- |
- |
1 |
- |
1 |
32 |
Sitjà Rabert 201241 |
- |
10 |
- |
4 |
6 |
- |
3 |
7 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
1 |
6 |
3 |
9 |
1 |
- |
8 |
2 |
- |
33 |
La Mantia 201242 |
3122 |
5 |
4 |
- |
1 |
3 |
- |
2 |
- |
2 |
3 |
3 |
- |
2 |
2 |
3 |
- |
- |
- |
- |
- |
- |
- |
1 |
4 |
- |
34 |
Wang 201143 |
458 |
4 |
3 |
- |
1 |
2 |
- |
2 |
- |
- |
- |
- |
- |
- |
1 |
3 |
- |
3 |
1 |
- |
3 |
- |
1 |
2 |
- |
2 |
35 |
Pucci 201144 |
2223 |
3 |
2 |
- |
1 |
2 |
- |
1 |
- |
- |
- |
- |
- |
- |
- |
- |
3 |
3 |
- |
- |
3 |
- |
- |
- |
3 |
- |
36 |
Koch 201145 |
70 |
2 |
1 |
- |
1 |
2 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
2 |
- |
1 |
- |
1 |
2 |
- |
- |
2 |
- |
- |
37 |
La Mantia 201046 |
1499 |
6 |
4 |
- |
2 |
4 |
1 |
1 |
- |
- |
- |
- |
- |
- |
5 |
1 |
- |
5 |
- |
1 |
5 |
1 |
- |
4 |
2 |
- |
38 |
Rose 201047 |
149 |
4 |
2 |
- |
2 |
1 |
1 |
2 |
- |
- |
- |
- |
- |
- |
1&4&1&2 |
1&1 |
1&1 |
1&3&1&3 |
1&1&1 |
1 |
4 |
- |
- |
3 |
1 |
- |
39 |
Rojas 201048 |
123 |
2 |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
- |
- |
- |
2 |
- |
- |
2 |
- |
- |
1 |
1 |
- |
2 |
- |
- |
40 |
Khan 200949 |
80 |
2 |
- |
1 |
1 |
- |
2 |
- |
- |
- |
- |
- |
- |
- |
- |
1 |
1 |
- |
2 |
- |
- |
- |
2 |
- |
- |
2 |
41 |
Ciccone 200850 |
183 |
3 |
- |
- |
- |
2 |
- |
1 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
42 |
Clerico 200851 |
1160 |
3 |
- |
- |
- |
2 |
- |
1 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
43 |
Casetta 200752 |
698 |
5 |
- |
- |
- |
3 |
- |
2 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
44 |
Khan 200753 |
1027 |
13 |
9 |
4 |
- |
3 |
7 |
3 |
- |
10 |
3 |
6 |
7 |
- |
10 |
3 |
- |
- |
- |
- |
12 |
- |
1 |
3 |
9 |
2 |
45 |
La Mantia 200754 |
224 |
4 |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
46 |
Pucci 200755 |
272 |
5 |
- |
- |
- |
1 |
- |
4 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
47 |
Mills 200756 |
367 |
10 |
- |
- |
- |
2 |
- |
8 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
48 |
Thomas 200657 |
1006 |
17 |
- |
- |
- |
3 |
1 |
13 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
49 |
Gray 200458 |
60 |
1 |
- |
- |
- |
1 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
50 |
Urciuoli 200459 |
1873 |
4 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
51 |
Bennett 200460 |
504 |
10 |
- |
- |
- |
2 |
- |
8 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
52 |
Shakespear 200361 |
- |
39 |
- |
- |
- |
3 |
1 |
32 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
53 |
Gray 200362 |
916 |
6 |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
54 |
Steultjens 200363 |
271 |
3 |
- |
- |
- |
1 |
- |
2 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
55 |
Solari 200264 |
198 |
7 |
- |
- |
- |
2 |
- |
5 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
56 |
Rice 200165 |
1301 |
8 |
- |
- |
- |
3 |
- |
5 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
57 |
Filippini 200066 |
377 |
6 |
2 |
- |
4 |
- |
- |
6 |
2 |
2 |
2 |
2 |
1 |
3 |
6 |
- |
- |
- |
- |
- |
- |
- |
- |
3 |
- |
3 |
All from 2016 until 2023 |
23859 |
132 |
80 |
16 |
36 |
49 |
17 |
66 |
42 |
48 |
30 |
62 |
26 |
32 |
48 |
57 |
27 |
7 |
3 |
2 |
54 |
31 |
47 |
56 |
23 |
47 |
All until 2015 |
72862 |
377 |
143 |
27 |
80 |
138 |
48 |
186 |
51 |
130 |
46 |
91 |
74 |
42 |
149 |
71 |
30 |
32 |
13 |
8 |
203 |
25 |
13 |
84 |
108 |
60 |
All |
96721 |
509 |
223 |
43 |
116 |
187 |
65 |
252 |
93 |
178 |
76 |
153 |
90 |
74 |
197 |
128 |
57 |
39 |
16 |
10 |
257 |
56 |
60 |
140 |
131 |
107 |
Figure 1.
Evaluating the extent of selection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 1.
Evaluating the extent of selection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 2.
Evaluating the extent of selection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 2.
Evaluating the extent of selection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Conversely, the group of blinding of participants and personnel (performance bias) exhibited the highest risk of bias (Figure 3), while the selective reporting (data reporting) group demonstrated the lowest risk of bias.
Figure 3.
Evaluating the extent of performance bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 3.
Evaluating the extent of performance bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
The risk of bias was also evaluated across different time frames. Specifically, the risk of bias was assessed in two distinct periods: up to 2015 and from 2016 to 2023. In the initial period, the most prevalent risk of bias was a low outcome for Selective Reporting (data reporting). However, in the recent years, the most common risk of bias shifted to a low outcome for Random Sequence Generation (selection bias).
Figures 1 to 8 provide a detailed representation of biases across these different time intervals.
Figure 4.
Evaluating the extent of detection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 4.
Evaluating the extent of detection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 5.
Evaluating the extent of attrition bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 5.
Evaluating the extent of attrition bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 6.
Evaluating the extent of performance and detection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 6.
Evaluating the extent of performance and detection bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 7.
evaluating the extent of repotting bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 7.
evaluating the extent of repotting bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 8.
Evaluating the extent of other possible bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Figure 8.
Evaluating the extent of other possible bias in trials incorporated into the systematic reviews of the Cochrane multiple sclerosis group
Discussion
The rapid increase in medical journals and articles has brought the structure of articles and research methodology into sharper focus. High-quality research is crucial as it directly impacts healthcare outcomes, influencing public health policies and treatments. Improving the quality and reducing bias in studies can enhance patient care and reduce healthcare costs.
The Cochrane Library, with its 53 review groups, including the Cochrane MS Group, provides a credible information base for medical decision-making. This systematic review, assesses the risk of biases in published RCTs on MS, within the Cochrane Database, known for its rigorous methodology and stringent bias assessment tools.
MS, a common neurological disease causing significant disability in young adults, necessitates high-quality clinical trials and systematic reviews.5 The management and treatment of this condition are continually evolving, with many RCTs evaluating interventions efficacy. These RCTs, when systematically reviewed, provide valuable insights that guide clinical decisions and health policies.1 However, the reliability of these systematic reviews hinges on the quality of the included RCTs. Biases in RCTs can lead to inaccurate conclusions and harmful clinical recommendations, making bias assessment crucial.
Previous studies have highlighted the variability in the quality of systematic reviews across medical fields. For instance, Gagnier and Kellam67 questioned the credibility of orthopedic systematic reviews, while another study found that only a small fraction of internal medicine systematic reviews achieved high scores on the AMSTAR scoring system.68
Salehi-Pourmehr et al69 reviewed Cochrane systematic reviews in urologic cancers, finding that the most common bias was unclear result for selection bias (allocation concealment and random sequence generation). The highest risk of bias was performance bias (blinding of participants and personnel), while the least was attrition bias (selective and incomplete outcome data). They also noted that some biases are decreasing over time, while some others are increasing.
Hajebrahimi et al70 examined the quality of systematic review articles in gynecologic cancers and found that the most common biases were unclear result for selection bias (allocation concealment), and performance bias (blinding of participants and personnel). Also, the highest risk of bias was in Blinding participants and personnel (performance bias), and Incomplete outcome data (attrition bias) while, the lowest risk was in Incomplete outcome data (attrition bias) and Random sequence generation (selection bias).
Despite some biases decreasing, others are increasing, and many remain unclear. This indicates that, despite advancements in study quality assessment and the promotion of systematic reviews, achieving ideal quality in clinical studies is still a work in progress.
Our assessment examined various biases, including selection, performance, detection, attrition, and reporting biases, which can compromise the internal validity of an RCT. Using the PRISMA tool, we found that all studies included in this review met the acceptable quality standards according to the JBI criteria. The most common risk of bias was a low result for selective reporting bias, followed by unclear result for allocation concealment (selection bias). The highest risk of bias was in blinding personnel and participants (performance bias), while the lowest was in selective reporting (reporting data). Selection bias, can lead to imbalances between groups. Also performance and detection biases can influence the outcomes. Additionally, attrition bias can skew the results and reporting bias can misrepresent the intervention’s effect.
Our preliminary findings indicate varying degrees of bias across RCTs, emphasizing the need for more rigorous conduct and reporting to minimize biases. This highlights the importance of considering bias risk when interpreting systematic reviews. Given that the current research is limited to Cochrane Library articles, future studies should also examine articles from other databases for various biases.
Conclusion
Based on the results of the current study, the risk of various biases in most studies conducted in recent years in the field of MS has been declining in all three groups: Low, Unclear, and High, compared to previous years. However, it should be noted that part of this issue may be due to the fewer number of articles entered in the study from 2016 onwards compared to the years before that. In conclusion, despite significant enhancements in improving the quality of studies, there is still a far way to achieve the ideal quality.
Competing Interests
The authors state no Competing interests.
Ethical Approval
This systematic review was conducted with a commitment to transparency and integrity. All included studies were selected based on predefined criteria to ensure an unbiased and comprehensive review.
References
- Rezaei-Ghaleh N, Azizi F. The impact factor-based quality assessment of biomedical research institutes in Iran: effect of impact factor normalization by subject. Arch Iran Med 2007; 10(2):182-9. [ Google Scholar]
- International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Pathology 1997; 29(4):441-7. doi: 10.1080/00313029700169515 [Crossref] [ Google Scholar]
- Harris JD, Quatman CE, Manring MM, Siston RA, Flanigan DC. How to write a systematic review. Am J Sports Med 2014; 42(11):2761-8. doi: 10.1177/0363546513497567 [Crossref] [ Google Scholar]
- Browne P, Chandraratna D, Angood C, Tremlett H, Baker C, Taylor BV. Atlas of multiple sclerosis 2013: a growing global problem with widespread inequity. Neurology 2014; 83(11):1022-4. doi: 10.1212/wnl.0000000000000768 [Crossref] [ Google Scholar]
- Kobelt G, Thompson A, Berg J, Gannedahl M, Eriksson J. New insights into the burden and costs of multiple sclerosis in Europe. Mult Scler 2017; 23(8):1123-36. doi: 10.1177/1352458517694432 [Crossref] [ Google Scholar]
- Hauser SL. Multiple sclerosis and other demyelinating diseases. In: Harrison’s Principles of Internal Medicine. McGraw-Hill; 1994. p. 2287.
- Brust J. Current Diagnosis & Treatment in Neurology. Lange Medical Books, McGraw-Hill; 2007.
- Palacios N, Alonso A, Brønnum-Hansen H, Ascherio A. Smoking and increased risk of multiple sclerosis: parallel trends in the sex ratio reinforce the evidence. Ann Epidemiol 2011; 21(7):536-42. doi: 10.1016/j.annepidem.2011.03.001 [Crossref] [ Google Scholar]
- Correale J, Gaitán MI. Multiple sclerosis and environmental factors: the role of vitamin D, parasites, and Epstein-Barr virus infection. Acta Neurol Scand 2015; 132(199):46-55. doi: 10.1111/ane.12431 [Crossref] [ Google Scholar]
- Garegnani L, Franco JV, Ciapponi A, Garrote V, Vietto V, Portillo Medina SA. Ventriculo-peritoneal shunting devices for hydrocephalus. Cochrane Database Syst Rev 2020; 6(6):CD012726. doi: 10.1002/14651858.CD012726.pub2 [Crossref] [ Google Scholar]
- Parks NE, Jackson-Tarlton CS, Vacchi L, Merdad R, Johnston BC. Dietary interventions for multiple sclerosis-related outcomes. Cochrane Database Syst Rev 2020; 5(5):CD004192. doi: 10.1002/14651858.CD004192.pub4 [Crossref] [ Google Scholar]
- Hayes S, Galvin R, Kennedy C, Finlayson M, McGuigan C, Walsh CD. Interventions for preventing falls in people with multiple sclerosis. Cochrane Database Syst Rev 2019; 11(11):CD012475. doi: 10.1002/14651858.CD012475.pub2 [Crossref] [ Google Scholar]
- Latorraca CO, Martimbianco AL, Pachito DV, Torloni MR, Pacheco RL, Pereira JG. Palliative care interventions for people with multiple sclerosis. Cochrane Database Syst Rev 2019; 10(10):CD012936. doi: 10.1002/14651858.CD012936.pub2 [Crossref] [ Google Scholar]
- Jagannath VA, Pucci E, Asokan GV, Robak EW. Percutaneous transluminal angioplasty for treatment of chronic cerebrospinal venous insufficiency (CCSVI) in people with multiple sclerosis. Cochrane Database Syst Rev 2019; 5(5):CD009903. doi: 10.1002/14651858.CD009903.pub3 [Crossref] [ Google Scholar]
- Amatya B, Young J, Khan F. Non-pharmacological interventions for chronic pain in multiple sclerosis. Cochrane Database Syst Rev 2018; 12(12):CD012622. doi: 10.1002/14651858.CD012622.pub2 [Crossref] [ Google Scholar]
- Köpke S, Solari A, Rahn A, Khan F, Heesen C, Giordano A. Information provision for people with multiple sclerosis. Cochrane Database Syst Rev 2018; 10(10):CD008757. doi: 10.1002/14651858.CD008757.pub3 [Crossref] [ Google Scholar]
- Jagannath VA, Filippini G, Di Pietrantonj C, Asokan GV, Robak EW, Whamond L. Vitamin D for the management of multiple sclerosis. Cochrane Database Syst Rev 2018; 9(9):CD008422. doi: 10.1002/14651858.CD008422.pub3 [Crossref] [ Google Scholar]
- Rietberg MB, Veerbeek JM, Gosselink R, Kwakkel G, van Wegen EE. Respiratory muscle training for multiple sclerosis. Cochrane Database Syst Rev 2017; 12(12):CD009424. doi: 10.1002/14651858.CD009424.pub2 [Crossref] [ Google Scholar]
- Zhang J, Shi S, Zhang Y, Luo J, Xiao Y, Meng L. Alemtuzumab versus interferon beta 1a for relapsing-remitting multiple sclerosis. Cochrane Database Syst Rev 2017; 11(11):CD010968. doi: 10.1002/14651858.CD010968.pub2 [Crossref] [ Google Scholar]
- Filippini G, Del Giovane C, Clerico M, Beiki O, Mattoscio M, Piazza F. Treatment with disease-modifying drugs for people with a first clinical attack suggestive of multiple sclerosis. Cochrane Database Syst Rev 2017; 4(4):CD012200. doi: 10.1002/14651858.CD012200.pub2 [Crossref] [ Google Scholar]
- La Mantia L, Di Pietrantonj C, Rovaris M, Rigon G, Frau S, Berardo F. Interferons-beta versus glatiramer acetate for relapsing-remitting multiple sclerosis. Cochrane Database Syst Rev 2016; 11(11):CD009333. doi: 10.1002/14651858.CD009333.pub3 [Crossref] [ Google Scholar]
- La Mantia L, Tramacere I, Firwana B, Pacchetti I, Palumbo R, Filippini G. Fingolimod for relapsing-remitting multiple sclerosis. Cochrane Database Syst Rev 2016; 4(4):CD009371. doi: 10.1002/14651858.CD009371.pub2 [Crossref] [ Google Scholar]
- He D, Zhang C, Zhao X, Zhang Y, Dai Q, Li Y. Teriflunomide for multiple sclerosis. Cochrane Database Syst Rev 2016; 3(3):CD009882. doi: 10.1002/14651858.CD009882.pub3 [Crossref] [ Google Scholar]
- Yang C, Hao Z, Zhang L, Zeng L, Wen J. Sodium channel blockers for neuroprotection in multiple sclerosis. Cochrane Database Syst Rev 2015; 2015(10):CD010422. doi: 10.1002/14651858.CD010422.pub2 [Crossref] [ Google Scholar]
- Tramacere I, Del Giovane C, Salanti G, D’Amico R, Filippini G. Immunomodulators and immunosuppressants for relapsing-remitting multiple sclerosis: a network meta-analysis. Cochrane Database Syst Rev 2015; 2015(9):CD011381. doi: 10.1002/14651858.CD011381.pub2 [Crossref] [ Google Scholar]
- Heine M, van de Port I, Rietberg MB, van Wegen EE, Kwakkel G. Exercise therapy for fatigue in multiple sclerosis. Cochrane Database Syst Rev 2015; 2015(9):CD009956. doi: 10.1002/14651858.CD009956.pub2 [Crossref] [ Google Scholar]
- Xu Z, Zhang F, Sun F, Gu K, Dong S, He D. Dimethyl fumarate for multiple sclerosis. Cochrane Database Syst Rev 2015; 2015(4):CD011076. doi: 10.1002/14651858.CD011076.pub2 [Crossref] [ Google Scholar]
- Khan F, Amatya B, Kesselring J, Galea MP. Telerehabilitation for persons with multiple sclerosis A Cochrane review. Eur J Phys Rehabil Med 2015; 51(3):311-25. [ Google Scholar]
- Rosti-Otajärvi EM, Hämäläinen PI. Neuropsychological rehabilitation for multiple sclerosis. Cochrane Database Syst Rev 2014; 2014(2):CD009131. doi: 10.1002/14651858.CD009131.pub3 [Crossref] [ Google Scholar]
- Xiao Y, Huang J, Luo H, Wang J. Mycophenolate mofetil for relapsing-remitting multiple sclerosis. Cochrane Database Syst Rev 2014; 2014(2):CD010242. doi: 10.1002/14651858.CD010242.pub2 [Crossref] [ Google Scholar]
- Liu J, Wang L, Zhan SY, Xia Y. Daclizumab for relapsing remitting multiple sclerosis. Cochrane Database Syst Rev. 2012(4):CD008127. 10.1002/14651858.CD008127.pub3.
- He D, Zhang Y, Dong S, Wang D, Gao X, Zhou H. Pharmacological treatment for memory disorder in multiple sclerosis. Cochrane Database Syst Rev. 2013(12):CD008876. 10.1002/14651858.CD008876.pub3.
- He D, Guo R, Zhang F, Zhang C, Dong S, Zhou H. Rituximab for relapsing-remitting multiple sclerosis. Cochrane Database Syst Rev. 2013(12):CD009130. 10.1002/14651858.CD009130.pub3.
- He D, Han K, Gao X, Dong S, Chu L, Feng Z, et al. Laquinimod for multiple sclerosis. Cochrane Database Syst Rev. 2013(8):CD010475. 10.1002/14651858.CD010475.pub2.
- Filippini G, Del Giovane C, Vacchi L, D’Amico R, Di Pietrantonj C, Beecher D, et al. Immunomodulators and immunosuppressants for multiple sclerosis: a network meta-analysis. Cochrane Database Syst Rev. 2013(6):CD008933. 10.1002/14651858.CD008933.pub2.
- Martinelli Boneschi F, Vacchi L, Rovaris M, Capra R, Comi G. Mitoxantrone for multiple sclerosis. Cochrane Database Syst Rev. 2013(5):CD002127. 10.1002/14651858.CD002127.pub3.
- Amatya B, Khan F, La Mantia L, Demetrios M, Wade DT. Non pharmacological interventions for spasticity in multiple sclerosis. Cochrane Database Syst Rev. 2013(2):CD009974. 10.1002/14651858.CD009974.pub2.
- Burton JM, O’Connor PW, Hohol M, Beyene J. Oral versus intravenous steroids for treatment of relapses in multiple sclerosis. Cochrane Database Syst Rev 2012; 12:CD006921. doi: 10.1002/14651858.CD006921.pub3 [Crossref] [ Google Scholar]
- Tejani AM, Wasdell M, Spiwak R, Rowell G, Nathwani S. Carnitine for fatigue in multiple sclerosis. Cochrane Database Syst Rev 2012; 2012(5):CD007280. doi: 10.1002/14651858.CD007280.pub3 [Crossref] [ Google Scholar]
- Xiao Y, Wang J, Luo H. Sildenafil citrate for erectile dysfunction in patients with multiple sclerosis. Cochrane Database Syst Rev. 2012(4):CD009427. 10.1002/14651858.CD009427.pub2.
- Sitjà Rabert M, Rigau Comas D, Fort Vanmeerhaeghe A, Santoyo Medina C, Roqué i Figuls M, Romero-Rodríguez D, et al. Whole-body vibration training for patients with neurodegenerative disease. Cochrane Database Syst Rev. 2012(2):CD009097. 10.1002/14651858.CD009097.pub2.
- La Mantia L, Vacchi L, Di Pietrantonj C, Ebers G, Rovaris M, Fredrikson S. Interferon beta for secondary progressive multiple sclerosis. Cochrane Database Syst Rev 2012; 1:CD005181. doi: 10.1002/14651858.CD005181.pub3 [Crossref] [ Google Scholar]
- Wang J, Xiao Y, Luo M, Luo H. Statins for multiple sclerosis. Cochrane Database Syst Rev 2011; 2011(12):CD008386. doi: 10.1002/14651858.CD008386.pub3 [Crossref] [ Google Scholar]
- Pucci E, Giuliani G, Solari A, Simi S, Minozzi S, Di Pietrantonj C, et al. Natalizumab for relapsing remitting multiple sclerosis. Cochrane Database Syst Rev. 2011(10):CD007621. 10.1002/14651858.CD007621.pub2.
- Koch MW, Glazenborg A, Uyttenboogaart M, Mostert J, De Keyser J. Pharmacologic treatment of depression in multiple sclerosis. Cochrane Database Syst Rev. 2011(2):CD007295. 10.1002/14651858.CD007295.pub2.
- La Mantia L, Munari LM, Lovati R. Glatiramer acetate for multiple sclerosis. Cochrane Database Syst Rev. 2010(5):CD004678. 10.1002/14651858.CD004678.pub2.
- Rose KJ, Burns J, Wheeler DM, North KN. Interventions for increasing ankle range of motion in patients with neuromuscular disease. Cochrane Database Syst Rev. 2010(2):CD006973. 10.1002/14651858.CD006973.pub2.
- Rojas JI, Romano M, Ciapponi A, Patrucco L, Cristiano E. Interferon Beta for primary progressive multiple sclerosis. Cochrane Database Syst Rev. 2010(1):CD006643. 10.1002/14651858.CD006643.pub3.
- Khan F, Ng L, Turner-Stokes L. Effectiveness of vocational rehabilitation intervention on the return to work and employment of persons with multiple sclerosis. Cochrane Database Syst Rev 2009; 2009(1):CD007256. doi: 10.1002/14651858.CD007256.pub2 [Crossref] [ Google Scholar]
- Ciccone A, Beretta S, Brusaferri F, Galea I, Protti A, Spreafico C. Corticosteroids for the long-term treatment in multiple sclerosis. Cochrane Database Syst Rev. 2008(1):CD006264. 10.1002/14651858.CD006264.pub2.
- Clerico M, Faggiano F, Palace J, Rice G, Tintorè M, Durelli L. Recombinant interferon beta or glatiramer acetate for delaying conversion of the first demyelinating event to multiple sclerosis. Cochrane Database Syst Rev. 2008(2):CD005278. 10.1002/14651858.CD005278.pub3.
- Casetta I, Iuliano G, Filippini G. Azathioprine for multiple sclerosis. Cochrane Database Syst Rev 2007; 2007(4):CD003982. doi: 10.1002/14651858.CD003982.pub2 [Crossref] [ Google Scholar]
- Khan F, Turner-Stokes L, Ng L, Kilpatrick T. Multidisciplinary rehabilitation for adults with multiple sclerosis. Cochrane Database Syst Rev 2007; 2007(2):CD006036. doi: 10.1002/14651858.CD006036.pub2 [Crossref] [ Google Scholar]
- La Mantia L, Milanese C, Mascoli N, D’Amico R, Weinstock-Guttman B. Cyclophosphamide for multiple sclerosis. Cochrane Database Syst Rev 2007; 2007(1):CD002819. doi: 10.1002/14651858.CD002819.pub2 [Crossref] [ Google Scholar]
- Pucci E, Branãs P, D’Amico R, Giuliani G, Solari A, Taus C. Amantadine for fatigue in multiple sclerosis. Cochrane Database Syst Rev 2007; 2007(1):CD002818. doi: 10.1002/14651858.CD002818.pub2 [Crossref] [ Google Scholar]
- Mills RJ, Yap L, Young CA. Treatment for ataxia in multiple sclerosis. Cochrane Database Syst Rev. 2007(1):CD005029. 10.1002/14651858.CD005029.pub2.
- Thomas PW, Thomas S, Hillier C, Galvin K, Baker R. Psychological interventions for multiple sclerosis. Cochrane Database Syst Rev 2006; 2006(1):CD004431. doi: 10.1002/14651858.CD004431.pub2 [Crossref] [ Google Scholar]
- Gray O, McDonnell GV, Forbes RB. Methotrexate for multiple sclerosis. Cochrane Database Syst Rev 2004; 2004(2):CD003208. doi: 10.1002/14651858.CD003208.pub2 [Crossref] [ Google Scholar]
- Urciuoli R, Cantisani TA, Carlini M, Giuglietti M, Botti FM. Prostaglandin E1 for treatment of erectile dysfunction. Cochrane Database Syst Rev. 2004(2):CD001784. 10.1002/14651858.CD001784.pub2.
- Bennett M, Heard R. Hyperbaric oxygen therapy for multiple sclerosis. Cochrane Database Syst Rev 2004; 2004(1):CD003057. doi: 10.1002/14651858.CD003057.pub2 [Crossref] [ Google Scholar]
- Shakespeare DT, Young CA, Boggild M. Anti-spasticity agents for multiple sclerosis. Cochrane Database Syst Rev 2000; 2003(4):CD001332. doi: 10.1002/14651858.cd001332 [Crossref] [ Google Scholar]
- Gray O, McDonnell GV, Forbes RB. Intravenous immunoglobulins for multiple sclerosis. Cochrane Database Syst Rev 2003; 2003(4):CD002936. doi: 10.1002/14651858.cd002936 [Crossref] [ Google Scholar]
- Steultjens EM, Dekker J, Bouter LM, Cardol M, Van de Nes JC, Van den Ende CH. Occupational therapy for multiple sclerosis. Cochrane Database Syst Rev 2003; 2003(3):CD003608. doi: 10.1002/14651858.cd003608 [Crossref] [ Google Scholar]
- Solari A, Uitdehaag B, Giuliani G, Pucci E, Taus C. Aminopyridines for symptomatic treatment in multiple sclerosis. Cochrane Database Syst Rev. 2002(4):CD001330. 10.1002/14651858.cd001330.
- Rice GP, Incorvaia B, Munari L, Ebers G, Polman C, D’Amico R. Interferon in relapsing-remitting multiple sclerosis. Cochrane Database Syst Rev 2001; 2001(4):CD002002. doi: 10.1002/14651858.cd002002 [Crossref] [ Google Scholar]
- Filippini G, Brusaferri F, Sibley WA, Citterio A, Ciucci G, Midgard R. Corticosteroids or ACTH for acute exacerbations in multiple sclerosis. Cochrane Database Syst Rev 2000; 2000(4):CD001331. doi: 10.1002/14651858.cd001331 [Crossref] [ Google Scholar]
- Gagnier JJ, Kellam PJ. Reporting and methodological quality of systematic reviews in the orthopaedic literature. J Bone Joint Surg 2013; 95(11):e77. doi: 10.2106/jbjs.l.00597 [Crossref] [ Google Scholar]
- Shea BJ, Bouter LM, Peterson J, Boers M, Andersson N, Ortiz Z. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One 2007; 2(12):e1350. doi: 10.1371/journal.pone.0001350 [Crossref] [ Google Scholar]
- Salehi-Pourmehr H, Naseri A, Mostafaei A, Vahedi L, Sajjadi S, Tayebi S. Misconduct in research integrity: assessment the quality of systematic reviews in Cochrane urological cancer review group. Turk J Urol 2021; 47(5):392-419. doi: 10.5152/tud.2021.21038 [Crossref] [ Google Scholar]
- Hajebrahimi S, Dalir Akbari N, Haji Kamanaj A, Hassannezhad S, Aminizadeh S, Darvishi F. Quality of the systematic reviews in Cochrane gynecological cancer group and their understudied RCTs. J Obstet Gynaecol India 2022; 72(Suppl 1):346-51. doi: 10.1007/s13224-022-01655-6 [Crossref] [ Google Scholar]