What contributions do languages other than English make on the results of meta-analyses?

doi:10.1016/S0895-4356(00)00188-8

Journal of Clinical Epidemiology

Volume 53, Issue 9, September 2000, Pages 964-972

https://doi.org/10.1016/S0895-4356(00)00188-8 Get rights and content

Abstract

Including only a portion of all available evidence may introduce systematic errors into the meta-analytic process and threaten its validity. We set out to examine whether language restricted meta-analyses, compared to language inclusive meta-analyses, provide different estimates of the effectiveness of interventions evaluated in randomized trials. We identified and retrieved all 79 meta-analyses from several disease areas in which explicit eligibility criteria regarding trial selection were reported. General characteristics and quality of reporting of the meta-analyses were assessed using a validated instrument. We explored the effects of language of publication of the randomized trials on the quantitative results using logistic regression analyses. Language restricted meta-analyses, compared to language inclusive meta-analyses, did not differ with respect to the estimate of benefit of the effectiveness of an intervention (ROR = 0.98; 95% CI: 0.81–1.17). These results were also robust after a series of sensitivity analyses. This study provides no evidence that language restricted meta-analyses lead to biased estimates of intervention effectiveness. We encourage others to replicate this study using different sampling frames, clinical topics and interventions.

Introduction

Meta-analysts have little control over random errors but can exert at least some control over systematic ones. Including only a portion of all available evidence may introduce systematic errors into the review process and threaten its validity. Grégoire and colleagues reported that 78% of identified meta-analyses of randomized trials had language of publication restrictions [1]. The majority (93%) of these restrictions were at the expense of excluding trials published in languages other than English (OEL).

One way to evaluate whether language restrictions are a sensible policy for meta-analysts is to assess the quality of reports of randomized trials. Language restrictions might be appropriate if the quality of reports of OEL were different compared to English language (EL) trials. In a previous study, members of our group [2] compared the methodological characteristics and analytical approaches of 133 EL randomized trials published between 1989 and 1994, with reports of 96 randomized trials published in French, German, Italian, and Spanish over the same time period and type of journal. Within the same language the reports were assessed under masked conditions, using a scale developed with appropriate rigorous standards [3].

The differences found in this study, between OEL and EL trials in the quality of reporting, with respect to randomization, allocation concealment, double-blinding, dropouts and withdrawals, or overall total score, were neither statistically nor substantively significant. The mean differences in the quality of reporting between OEL and EL trials was 5% for the total score, and ranged from 0% to 4% for individual items. Similar results have recently been reported elsewhere [4]. However, these studies did not address whether the exclusion of OEL alters the statistical results of a meta-analysis. This study addresses that question.

Section snippets

Selection of meta-analyses

We selected our meta-analyses from a collection of 251 meta-analyses of randomized trials from a larger database (n = 455) of such studies [5]. The details of the search strategy used to identify and retrieve the meta-analyses included in this study are reported elsewhere 6, 7. Briefly, a refined MEDLINE search strategy identified the meta-analyses. This search was supplemented with a search of the Cochrane Database of Systematic Reviews (1996, Issue 1). Meta-analyses were eligible if they

Between meta-analyses comparisons

Of 251 meta-analyses reviewed, 79 met our inclusion criteria (Table 1 and Appendix B). Only 19 meta-analyses actually included OEL in their quantitative analysis. One of these meta-analyses was dropped from further analysis because it did not report any binary outcomes, leaving 18 for further analyses. The remaining meta-analyses either had no language restrictions but did not incorporate OEL into a quantitative analysis (n = 22), or explicitly excluded such trials from their study (n = 38).

Discussion

Our primary result indicated only a 2% difference (ROR = 0.98), on average, between the treatment estimates with versus without explicit restrictions on the language of publication of the trials included. Combined with the narrow confidence intervals (0.81–1.17) and several sensitivity analyses, these results suggest that it is unlikely that any important clinical differences were missed. These results provide empirical evidence regarding an important issue that meta-analysts and others have

Acknowledgements

We thank Alison Jones, Leah Le Page, and Michael Saginur for helping to coordinate the study. Alejandro R. Jadad, MD, is a National Health Research Scholar, Health Canada. This research was funded by the Medical Research Council of Canada, Grant # 3705.

References (31)

G Grégoire et al.
Selecting the language of the publications included in a meta-analysisis there a tower of babel bias?
J Clin Epidemiol
(1995)
D Moher et al.
Completeness of reporting of trials published in languages other than EnglishImplications for the conduct and reporting of systematic reviews
Lancet
(1996)
A.R Jadad et al.
Assessing the quality of reports of randomized clinical trialsis blinding necessary?
Control Clin Trials
(1996)
M Egger et al.
Language bias in randomised controlled trials published in English and German
Lancet
(1997)
D Moher et al.
Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
Lancet
(1998)
A.D Oxman et al.
Validation of an index of the quality of review articles
J Clin Epidemiol
(1991)
S Yusuf et al.
Beta-blockade during and after myocardial infarctionan overview of the randomized trials
Prog Cardiovasc Dis
(1985)
K Dickersin et al.
Identification of meta-analysesThe need for standard terminology
Control Clin Trials
(1990)
K.S Khan et al.
Empirical evidence of bias in infertility researchOverestimation of treatment effect in crossover trials using pregnancy as the outcome measure
Fertil Steril
(1996)
J.A Berlin
Does blinding of readers affect the results of meta-analyses?
Lancet
(1997)

C.M.J.E Vandenbroucke-Grauls et al.

Effect of selective decontamination of the digestive tract on respiratory tract infections and mortality in the intensive care unit

Lancet

(1991)

A.R Jadad et al.

Meta-analyses to evaluate analgesic interventionsa systematic qualitative review of their methodology

J Clin Epidemiol

(1996)

A Vickers et al.

Do certain countries produce only positive results? A systematic review of controlled trials

Control Clin Trials

(1998)

D Moher et al.

Assessing the quality of randomized controlled trialsimplications for the conduct of meta-analyses

Health Technology Assessment

(1999)

A.R Jadad et al.

Methodology and reports of systematic reviews and meta-analysesA comparison of Cochrane reviews with articles published in paper-based journals

JAMA

(1998)

Cited by (385)

Prevalence of aggressive care among patients with cancer near the end of life: a systematic review and meta-analysis
2024, eClinicalMedicine
Aggressive care near patients’ end-of-life (EOL) entails limited therapeutic values, high costs, and compromised quality of life (QoL). In this study, we aimed to estimate the global prevalence of aggressive care in patients with cancer and explore potential subgroup differences.
We searched PubMed, Embase, and the Cochrane Library from database inception to Feb 16, 2024. Eligible studies reported the prevalence of aggressive EOL care using at least one quantifiable measure. Random-effects models were used to derive the pooled prevalence and subgroup analyses were performed to investigate differences in the prevalence of aggressive care across regions, the country's level of economic development, tumor types, ages, and sample sizes. This review is registered with PROSPERO, number CRD42023467839.
A total of 129 studies were included in this systematic review, of which 118 (91.5%) were from high-income countries. Studies were mostly conducted in the Americas (60, 46.5%), Europe (34, 26.4%), and Western Pacific (31, 24.0%). Measures of aggressive care were inconsistent across studies, with the most commonly used measure being the use of chemotherapy in the last 14 days of life (DOLs) (n = 87, 67.4%) and intensive care unit (ICU) stay in the last 30 DOLs (n = 87, 67.4%). The prevalence of the five claims-based measures of aggressive care, i.e., chemotherapy in the last 14 DOLs, ICU stay in the last 30 DOLs, repeated hospital admission in the last 30 DOLs, repeated emergency room (ER) visit in the last 30 DOLs, and hospice care <3 days before death were 11.6% (95% CI, 9.8%–13.4%), 14.4% (95% CI, 11.8%–17.0%), 17.9% (95% CI, 14.4%–21.4%), 14.8% (95% CI, 12.0%–17.6%), and 14.4% (95% CI, 11.2%–17.6%), respectively. Regional differences were statistically significant in the prevalence of ICU stay and repeated hospital admission in the last 30 DOLs (p < 0.01; p = 0.03). Patients with hematologic malignancies were more likely to receive aggressive care than those with solid tumors, as seen in their higher rates of chemotherapy in the last 14 DOLs (21.7% versus 11.6%; p = 0.03), ICU stay in the last 30 DOLs (25.5% versus 10.8%; p < 0.01), and hospice care <3 days before death (26.7% versus 14.2%; p < 0.01). In addition, the prevalence of chemotherapy in the last 14 DOLs (26.2%; p < 0.01) and repeated hospital admission in the last 30 DOLs (31.4%; p < 0.01) were highest among pediatric patients with cancer.
This meta-analysis found that aggressive EOL care was common in patients with cancer, regardless of the definition used, and varied by regions and populations. It is necessary to be aware of the global burden of aggressive care for patients with cancer near their EOL and take prompt action to address it.
National Natural Science Foundation of China (Grant No. 72274004).
Follow up after surgery for colorectal liver metastases: A systematic review
2023, European Journal of Surgical Oncology
Recurrence post hepatectomy for colorectal liver metastases (CRLM) occurs in 70 % of patients within two years. No established guidance on the method or intensity of follow-up currently exists. The aim of this systematic review was to summarise literature and determine whether it is possible to identify an optimal follow up regime. To this date there are no randomised prospective studies investigating this.
A systematic review was performed according to PRISMA guidelines. Outcomes included general demographics, method, frequency and duration of follow up, survival and recurrence data. Quality assessment of the papers was performed.
Twenty-five articles published between 1994 and 2022 were included, including 9945 patients. CT was the most common imaging modality (n = 14) and CEA most common blood test (n = 11). Intensity of follow up was higher in the first two years post resection and only two papers continued follow up post 5 years resection. There was wide variation in outcome measures - Overall survival (OS) was most commonly reported. Nine papers reported OS ranging between 39 and 78.1 %.
There is wide variation in follow up methods and outcome reporting. There is no strong evidence to support intensive follow up, and the benefits of long term follow up are also unknown due to the lack of patient centred data. High quality, prospective studies should be the focus of future research as further retrospective data is unlikely to resolve uncertainties around optimal follow up.
Effects of urbanisation on PM<inf>2.5</inf> concentrations: A systematic review and meta-analysis
2023, Science of the Total Environment
While urbanisation greatly improves a population's quality of life, it also has significant effects on urban air pollution. Previous studies have determined how urbanisation affects PM_2.5 concentrations; the findings, however, have not been consistent. This study conducts a meta-analysis to systematically organise existing research and draw more conclusive and broadly applicable results regarding the impact of different factors of urbanisation on PM_2.5 concentrations. The main research findings are as follows: (1) the Environmental Kuznets Curve (EKC) is proven to hold true in terms of the effect of population and land urbanisation on PM_2.5 concentrations, while there is no consistent conclusion on the non-linear relationship between economic urbanisation and PM_2.5 concentrations; (2) publication bias is evident in research on the economic and comprehensive urbanisation dimensions under linear assumptions; (3) there are notable heterogeneities in existing research in this field. The meta-regression model further indicates that model design, sample design, and publication characteristics might be responsible for these heterogeneities. This study innovatively applies a meta-analysis to investigate the effect of urbanisation on PM_2.5 concentrations. The findings will contribute to scholars designing more rigorous research frameworks in this field.
Prevalence of human papillomavirus DNA and p16<sup>INK4a</sup> positivity in vulvar cancer and vulvar intraepithelial neoplasia: a systematic review and meta-analysis
2023, The Lancet Oncology
Human papillomavirus (HPV) DNA and p16^INK4a positivity have crucial roles in the pathogenesis of vulvar cancer and vulvar intraepithelial neoplasia. We aimed to examine the pooled prevalence of HPV DNA and p16^INK4a positivity in vulvar cancer and vulvar intraepithelial neoplasia worldwide.
In this systematic review and meta-analysis, we searched PubMed, Embase, and the Cochrane Library databases for studies published between Jan 1, 1986, and May 6, 2022, that reported the prevalence of HPV DNA, or p16^INK4a positivity, or both, in histologically verified vulvar cancer or vulvar intraepithelial neoplasia. Studies on a minimum of five cases were included. Study-level data were extracted from the published studies. Random effect models were used to examine the pooled prevalence of HPV DNA and p16^INK4a positivity in both vulvar cancer and vulvar intraepithelial neoplasia, which were further investigated using stratified analyses by histological subtype, geographical region, HPV DNA or p16^INK4a detection method, tissue sample type, HPV genotype, publication year, and age at diagnosis. Additionally, meta-regression was applied to explore sources of heterogeneity.
We retrieved 6393 search results, of which 6233 were excluded for being duplicates or after application of our inclusion and exclusion criteria. We also identified two studies from manual searches of references lists. 162 studies were eligible for inclusion in the systematic review and meta-analysis. The prevalence of HPV in vulvar cancer (91 studies; n=8200) was 39·1% (95% CI 35·3–42·9) and in vulvar intraepithelial neoplasia (60 studies; n=3140) was 76·1% (70·7–81·1). The most predominant HPV genotype in vulvar cancer was HPV16 (78·1% [95% CI 73·5–82·3]), followed by HPV33 (7·5% [4·9–10·7]). Similarly, HPV16 (80·8% [95% CI 75·9–85·2]) and HPV33 (6·3% [3·9–9·2]) were also the most two predominant HPV genotypes in vulvar intraepithelial neoplasia. The distribution of type-specific HPV genotypes in vulvar cancer among geographical regions was different, with HPV16 varying between regions, showing a high prevalence in Oceania (89·0% [95% CI 67·6–99·5]) and a low prevalence in South America (54·3% [30·2–77·4]). The prevalence of p16^INK4a positivity in patients with vulvar cancer was 34·1% (95% CI 30·9–37·4; 52 studies; n=6352), and it was 65·7% (52·5–77·7; 23 studies; n=896) in patients with vulvar intraepithelial neoplasia. Furthermore, among patients with HPV-positive vulvar cancer, p16^INK4a positivity prevalence was 73·3% (95% CI 64·7–81·2), compared with 13·8% (10·0–18·1) in HPV-negative vulvar cancer. The prevalence of double positivity for HPV and p16^INK4a was 19·6% (95% CI 16·3–23·0) in vulvar cancer and 44·2% (26·3–62·8) in vulvar intraepithelial neoplasia. Most analyses had large heterogeneity (I²>75%).
The high prevalence of HPV16 and HPV33 in vulvar cancer and vulvar intraepithelial neoplasia emphasised the importance of nine-valent HPV vaccination in preventing vulvar neoplasm. Additionally, this study highlighted the potential clinical significance of double positivity for HPV DNA and p16^INK4a in vulvar neoplasm.
Taishan Scholar Youth Project of Shandong Province, China.
Risk factors for post sternotomy wound complications across the patient journey: A systematised review of the literature
2022, Heart and Lung
Around 36,000 cardiac operations are undertaken in the United Kingdom annually, with most procedures undertaken via median sternotomy. Wound complications occur in up to 8% of operations, with an associated mortality rate of around 47% in late or undetected cases.
To undertake a systematised literature review to identify pre-operative, peri-operative and post-operative risk factors associated with sternal wound complications.
Healthcare databases were searched for articles written in the English language and published between 2013 and 2021. Inclusion criteria were quantitative studies involving patients undergoing median sternotomy for cardiac surgery; sternal complications and risk factors.
1360 papers were identified, with 25 included in this review. Patient-related factors included: high BMI; diabetes; comorbidities; gender; age; presenting for surgery in a critical state; predictive risk scores; vascular disease; severe anaemia; medication such as steroids or α-blockers; and previous sternotomy. Peri-operative risk increased with specific types and combinations of surgical procedures. Sternal reopening was also associated with increased risk of sternal wound infection. Post-operative risk factors included a complicated recovery; the need for blood transfusions; respiratory complications; renal failure; non-diabetic hyperglycaemia; sternal asymmetry and sepsis.
Pre, peri and post-operative risk factors increase the risk of sternal wound complications in cardiac surgery. Generic risk assessment tools are primarily designed to provide mortality risk scores, with their ability to predict risk of wound infection questionable. Tools that incorporate factors throughout the operative journey are required to identify patients at risk of surgical wound infection.
A systematic review of post-pancreatectomy haemorrhage management stratified according to ISGPS grading
2022, HPB
Morbidity and mortality from post-pancreatectomy haemorrhage (PPH) remains high. The International Study Group of Pancreatic Surgery (ISGPS) published guidelines to standardise definitions of PPH severity, management and reporting. This study aimed to i) identify the number of studies reporting PPH using ISGPS guidelines (Grade A, B or C) and ii) describe treatment modality success by grade.
A systematic literature review was performed, identifying studies reporting PPH by ISGPS Grade and their subsequent management.
Of 62 studies reporting on PPH management, 17 (27.4%) stratified by ISGPS guidelines and included 608 incidences of PPH: 48 Grade A, 274 Grade B (62 early, 166 late, 46 unspecified) and 286 Grade C.
96% of Grade A PPH were treated conservatively. Of 62 early Grade B, 54.8% were managed conservatively and 37.1% surgically. Late Grade B were managed non-operatively in 25.3% (42/166), with successful endoscopy in 90.9% (10/11) and angiography in 90.3% (28/31). In Grade C, endoscopic treatment was successful in 64.4% (29/45) and angiography in 90.8% (108/119). Surgical intervention was required in 43.5% early Grade B, 7.8% late Grade B and 33.2% Grade C.
PPH grading is underreported and despite guidelines, inconsistencies remain when using definitions and reporting of outcomes.

View all citing articles on Scopus

View full text

Original articlesWhat contributions do languages other than English make on the results of meta-analyses?

Abstract

Introduction

Section snippets

Selection of meta-analyses

Between meta-analyses comparisons

Discussion

Acknowledgements

J Clin Epidemiol

Lancet

Control Clin Trials

Lancet

Lancet

J Clin Epidemiol

Prog Cardiovasc Dis

Control Clin Trials

Fertil Steril

Lancet

Lancet

J Clin Epidemiol

Control Clin Trials

Assessing the quality of randomized controlled trialsimplications for the conduct of meta-analyses

Health Technology Assessment

Methodology and reports of systematic reviews and meta-analysesA comparison of Cochrane reviews with articles published in paper-based journals

JAMA

Original articles
What contributions do languages other than English make on the results of meta-analyses?