In this paper, the authors conducted a meta-analysis which demonstrated that the risk of revision was increased with cementless fixation of total knee arthroplasties when compared with cemented ones. In recent years with the rapid accumulation of medical information meta-analysis has gained popularity and the number of such studies published in medical journals has dramatically increased. Meta-analysis is a quantitative approach for systematically combining the results of previous research in order to substantiate further conclusions. Therefore, the influence of primary studies and particularly study design is of crucial importance. In this paper, the authors combined observational and randomised studies. This choice is debatable and the reader should be aware of the strengths and weaknesses of this approach.
Considering primary studies, observational and randomised controlled trials have both strength and weaknesses and their respective merits have been extensively debated over the last 10 years. It is generally admitted that observational studies are subject to more potential sources of bias than randomised trials. This is generally attributed to selection bias introduced by authors when deciding whom to treat and to an enhance response to treatment preference among those with strong preference for a particular treatment. Randomised controlled trials have restricted eligibility criteria that may raise doubts on external validity. These criteria may affect patients, disease, care givers and dramatically restrict the generalisability of the results.
Inclusion of non-randomised studies in meta-analysis remains debatable. This can be a source of discordant results.1 In a landmark paper Benson and Hartz2 showed that observational and randomised studies reached the same conclusions and did not differ in estimates of the effect. Shrier et al3 showed in a random sample of meta-analysis that six of the 16 reviews included two studies or fewer; furthermore 158 of the 183 analyses conducted in seven additional studies were limited to two or fewer studies. They advocated that inclusion of observational studies was a way to add information in such cases and may aid in clinical reasoning. However, in a recent study Kunz, Vist and Oxman4 performed a sytematic review of meta-analyses that compared randomised versus non-randomised trials, randomised trials with adequately versus inadequately concealed allocation, or high versus low quality trials where selection bias could not be separated from other sources of bias. Among the 22 studies that compared randomised and non-randomised studies 15 showed important differences in estimates of effect. On average, non-randomised studies and randomised studies with inadequate concealed allocation result in overestimates of effect. But, this bias can go in either direction, can reverse the effect (it is therefore attributed to higher quality care or to greater ability to benefit) or can mask an effect. This uncertainty has led authors to reinforce the role of randomised clinical trials with adequate allocation concealment.
In conclusion, inclusion or not of observational studies in a meta-analysis has to be carefully weighed during the design process of a meta-analysis. If it is decided to do so, the study design should be explored as an explanatory variable. If the study design is associated with a difference in the estimate of the effect then the potential reasons for discrepancies have to be listed and discussion about the probabilities, direction and magnitude of the bias should be conducted including the pros and cons of each study design. From a clinical point of view only a thorough analysis of the review, including all the aspects described above, will help to decide to whether the conclusion of a review can be applied to clinical practice.
Nizard R, Professor