Quality assessment of studies comparing percutaneous ablative treatments in hepatocellular carcinoma

We appreciate the interest shown by Huang et al. towards our study and we thank them for their letter [1] which points out some aspects in our paper that evidently need further explanation.

They are absolutely right in highlighting the substantial difference in quality assessment between retrospective studies and randomised controlled trials (RCTs), but this issue is widely addressed in Supplementary Table 1 of our meta-analysis where accurate description of all the items assessed for each scale is provided [2]. Huang’s criticism leads us to suspect that he did not have access to the supplementary material of our paper; therefore we report below the aforementioned table for clarity purposes (Table 1).

Table 1. Risk of bias assessment and quality of included studies.

Observational studies^a
	Selection		Comparability		Outcome		Overall quality
Ohmoto, 2009	***		**		**		7
Lu, 2005	**		*		**		5
Ding, 2013	***		**		**		7
Zhang, 2013	***		**		**		7
Abdelaziz, 2014	**		*		**		5
Vogl, 2015	**		**		**		6
Randomised controlled trial^b
	1	2	3	4	5	6	7
Shibata, 2002	L	L	U	U	H	L	L	M

L, low; H, high; U, unclear; M, moderate.

a Study quality assessment performed by means of Newcastle–Ottawa scale (each asterisk represents whether the respective criterion within the subsection was satisfied).

b Cochrane Collaboration’s tool for assessing the risk of bias across 7 domains: 1, Random sequence generation); 2, Allocation concealment; 3, Blinding of participants and personnel; 4, Blinding of outcome assessment; 5, Incomplete outcome data; 6, Selective reporting; 7, Other bias.

According to Cochrane guidelines, to be considered high quality RCTs are required to present low risk of bias for all domains, while a high-quality retrospective study should fulfil at least two criteria for each of the three domains assessed in the Newcastle–Ottawa scale (selection, comparability, and outcome) [3].

Of course we agree with Huang et al. on the superior methodology of RCTs as compared to observational studies, but the ‘equation’ they propose (moderate-quality RCTs = high-quality retrospective studies) is weak and rather debatable and does not find any support in the literature.

The very fact, indeed, that separate scales have been developed for the two settings accounts for the methodological differences between RCTs and retrospective series.

Analysis restricted to the sole available RCT [4] was performed with regard to complete response rate and the odds ratio was not significant (0.36, 95% confidence interval 0.07–1.94, p = 0.23), as reported in our paper. Unfortunately this RCT did not report survival data, therefore a separate analysis of this outcome was not feasible due to missing data.

Finally, we have adequately acknowledged among the limitations of our paper the lack of standardisation of equipment which may restrict direct comparison of the studies included in the meta-analysis; furthermore we clearly stated among the weaknesses that ‘a subgroup analysis based on equipment was not possible due to the small numbers within each treatment group.’ However, we do not consider the imaging technique adopted as guidance of ablative procedure as a real limitation because all the studies except that by Vogl et al. [5] used ultrasound, while in the German series [5] all ablations (in both treatment cohorts) were performed under CT fluoroscopic guidance. Therefore, we do not see how this highly homogeneous aspect could have represented a source of heterogeneity able to affect our results.

Antonio Facciorusso and Nicola Muscatiello
Gastroenterology Unit, Department of Medical Sciences, University of Foggia, Italy
[email protected]