PACE is valueless for one reason: the combination of lack of blinding of treatments and choice of subjective primary endpoint. Neither of these alone need be a fatal design flaw but the combination is.
The only possible mitigation of this flaw would be if:
1. There were no acceptable alternatives to a subjective primary outcome measure, and
2. The authors fully understood the nature of the flaw and the reasons why it is so serious, and
3. The authors were meticulous in trying to avoid all forms of bias that are likely to arise from the flaw.
It has been apparent from the outset that alternative objective endpoints were available. These might not be ideal in a blinded study (the blinded Norwegian rituximab study chose a subjective end point for preference) but an objective primary endpoint would have been practical. The objective endpoints in PACE used as secondary measures tended to show no difference in outcome.
The authors (and colleagues) have repeatedly indicated in public that they do not understand the flaw in the PACE trial design, as made clear by claims that the study is ‘robust’ and of high design quality. Moreover, the authors have not been meticulous in trying to avoid bias that might arise. On the contrary they appear to have acted in ways more or less guaranteed to maximise bias. (The details are well documented but are secondary to the main problem.)
The recent follow up study may be of some use in confirming the sensible conclusion to draw from the objective measures used in PACE – that it provides no evidence for any benefit from CBT or GET. It strengthens the case by suggesting that at longer term follow up even the subjective measure differences disappear. It also highlights again the authors’ lack of understanding of trial design problems. Moreover, as outlined by Dr James Coyne, it provides further evidence for the authors’ tendency to continually introduce bias in their analysis, apparently through a lack of understanding of basic data interpretation and statistics. One might expect errors of this sort from a clinical trainee, but not from a professional research team.
I am surprised and troubled that clinical and research colleagues in the ME/CFS field have said so little about the problems with this study. It has been left to David Tuller and James Coyne to raise the issue again. Regardless of other issues relating to the illness (that have been grossly misrepresented in the popular media recently) it needs to be recognised and agreed that the PACE study is not a suitable evidence base for clinical practice guidelines.
Professor Emeritus, Dept Medicine, UCL