Do people really recover as a result of CBT and graded exercise? Sadly not, says a new analysis of PACE
People with ME/CFS dream of recovery — of getting their health and lives back. So the results published by the PACE trial, showing that 22% of participants recovered after just 12 months of CBT or graded exercise, looked like good news for patients. For those in the control group with no therapy, only 7% recovered.
But alas, “recovery”, as defined by PACE, is not what it seems. It’s not even what the authors originally said it would be in the study protocol. And a new paper (of which I’m a co-author) concludes that the changes made to the definition of recovery were not justified and led to the publication of recovery rates that were inflated and misleading, to patients and clinicians alike.
In fact, reanalysis according to the original, protocol-specified definition of recovery shows that CBT and graded-exercise recovery rates were not significantly better than having no therapy at all.
The new paper is by psychology researcher Dr Carolyn Wilshire, patient-researchers Tom Kindlon and Alem Matthees, and me. It lays bare how the PACE authors watered down the recovery criteria they had specified before the trial began; and, using data recently released after a Freedom of Information (FOI) battle by Alem Matthees, presents analyses to reveal how those changes artificially boosted recovery rates.
To count as “recovered” in the PACE trial, patients had to meet criteria in four areas. Two were for self-rated physical function and fatigue — the trial’s primary outcomes. Patients also had to rate their health as substantially improved overall. Finally, they had to be judged as no longer meeting a case definition for CFS. However, the PACE authors relaxed the thresholds for each of the four criteria so that in the published paper, it was much easier for patients to count as “recovered” than if the protocol-specified criteria had been used.
The graph below shows just how much difference these changes made:
Here’s how it was done, and how PACE’s revised version describes something less than recovery:
1. Recovery of self-reported physical function
To determine recovery of physical function, PACE used the SF-36 physical function subscale (see items 3-12), which asks participants to rate activities of daily living from self-care through to running and gives an overall score ranging from 0 (highly disabled) to 100 (functioning well). 90% of healthy adults of working age score 90 or more, and the PACE protocol defines recovery of physical function as a score of 85 or more.
By the time the authors published their recovery paper, though, they’d lowered that threshold to just 60. This is bizarre, given that PACE used a higher score (65) to accept patients as disabled enough to join the trial in the first place! Incredibly, 13% of patients had already “recovered” physical function before they’d had any therapy at all. The new “recovery” threshold of 60 is close to the average SF-36 scores for patients with rheumatoid arthritis and congestive heart failure.
Analysis of the FOI data showed that lowering the SF-36 threshold increased the proportion of those “recovering” physical function more than threefold, from 14% to 45%.
How on Earth could the PACE authors justify such a change? In their recovery paper, they wrote that the original threshold of 85 was so high that “approximately half the general working age population would not meet it”. The new paper points out this isn’t correct. The population data they referred to was for all adults, of whom almost a third were aged over 60 and one in five had a chronic illness or disability. PACE participants were almost all aged under 60, and anyone with a fatiguing illness apart from ME/CFS was excluded from the trial, so the comparison population PACE used was inappropriate. But even using an inappropriately elderly and sick population, the maths is still wrong: only 28% scored under 85, not “approximately half”.
2. Recovery from fatigue
Fatigue was measured by the Chalder Fatigue Questionnaire, which asks eleven different questions related to fatigue, such as “Do you need to rest more?” The questionnaire can be scored as an 11-point scale (as specified in PACE’s protocol) or as a 33-point scale, with a complex relationship between the two. Recovery was originally defined as a score of 3/11 or less, but later changed to 18/33 or less. This boosted recovery from fatigue from 15% according to the protocol, to 29% in the published results.
Again, as with the physical function criterion, the fatigue threshold was relaxed so far that some patients (seven) had already ‘recovered’ from their fatigue when they joined the trial, even though, at that point, they were required to have fatigue that was “disabling”.
The PACE authors justified relaxing this threshold with data from a paper that also inappropriately included a third who were unwell, some of them with fatiguing illnesses — including CFS itself.
3. Overall change in health
At the end of the trial patients rated how their health had changed since the start, with options ranging from “very much worse” through to ”very much better”. The protocol classed “very much better” as defining recovery on this measure — and this maximum score seems appropriate for patients starting from a level of severe, disabling fatigue and then recovering. However, the PACE recovery paper relaxed the definition of recovery to also include patients who were merely “much better”. This increased the proportion “recovering” on this measure from 12% to 34%.
The study authors said that “participants rating their overall health as ’much better’ represented the process of recovery”. Improvement, however, is not recovery.
4. No longer meeting CFS case definition
At the end of the trial, medical staff decided if each participant still met the Oxford case definition for CFS that was used to recruit patients into the trial. 24% no longer did so and thus, according to the protocol, these patients met the fourth and final requirement for recovery.
Yet again, the PACE authors relaxed the threshold, by adding conditions to the case definition. Now, a patient could be considered “recovered” even if they still met the Oxford criteria as used in the clinic, as long as they had even slightly better levels of fatigue or physical function than the disabling levels required for patients to be eligible to join the trial. This meant that very ill sick patients, who would normally be in treatment for CFS, could be classed as “recovered”. This change doubled the proportion not meeting the Oxford criteria — that is, “recovered” — to 48%.
The new “recovery”: not getting your health back
To recap, PACE relaxed the fatigue and physical function recovery thresholds so far that patients could (and some did) count as recovered at the same time as being fatigued and disabled enough to join the trial. They counted patients as recovered who said they weren’t “very much better”, but merely “much better”, and even if clinical staff judged them to still meet Oxford criteria for CFS. This is not recovery.
Overall recovery rates
Not surprisingly, the protocol-specified criteria give much lower recovery rates than the ones that PACE published. Recovery rates fall from 22% to 7% for CBT, from 22% to 4% for graded exercise, and from 7% to 3% for no therapy.
Not only that, but the recovery rates for CBT and graded exercise are no longer statistically significantly higher than those for no therapy at all: that is, the trial provides no evidence that patients can recover as a result of CBT and graded exercise.
The changes to PACE’s recovery measures resulted in inflated and misleading “recovery” rates that have nothing to do with patients’ dreams of getting their health back. Patients and clinicians need to have accurate information about the chances of recovery as it’s understood by people in the real world. Our paper reveals that PACE’s own data show that CBT and GET have no effect upon the chances of recovery.
Follow Simon McGrath (@sjmnotes) on twitter