Do people really recover as a result of CBT and graded exercise? Sadly not, says a new analysis of PACE
People with ME/CFS dream of recovery — of getting their health and lives back. So the results published by the PACE trial, showing that 22% of participants recovered after just 12 months of CBT or graded exercise, looked like good news for patients. For those in the control group with no therapy, only 7% recovered.
But alas, “recovery”, as defined by PACE, is not what it seems. It’s not even what the authors originally said it would be in the study protocol. And a new paper (of which I’m a co-author) concludes that the changes made to the definition of recovery were not justified and led to the publication of recovery rates that were inflated and misleading, to patients and clinicians alike.
In fact, reanalysis according to the original, protocol-specified definition of recovery shows that CBT and graded-exercise recovery rates were not significantly better than having no therapy at all.
The new paper is by psychology researcher Dr Carolyn Wilshire, patient-researchers Tom Kindlon and Alem Matthees, and me. It lays bare how the PACE authors watered down the recovery criteria they had specified before the trial began; and, using data recently released after a Freedom of Information (FOI) battle by Alem Matthees, presents analyses to reveal how those changes artificially boosted recovery rates.
Defining recovery
To count as “recovered” in the PACE trial, patients had to meet criteria in four areas. Two were for self-rated physical function and fatigue — the trial’s primary outcomes. Patients also had to rate their health as substantially improved overall. Finally, they had to be judged as no longer meeting a case definition for CFS. However, the PACE authors relaxed the thresholds for each of the four criteria so that in the published paper, it was much easier for patients to count as “recovered” than if the protocol-specified criteria had been used.
The graph below shows just how much difference these changes made:
Here’s how it was done, and how PACE’s revised version describes something less than recovery:
1. Recovery of self-reported physical function
To determine recovery of physical function, PACE used the SF-36 physical function subscale (see items 3-12), which asks participants to rate activities of daily living from self-care through to running and gives an overall score ranging from 0 (highly disabled) to 100 (functioning well). 90% of healthy adults of working age score 90 or more, and the PACE protocol defines recovery of physical function as a score of 85 or more.
By the time the authors published their recovery paper, though, they’d lowered that threshold to just 60. This is bizarre, given that PACE used a higher score (65) to accept patients as disabled enough to join the trial in the first place! Incredibly, 13% of patients had already “recovered” physical function before they’d had any therapy at all. The new “recovery” threshold of 60 is close to the average SF-36 scores for patients with rheumatoid arthritis and congestive heart failure.
Analysis of the FOI data showed that lowering the SF-36 threshold increased the proportion of those “recovering” physical function more than threefold, from 14% to 45%.
How on Earth could the PACE authors justify such a change? In their recovery paper, they wrote that the original threshold of 85 was so high that “approximately half the general working age population would not meet it”. The new paper points out this isn’t correct. The population data they referred to was for all adults, of whom almost a third were aged over 60 and one in five had a chronic illness or disability. PACE participants were almost all aged under 60, and anyone with a fatiguing illness apart from ME/CFS was excluded from the trial, so the comparison population PACE used was inappropriate. But even using an inappropriately elderly and sick population, the maths is still wrong: only 28% scored under 85, not “approximately half”.
2. Recovery from fatigue
Fatigue was measured by the Chalder Fatigue Questionnaire, which asks eleven different questions related to fatigue, such as “Do you need to rest more?” The questionnaire can be scored as an 11-point scale (as specified in PACE’s protocol) or as a 33-point scale, with a complex relationship between the two. Recovery was originally defined as a score of 3/11 or less, but later changed to 18/33 or less. This boosted recovery from fatigue from 15% according to the protocol, to 29% in the published results.
Again, as with the physical function criterion, the fatigue threshold was relaxed so far that some patients (seven) had already ‘recovered’ from their fatigue when they joined the trial, even though, at that point, they were required to have fatigue that was “disabling”.
The PACE authors justified relaxing this threshold with data from a paper that also inappropriately included a third who were unwell, some of them with fatiguing illnesses — including CFS itself.
3. Overall change in health
At the end of the trial patients rated how their health had changed since the start, with options ranging from “very much worse” through to ”very much better”. The protocol classed “very much better” as defining recovery on this measure — and this maximum score seems appropriate for patients starting from a level of severe, disabling fatigue and then recovering. However, the PACE recovery paper relaxed the definition of recovery to also include patients who were merely “much better”. This increased the proportion “recovering” on this measure from 12% to 34%.
The study authors said that “participants rating their overall health as ’much better’ represented the process of recovery”. Improvement, however, is not recovery.
4. No longer meeting CFS case definition
At the end of the trial, medical staff decided if each participant still met the Oxford case definition for CFS that was used to recruit patients into the trial. 24% no longer did so and thus, according to the protocol, these patients met the fourth and final requirement for recovery.
Yet again, the PACE authors relaxed the threshold, by adding conditions to the case definition. Now, a patient could be considered “recovered” even if they still met the Oxford criteria as used in the clinic, as long as they had even slightly better levels of fatigue or physical function than the disabling levels required for patients to be eligible to join the trial. This meant that very ill sick patients, who would normally be in treatment for CFS, could be classed as “recovered”. This change doubled the proportion not meeting the Oxford criteria — that is, “recovered” — to 48%.
The new “recovery”: not getting your health back
To recap, PACE relaxed the fatigue and physical function recovery thresholds so far that patients could (and some did) count as recovered at the same time as being fatigued and disabled enough to join the trial. They counted patients as recovered who said they weren’t “very much better”, but merely “much better”, and even if clinical staff judged them to still meet Oxford criteria for CFS. This is not recovery.
Overall recovery rates
Not surprisingly, the protocol-specified criteria give much lower recovery rates than the ones that PACE published. Recovery rates fall from 22% to 7% for CBT, from 22% to 4% for graded exercise, and from 7% to 3% for no therapy.
Not only that, but the recovery rates for CBT and graded exercise are no longer statistically significantly higher than those for no therapy at all: that is, the trial provides no evidence that patients can recover as a result of CBT and graded exercise.
The changes to PACE’s recovery measures resulted in inflated and misleading “recovery” rates that have nothing to do with patients’ dreams of getting their health back. Patients and clinicians need to have accurate information about the chances of recovery as it’s understood by people in the real world. Our paper reveals that PACE’s own data show that CBT and GET have no effect upon the chances of recovery.
Follow Simon McGrath (@sjmnotes) on twitter
13 thoughts on “The PACE trial: Where “recovery” doesn’t mean getting your health back”
How were patients that did not complete the study accounted for in the original results? For most research with which I’m familiar, the number of people who do not complete are reported, but nothing further is given. I am wondering if a significant number dropped out (especially of the graded-exercise regimen) from getting worse, and if, consequently, these data were excluded altogether even though it would be important information.
I’ve wondered that as well, Anita.
I think drop out rates were pretty low, and when people did drop out, they made decent efforts to get final scores from them – it’s one of the strengths of the study. And may account for why PACE results were notably less impressive than earlier trials. I don’t think the drop out rate for GET was significantly higher than the other groups (though more GET participants didn’t complete the six minuue walking test at 52 weeks, perhaps due to more experience of the effects of walking that kind of distance, we dont know).
Important to acknowledge the strengths of the study, such as they were. 😉
Anita, I accept Simon’s reply about completion rates overall. However, in an attempt to explain the poor results of the 6 minute waking test, the authors say (in seemingly complaining tone) that they had data for only 72% of participants, ‘which is less than for the self-report measures.’ They never explain the reasons for the 28% missing data but do mention that they did not want to encourage patients to walk faster, as is apparently done in this test, ‘due to concerns about patients with CFS coping with physical exertion’. Some recovery! This gives a clue as to the extent of it when it comes to the real world.
Fantastic article. Well done Simon and the other authors.
Thanks!
Not only is the outright manipulation of the statistics wholly unacceptable and must surely breach all scientific ethics when conducting research, it is heartbreaking for those of us who have struggled living with this disease for decades. I now hope and pray that all future research will be carried out under stringent regulations with a panel of objective overseers to ensure the work is always undertaken to highest standards.
I am appalled by the medical journals who published the results which resulted in years of mistreatment by patient’s medical consultants, who, without any further evidence only had this trial to guide them.
I hope that the national medical authorities will now retract the treatment protocols before any further damage is caused. How dreadfully sad.
Note how important it is to stress that the analysis proves CBT/GET does NOT work, rather than it merely doesn’t prove it works (but new studies might) as some people maintain to continue their propaganda.
Please leave ME out of it though. PACE is a CFS study, not a ME study.
Hard to even call it that, from my point of view! The Fukuda criteria is a lot stricter than what they’ve been using.
This article was a good analysis of the results of the study. We know from many news articles that their is positive bias in outcome reporting-and since doctors are so poorly trained in statistics or evaluating research- they have assumed too much from cbt and graded exercise studies for cfs/me.
The PACE studies- like s many studies looks at too few variables and is a gross oversimplification. Lets face it the underlying assumption to both reseearch and medical practice is that a fairly simplistic treatment regimen will suffice for people with me cfs( or any other condition for that matter) What is the proof that the best treatment consists of a few treatments? Since few studies have utilized treat t target and used many treatments for cfs me- researchers cannot prove a negative. And so it is clear that researchers are stuck in a reinforcement loop- as are practitioners. It makes no sense to believe a simplistic reductionistic treatment regimen will do much for a chronic condition- and that is probably why chrnic conditions are on the rise. Chemical and nonchemical exposures are on the rise, nutritional deficiencies are on the rise, social stress, etc. And so i dont think well of the siumple minded paradigm bound research of the PACE trial. Its time to rid ourselves of dysfunctional paradigms that make no way for people with chronic conditions like CFS me.
Private research is proving that M E is a physical illness. Abnormalities have been found in the blood cells of patients by Scientists at Aberdeen University (with private funding and they have yet to find funding to do further follow-up research) and various others. How can the medical profession even consider using CBT and GET as the main treatment for M E patients – let alone wasting £5 million on a flawed trial! My poor daughter has had this illness since she was 11 and is now 35, and nothing has gone forward in all these years, It is Disgusting that this illness is still not recognized for what it is – Neurological., and that Doctors are not educated in the severity of the suffering that people go through year after year after year……. Other researchers… Dr Enlander Belfast Dr, and Faculty member of Mount Sinai Medical school, is researching and treating M E patients successfully with antivirals. Prof Jose Montoya Stanford Uni. California has treated patients successfully with anti virals. Dr Theodore Henderson Consultant PSYCHIATRIST in Centennial, Colorado TREATS also with ANTIVIRALS SUCCESSFULY. They all say that M E is a physical disease; and there are more… IN THE FACE OF ALL THIS EVIDENCE how can they CONTINUE with YET ANOTHER CBT/GET TRIAL?! Also It is about time NICE GUIDELINES CHANGED!
At this point, they’ve changed their tune to, “it is a physical illness, but we find that CBT is the way to treat it most effectively.” By claiming mind/body interconnectedness, they are able to say that this is the case no matter what proof is presented for the illness. It’s a very convenient argument because it does not rely on proof. That means that anyone should be able to see through it, but it also means that they never have to argue. Until major drug trials come along to show that CBT is not the ‘best available treatment’, that will continue to be their argument, I fear.
Comments are closed.