What’s wrong in The Lancet

When the PACE trial authors calculated their “normal range” for fatigue and physical function for the Lancet paper, they made methodological mistakes. For example, they ignored the fact that the survey data that they were using to calculate their physical function “normal range” didn’t fit a bell-shaped curve and they used statistical measures — the mean and standard deviation — that shouldn’t be used in that situation. These mistakes led them to calculate that a physical function rating of 60 or above on the well-known 100-point SF-36 questionnaire scale placed patients in the “normal range”.

A score of 60 indicated worse physical function than patients had needed in order to be disabled enough to enter the trial but the authors still used it to define the “normal range” and failed to point out or discuss the contradiction in the paper.

SF-36 physical function scale

100 healthy physical function
65 or below disabled enough to enter the trial
60 or above “normal range” for physical function
57 average score for patients with Class II congestive heart failure
0 most extreme disability

Methodological problems also affected the fatigue scale, including that the study authors didn’t remove patients with fatiguing health problems from the reference sample. And for their new recovery threshold they used a different way of scoring patients’ responses that translated to a score anywhere between 4 and 9 on the 11-point scale, depending on exactly how each patient filled in the questionnaire. Patients had to have scored 6 or worse on the 11-point fatigue scale to qualify for the trial and their fatigue could worsen during the trial up to a score of 9 — close to the maximum — and yet they would be classed as within the “normal range” for fatigue.

But the authors treated the “normal ranges” as though being within them was evidence of good enough health to reflect effective treatment. They wrote:

“No more than 30% of participants were within normal ranges for both outcomes and only 41% rated themselves as much better or very much better in their overall health. We suggest that these findings show that either CBT or GET, when added to SMC, is an effective treatment for chronic fatigue syndrome, and that the size of this effect is moderate.”

An accompanying guest editorial said:

“PACE used a strict criterion for recovery: a score on both fatigue and physical function within the range of the mean plus (or minus) one standard deviation of a healthy person’s score. In accordance with this criterion, the recovery rate of cognitive behavioural and graded exercise therapy was about 30%”.

This claim of “recovery” was widely reported in the media, with headlines such as the Daily Mail’s “Got ME? Fatigued patients who go out and exercise have best hope of recovery, finds study”. The British Medical Journal wrote that 30% of the patients were “cured”.

The Lancet published letters criticising the “normal range” analyses but failed to retract either the findings based on them or the “recovery” claim made in its own editorial.

In response to a complaint, the Press Complaints Commission ruled that The Lancet had “failed to take care not to publish inaccurate or misleading information” but considered that, by publishing letters criticising the paper, The Lancet had done enough to remedy the situation.

We disagree.

The misleading statement in the paper still stands, based upon a ludicrous definition of a “normal range” of fatigue and physical function that includes considerable disability. The misleading claim of recovery still stands in the guest editorial. And the most prominent, top two links — “Editor’s links” — on the Lancet webpage where the PACE paper appears, are to Guardian and CNN articles that repeat the claims of “recovery”.

The Lancet’s own guidelines state that “any substantial error in any article published in The Lancet should be corrected as soon as possible” and that they “will always correct… any factual error in interpretation of results.”

We call upon The Lancet to uphold their own standards and formally retract  all claims based on the “normal range”.


[button_color url=”http://my.meaction.net/petitions/pace-trial-needs-review-now” content=”Sign the PACE petition” target=”http://my.meaction.net/petitions/pace-trial-needs-review-now”]