#MEAction's NIH Study Response

NIH’s recent paper on ME/CFS elicited strong reactions from the community.  While we recognize the incredible sacrifice of the patient community who gave their health and time to make this study possible, ultimately, we caution against drawing any sweeping conclusions from this study. 

The paper draws conclusions based on an atypical cohort that may not be representative of the ME/CFS community, and from a very small sample size, and there are also conclusions drawn in regards to “effort preference” that may not be supported by the evidence.  

Read #MEAction’s analysis below:

BACKGROUND

Recently, the NIH released a long-awaited study on people living with ME/CFS. The study was many years in the making, and required the advocacy of many to happen, including Beth Mazur, #MEAction’s co-founder. The participants donated so much time and energy, which is a genuine sacrifice in people living with ME/CFS. #MEAction wants to thank and acknowledge everyone who participated and everyone who offered to participate.

It’s likely that the National Academy of Medicine report, released just before the study began, provided the push for NIH to proceed. In late 2015, when the study was announced, members of the community responded with excitement and trepidation. Many stakeholders found the methodology and the choice of lead investigators troubling.

In early 2016, the USAWG (USA Working Group) solicited community input, and #MEAction summarized stakeholder input into specific questions and concerns about the study to be addressed by NIH. Roughly, these questions fell into one of the following categories:

• Selection of study personnel: some researchers held beliefs about chronic illnesses that were unlikely to map to a biomedical framework

• Selection of comparator groups: comparator populations included other diseases that are likewise not well understood, rather than a clear, well-characterized cohort

• Study size: The original plan for n=40 patients was considered too small to justify the time and cost

• Subject selection: There were questions around inclusion/exclusion criteria and confusion around subject selection in general

Three studies utilizing this data have now been published:

• 2019, Myalgic encephalomyelitis/chronic fatigue syndrome patients exhibit altered T cell metabolism and cytokine associations

• 2020, Characterization of Post–exertional Malaise in Patients With Myalgic Encephalomyelitis/Chronic Fatigue Syndrome

• 2024, Deep phenotyping of post-infectious myalgic encephalomyelitis/chronic fatigue syndrome 

SELECTION CRITERIA AND ITS EFFECT ON SAMPLE SIZE

We can’t examine the results before we discuss inclusion/exclusion criteria. What made it appropriate for a patient to be part of this study?  What eliminated a patient from consideration?

One of the inclusion/exclusion criteria was that a number of common comorbidities had to be ruled out. In so doing, researchers sought to identify a ‘pure ME/CFS’ phenotype. However, the majority of patients—over 80%, according to one study with over a thousand ME/CFS patients—have at least one (diagnosed) comorbid condition. The actual number of people with ME/CFS who have a comorbid condition may be even higher, given that clinicians may misattribute new symptoms to pre-existing diagnoses—especially if the initial diagnosis is the subject of stigma, as in ME/CFS. This is sometimes known as diagnostic overshadowing.  

NIH researchers didn’t stop at ruling out people with pre-existing diagnoses: they also tested subjects to determine if there were any undiagnosed comorbid conditions.

They also chose to only examine patients in the first five years of illness. The Hornig et al. study shows that patterns of cytokines, markers of immune activity, significantly change at around three years. That study was also published in 2015. It’s unclear why five years was chosen as the cutoff point.

As a result, hundreds of potential ME/CFS participants were narrowed down to seventeen. Isolating such a small and rarefied fraction of potential participants could mean researchers did not identify a group of patients who are representative of the whole; this is known as selection bias. It’s important to recognize that the paper draws conclusions based on this small sample size and an atypical cohort.

SELECTION BIAS MAY AFFECT RECOVERY CLAIMS

Claims of recovery have to be viewed in the context of this potential selection bias. Given the 5% recovery rate identified by Cairns et al., four out of seventeen, or ~24% of participants experiencing ‘spontaneous recovery’ is remarkable. Certainly these were earlier-stage patients, and the claim has been made that recovery early on is more likely, though this is far from a universal finding. However, there are multiple indicators that NIH selected for patients who were more likely to recover:

• Multiple studies have pointed out that more severely-affected patients have more comorbidities, as do patients who have been ill longer. By identifying patients with solely one diagnosis, NIH selected for people who may have a milder presentation of ME/CFS than average. Perhaps milder cases are more likely to recover, or ‘functionally’ recover—that is, to have so little impairment that they wouldn’t meet diagnostic criteria anymore, even if they did still experience symptoms.

• NIH’s study required overnight stays, exercise testing, and numerous physiologically stressful undertakings. It’s likely that only moderately-ill patients could complete these tasks without long-term loss of functional capacity.  

• Participation in these studies required a certain degree of access to resources: access to the diagnosis itself, knowledge of the NIH study, the resources needed to travel to NIH, and the time taken to undergo each of the study procedures, the resources to take off of work or school to participate. This may also mean access to specialist clinicians, drugs being trialed, and that rarest of commodities: rest. All of these may also affect the likelihood of recovery.

Finally, characterizing these recoveries as ‘spontaneous’ is likely an over-interpretation of data available to the researchers regarding the nature of these patients’ recovery or improvement.

DID THE STUDY REQUIRE SUBJECTS HAVE PEM?

Yes. But readers’ confusion is understandable.

The paper states that patients had to meet Canadian Consensus, Institute of Medicine or Fukuda criteria. Those who are familiar with these criteria may be aware that each includes PEM as a potential feature, but Fukuda does not require it. However, not only did patients also have to be screened by a panel of expert clinicians, including some very familiar faces, and receive unanimous approval from each of the clinicians on the panel that the patient had ME/CFS, Avindra Nath has stated multiple times that every patient must have PEM to participate. In other words, if a subject met Fukuda criteria, they still had to have PEM. This is confirmed in Supplement 1 in the study materials.

ABOUT "EFFORT PREFERENCE"

Effort preference is a term used primarily in psychology that essentially means ‘willingness to exert’. It should go without saying that people with ME/CFS are leery of over-exertion and likely have lower effort preference than healthy controls; that would most certainly be the case in many disabilities. It goes without saying to the extent that it seems like a very strange thing to bother to measure. The paper at least acknowledges this is likely due to pacing.

Effort preference is studied in mental illness, though whether it should be is another question. However, it’s so rare to measure effort preference in neurological disease—save Parkinson’s, where dopamine, deeply intertwined with effort/reward dynamics is central—that when I searched all I found were mentions of this study.

Yet this finding in particular is central to the paper’s argument: 

“Among the many physical and cognitive complaints, one defining feature of PI-ME/CFS was an alteration of effort preference, rather than physical or central fatigue, due to dysfunction of integrative brain regions potentially associated with central catechol pathway dysregulation, with consequences on autonomic functioning and physical conditioning.”

Being explicit in this particular case: the evidence in the paper does not support this. It goes out on a very shaky limb.

Brain regions aren’t stamped with singular and particular purpose. The right temporoparietal junction does more than one thing. However, there are some aspects of its function that have been well-studied: attention, and understanding social cues. If that region of the brain engaged as a subject worked on a task, the most likely hypothesis would be that they had difficulty focusing on said task: a central feature of disease presentation in ME/CFS. However, this would be a tentative conclusion, because every part of the brain does more than one thing. Such findings would be couched in the language of uncertainty.

CONCLUSIONS AND THE WAY FORWARD

The study was incredibly thorough and the tests performed were varied. It’s interesting to see so many different kinds of testing performed. A few highlights:

• ACEs (adverse childhood events) were comparable in people with ME/CFS and healthy controls. Decades ago it was a popular claim that many chronic illnesses were conversion disorder due to childhood trauma; this paper didn’t support that conclusion.

• Evidence of immune dysregulation in terms of naive B cells and of T cell activation, as found in other studies; evidence of metabolomic changes including butyrate levels, as found in other studies. It’s good likewise to have replication of autonomic findings in people with ME.

• The authors’ suggestion of immune checkpoint inhibitors for people with ME/CFS is promising.

But the positive aspects of the work do have to be understood in the context of its limitations.

NIH spent $8M to gather data from seventeen subjects, chosen in a way that may bias their conclusions—and they structured those conclusions around a core of psychosomatic reasoning. One immediate way to take action is by working on the ME/CFS Research Roadmap research priorities. The public comment period closes on March 8th. 

How NIH acts on this community input will matter even more in the light of this recent paper.