Archived Comments for:
The feasibility and acceptability of conducting a trial of specialist medical care and the Lightning Process in children with chronic fatigue syndrome: feasibility randomized controlled trial (SMILE study)
The validity of using subjective outcome measures as primary outcomes is questionable in such a trial
Tom Kindlon, Irish ME/CFS Association
29 July 2014
Crawley and colleagues suggest dropping school attendance as a primary outcome from the full study, and replacing it with self-report outcomes "such as the SF-36 or the Chalder Fatigue Scale" and using "school attendance as a secondary outcome"(1). And indeed, this is what they have done with the full study which has two primary outcome measures: "Chalder Fatigue Scale at 6 months" and "SF 36 physical function short form at 6 months"(2). I question the wisdom of using self-report measures as primary outcomes for such a trial.
The authors do not give much information about the details of the Lightning Process (they do mention it is "developed from osteopathy, life coaching and Neuro-linguistic programming (NLP))" but here are some descriptions from other sources (individuals who have undertaken a course) (note these are not descriptions from participants from the trial itself):
(a) "It felt very naughty but I whispered to one of the woman (sic) sitting next to me 'how are you, is this working for you?'. She was reluctant to answer, to say anything but that she was doing well would be to go against the process because that is a negative thought. It was pointless asking really. Still I wanted it to work, but I was starting to worry about the fact that I was not only not feeling any better the effort of doing the course, not getting my normal rest was making me feel worse. But these were negative thoughts. I started to ruthlessly suppress them like I had been shown." (3)
(b) ¿They tell you that you're not allowed to say that you're ill anymore or that you have any symptoms. They force you to ignore the symptoms because they say that the symptoms don't really exist. They force you to do activities even though it's making you really ill, but you're not allowed to say so.¿ (4)
Another, more thorough, description can be found on the Skeptic's Dictionary website(5).
I have seen many similar descriptions to these in discussions in many private fora. Here's one example: "So the first step in the process is to recognise when you¿re in THE PIT. Maybe sometimes or all the time. It¿s important to recognise what you say to yourself as you go into the pit. For example ¿I¿m feeling really ill this morning¿, ¿if I do this, then I¿ll get exhausted¿, ¿last time I did this I got really ill for days¿, ¿I can never eat this¿ etc. This takes some practise but we were assured that you always say something in your head as you go into the pit. As soon as you spot one of your ¿pit¿ phrases you want to STOP yourself right away. So imagine you¿re on the mat and you start to say ¿I feel really ill today¿. Before you get to the end of this phrase you will interrupt with a very firm, loud ¿STOP¿ (yes, talk out loud to yourself!) and jump into the STOP position as described above. So you jump outside the mat, to your left. Now you¿re here you have interrupted your bad thought patterns."
In essence when one is doing the lightning process, both during the three days training and after, you are to declare yourself well. You are not to say (or think) you have symptoms and you are not to say (or think) you have limitations. In other words, patients are 'trained' to dismiss and deny their symptoms and illness. Patients are instructed not to ¿do¿ ME or CFS anymore. In such a scenario, subjective reports from the patient are no longer reliable. Hence the need for objective measures in such trials.
To a lesser extent, it can also be argued that subjective measures are not ideal for all participants in this trial, including the control group. With graded activity-oriented therapies, which all the participants undertake (1), there may be response bias (6). This was seen in a review of three Dutch trials of graded activity-oriented cognitive behaviour therapy (CBT) for CFS. While the CBT participants reported improvements in fatigue (and also SF-36 physical functioning although that was not measured in all of the trials), no improvements in objectively measured activity were reported over the control group(7). Similarly a mediation analysis showed changes in physical activity were not related to changes in fatigue. A similar effect with CBT could be in PACE Trial, the largest such trial in the CFS field(8). Although participants who had undergone CBT reported improvements in scores on both the Chalder Fatigue Scale and SF 36 physical function subscale, no improvements were reported in the six minute walking distance compared to the group who had undergone no additional individual therapy (all participants had specialist medical care). Similarly, CBT did not significantly reduce employment losses, overall service costs, welfare benefits or other financial payments(9).
Objective measures of physical activity are one type of objective measurement that could be employed. This could be used not just to measure the total quantity of activity but also to check the intensity of activity as people with ME/CFS may perform lower levels of more intensive activity(10). Tests of neuropsychological performance could also be used, given the impairments that have previously been reported(11). If necessary, a webpage could be created that could be accessed remotely.
One particular problem with the 0-33 scoring method for the Chalder fatigue scale is that patients can give unusually low, or artificially good, scores. The scale consists of 11 questions. For each question (e.g. "Do you have less strength in your muscles?"), patients have to say whether they have the symptom: "Less than usual" (score of 0); "No more than usual" (score of 1); "More than usual" (score of 2) or ¿Much more than usual¿ (score of 3). So healthy people should score around 11 and indeed in the only population study I can recall that gave data for a healthy (no disease/current health problem) group, the mean was 11.2 (12).
However, some unusual scoring is possible. This was seen in a multiple sclerosis trial, of a similar CBT intervention to the graded activity-oriented CBT used in CFS (13). Participants in the CBT leg of the trial entered with a Chalder fatigue scale score of mean (sd): 20.94 (4.25). Two months after therapy, the mean (sd) was 7.90 (4.34) i.e. this groups suffering from multiple sclerosis had a much better score than one sees in healthy people!
Such averages could also hide non-responders or, possibly more seriously, people who deteriorated, depending on how the data was analysed. For example, if two participant deteriorated and ended up with a Chalder fatigue scale scores of 32, but 8 others had an average score of 2, this would give an average score of 8, again giving the impression that this group had less fatigue than healthy people. This issue of supranormal scoring with likert scoring (0-3) can be dealt with by utilising the commonly used bimodal scoring method for the Chalder fatigue scale. However, both it and likert scoring suffer from ceiling effects in CFS populations so other scales are likely preferable (14,15). If the Chalder fatigue scale is used, the original developers of the scale suggested, based on their analyses, that "it would probably be more useful to have two scores, one for physical fatigue and one for mental fatigue"(16).
References:
1. Crawley E, Mills N, Beasant L, Johnson D, Collin SM, Deans Z, White K, Montgomery A. The feasibility and acceptability of conducting a trial of specialist medical care and the Lightning Process in children with chronic fatigue syndrome: feasibility randomized controlled trial (SMILE study). Trials. 2013 Dec 5;14:415.
2. Specialist Medical Intervention & Lightning Evaluation: Comparing specialist medical care with specialist medical care plus the Lightning Process for Chronic Fatigue Syndrome or Myalgic Encephalopathy (CFS/ME) ISRCTN81456207 http://www.controlled-trials.com/ISRCTN81456207'>Link</a (last accessed: December 20, 2013)
3. The Lightning Process Didn't Work For me.
http://sallycats.hubpages.com/hub/The-Lightning-Process-Didnt-Work-For-me'>Link</a (last accessed: December 20, 2013)
4. LP & ME What is the truth Identities withheld. http://groups.yahoo.com/neo/groups/MERSC-Viewpoint/conversations/topics/840?var=1'>Link</a (last accessed: December 20, 2013)
5. The Skeptic's Dictionary, From Abracadabra to Zombies, Phil Parker Lightning Process
http://www.skepdic.com/lightningprocess.html'>Link</a (last accessed: December 20, 2013)
6. Kewley AJ. Does Cognitive Behavioral Therapy or Graded Exercise Therapy Reduce Disability in Chronic Fatigue Syndrome Patients? Objective Measures Are Necessary. Clinical Psychology: Science and Practice 2013 20:321-322.
7. Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg, G. (2010). How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychological Medicine, 40(8), 1281¿1287.
8. White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O'Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M; PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet. 2011 Mar 5;377(9768):823-36
9. Biophile. CBT and GET did not significantly reduce employment losses, overall service costs, welfare benefits or other financial payments.
http://www.plosone.org/annotation/listThread.action?root=52797'>Link</a
(last accessed: December 20, 2013)
10. Newton JL, Pairman J, Hallsworth K, Moore S, Plötz T, Trenell MI. Physical activity intensity but not sedentary activity is reduced in chronic fatigue syndrome and is associated with autonomic regulation. QJM. 2011 Aug;104(8):681-7
11. Cockshell SJ, Mathias JL. Cognitive functioning in chronic fatigue syndrome: a meta-analysis. Psychol Med. 2010 Aug;40(8):1253-67.
12. Loge JH, Ekeberg O, Kaasa S. Fatigue in the general Norwegian population: normative data and associations. J Psychosom Res 1998; 45:53-65.
13. van Kessel K, Moss-Morris R, Willoughby E, Chalder T, Johnson MH, Robinson E. A randomized controlled trial of cognitive behavior therapy for multiple sclerosis fatigue. Psychosom Med. 2008 70:205-13
14. Stouten B. Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution. BMC Health Serv Res. 2005 May 13;5:37.
15. Goudsmit, EM., Stouten, B and Howes, S. (2008). Fatigue in myalgic encephalomyelitis. Bulletin of the IACFS/ME, 16(3), 3-10.
16. Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessely S, Wright D, Wallace EP. Development of a fatigue scale. J Psychosom Res. 1993;37(2):147-53.
Competing interests
I am the Assistant Chairperson and Information Officer for the Irish ME/CFS Association. All my work for the Association is voluntary.
Trials
The validity of using subjective outcome measures as primary outcomes is questionable in such a trial
29 July 2014
Crawley and colleagues suggest dropping school attendance as a primary outcome from the full study, and replacing it with self-report outcomes "such as the SF-36 or the Chalder Fatigue Scale" and using "school attendance as a secondary outcome"(1). And indeed, this is what they have done with the full study which has two primary outcome measures: "Chalder Fatigue Scale at 6 months" and "SF 36 physical function short form at 6 months"(2). I question the wisdom of using self-report measures as primary outcomes for such a trial. The authors do not give much information about the details of the Lightning Process (they do mention it is "developed from osteopathy, life coaching and Neuro-linguistic programming (NLP))" but here are some descriptions from other sources (individuals who have undertaken a course) (note these are not descriptions from participants from the trial itself): (a) "It felt very naughty but I whispered to one of the woman (sic) sitting next to me 'how are you, is this working for you?'. She was reluctant to answer, to say anything but that she was doing well would be to go against the process because that is a negative thought. It was pointless asking really. Still I wanted it to work, but I was starting to worry about the fact that I was not only not feeling any better the effort of doing the course, not getting my normal rest was making me feel worse. But these were negative thoughts. I started to ruthlessly suppress them like I had been shown." (3) (b) ¿They tell you that you're not allowed to say that you're ill anymore or that you have any symptoms. They force you to ignore the symptoms because they say that the symptoms don't really exist. They force you to do activities even though it's making you really ill, but you're not allowed to say so.¿ (4) Another, more thorough, description can be found on the Skeptic's Dictionary website(5). I have seen many similar descriptions to these in discussions in many private fora. Here's one example: "So the first step in the process is to recognise when you¿re in THE PIT. Maybe sometimes or all the time. It¿s important to recognise what you say to yourself as you go into the pit. For example ¿I¿m feeling really ill this morning¿, ¿if I do this, then I¿ll get exhausted¿, ¿last time I did this I got really ill for days¿, ¿I can never eat this¿ etc. This takes some practise but we were assured that you always say something in your head as you go into the pit. As soon as you spot one of your ¿pit¿ phrases you want to STOP yourself right away. So imagine you¿re on the mat and you start to say ¿I feel really ill today¿. Before you get to the end of this phrase you will interrupt with a very firm, loud ¿STOP¿ (yes, talk out loud to yourself!) and jump into the STOP position as described above. So you jump outside the mat, to your left. Now you¿re here you have interrupted your bad thought patterns." In essence when one is doing the lightning process, both during the three days training and after, you are to declare yourself well. You are not to say (or think) you have symptoms and you are not to say (or think) you have limitations. In other words, patients are 'trained' to dismiss and deny their symptoms and illness. Patients are instructed not to ¿do¿ ME or CFS anymore. In such a scenario, subjective reports from the patient are no longer reliable. Hence the need for objective measures in such trials. To a lesser extent, it can also be argued that subjective measures are not ideal for all participants in this trial, including the control group. With graded activity-oriented therapies, which all the participants undertake (1), there may be response bias (6). This was seen in a review of three Dutch trials of graded activity-oriented cognitive behaviour therapy (CBT) for CFS. While the CBT participants reported improvements in fatigue (and also SF-36 physical functioning although that was not measured in all of the trials), no improvements in objectively measured activity were reported over the control group(7). Similarly a mediation analysis showed changes in physical activity were not related to changes in fatigue. A similar effect with CBT could be in PACE Trial, the largest such trial in the CFS field(8). Although participants who had undergone CBT reported improvements in scores on both the Chalder Fatigue Scale and SF 36 physical function subscale, no improvements were reported in the six minute walking distance compared to the group who had undergone no additional individual therapy (all participants had specialist medical care). Similarly, CBT did not significantly reduce employment losses, overall service costs, welfare benefits or other financial payments(9). Objective measures of physical activity are one type of objective measurement that could be employed. This could be used not just to measure the total quantity of activity but also to check the intensity of activity as people with ME/CFS may perform lower levels of more intensive activity(10). Tests of neuropsychological performance could also be used, given the impairments that have previously been reported(11). If necessary, a webpage could be created that could be accessed remotely. One particular problem with the 0-33 scoring method for the Chalder fatigue scale is that patients can give unusually low, or artificially good, scores. The scale consists of 11 questions. For each question (e.g. "Do you have less strength in your muscles?"), patients have to say whether they have the symptom: "Less than usual" (score of 0); "No more than usual" (score of 1); "More than usual" (score of 2) or ¿Much more than usual¿ (score of 3). So healthy people should score around 11 and indeed in the only population study I can recall that gave data for a healthy (no disease/current health problem) group, the mean was 11.2 (12). However, some unusual scoring is possible. This was seen in a multiple sclerosis trial, of a similar CBT intervention to the graded activity-oriented CBT used in CFS (13). Participants in the CBT leg of the trial entered with a Chalder fatigue scale score of mean (sd): 20.94 (4.25). Two months after therapy, the mean (sd) was 7.90 (4.34) i.e. this groups suffering from multiple sclerosis had a much better score than one sees in healthy people! Such averages could also hide non-responders or, possibly more seriously, people who deteriorated, depending on how the data was analysed. For example, if two participant deteriorated and ended up with a Chalder fatigue scale scores of 32, but 8 others had an average score of 2, this would give an average score of 8, again giving the impression that this group had less fatigue than healthy people. This issue of supranormal scoring with likert scoring (0-3) can be dealt with by utilising the commonly used bimodal scoring method for the Chalder fatigue scale. However, both it and likert scoring suffer from ceiling effects in CFS populations so other scales are likely preferable (14,15). If the Chalder fatigue scale is used, the original developers of the scale suggested, based on their analyses, that "it would probably be more useful to have two scores, one for physical fatigue and one for mental fatigue"(16). References: 1. Crawley E, Mills N, Beasant L, Johnson D, Collin SM, Deans Z, White K, Montgomery A. The feasibility and acceptability of conducting a trial of specialist medical care and the Lightning Process in children with chronic fatigue syndrome: feasibility randomized controlled trial (SMILE study). Trials. 2013 Dec 5;14:415. 2. Specialist Medical Intervention & Lightning Evaluation: Comparing specialist medical care with specialist medical care plus the Lightning Process for Chronic Fatigue Syndrome or Myalgic Encephalopathy (CFS/ME) ISRCTN81456207 http://www.controlled-trials.com/ISRCTN81456207'>Link</a (last accessed: December 20, 2013) 3. The Lightning Process Didn't Work For me. http://sallycats.hubpages.com/hub/The-Lightning-Process-Didnt-Work-For-me'>Link</a (last accessed: December 20, 2013) 4. LP & ME What is the truth Identities withheld. http://groups.yahoo.com/neo/groups/MERSC-Viewpoint/conversations/topics/840?var=1'>Link</a (last accessed: December 20, 2013) 5. The Skeptic's Dictionary, From Abracadabra to Zombies, Phil Parker Lightning Process http://www.skepdic.com/lightningprocess.html'>Link</a (last accessed: December 20, 2013) 6. Kewley AJ. Does Cognitive Behavioral Therapy or Graded Exercise Therapy Reduce Disability in Chronic Fatigue Syndrome Patients? Objective Measures Are Necessary. Clinical Psychology: Science and Practice 2013 20:321-322. 7. Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg, G. (2010). How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychological Medicine, 40(8), 1281¿1287. 8. White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O'Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M; PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet. 2011 Mar 5;377(9768):823-36 9. Biophile. CBT and GET did not significantly reduce employment losses, overall service costs, welfare benefits or other financial payments. http://www.plosone.org/annotation/listThread.action?root=52797'>Link</a (last accessed: December 20, 2013) 10. Newton JL, Pairman J, Hallsworth K, Moore S, Plötz T, Trenell MI. Physical activity intensity but not sedentary activity is reduced in chronic fatigue syndrome and is associated with autonomic regulation. QJM. 2011 Aug;104(8):681-7 11. Cockshell SJ, Mathias JL. Cognitive functioning in chronic fatigue syndrome: a meta-analysis. Psychol Med. 2010 Aug;40(8):1253-67. 12. Loge JH, Ekeberg O, Kaasa S. Fatigue in the general Norwegian population: normative data and associations. J Psychosom Res 1998; 45:53-65. 13. van Kessel K, Moss-Morris R, Willoughby E, Chalder T, Johnson MH, Robinson E. A randomized controlled trial of cognitive behavior therapy for multiple sclerosis fatigue. Psychosom Med. 2008 70:205-13 14. Stouten B. Identification of ambiguities in the 1994 chronic fatigue syndrome research case definition and recommendations for resolution. BMC Health Serv Res. 2005 May 13;5:37. 15. Goudsmit, EM., Stouten, B and Howes, S. (2008). Fatigue in myalgic encephalomyelitis. Bulletin of the IACFS/ME, 16(3), 3-10. 16. Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessely S, Wright D, Wallace EP. Development of a fatigue scale. J Psychosom Res. 1993;37(2):147-53.Competing interests
I am the Assistant Chairperson and Information Officer for the Irish ME/CFS Association. All my work for the Association is voluntary.