Prestigous Journals Have Stopped Looking at Real World Mental Health Outcomes

Papers in Journals such as The Lancet, Behaviour Research and Therapy and Behavioural and Cognitive Psychotherapy have in recent years relied entirely on psychometric tests completed by clients, with no independent assessment by an outside body using a ‘gold standard’ diagnostic interview. The sole use of psychometric tests is great for academic clinicians, research papers can be produced at  pace and at little cost, securing places in academia. Conferences are dominated by their offerings but actually nothing is changing in the real world of clients.



The Lancet paper on the PACE trial on CBT  for chronic fatigue syndrome [Sharpe et al (2015) Rehabilitative treatments for chronic fatigue syndrome Lancet Psychiatry, 2, 1067-1074] provides a great example of how to ‘muddy the waters’. The authors presented CBT as making a major contribution to the treatment of CFS. But Bakanuria (2017) [ Chronic fatigue syndrome prevalence is grossly overestimated using Oxford criteria compared to Centers for Disease Control (Fukuda) criteria in a U.S population study. Fatigue: Biomedicine, Health and Behavior, ps 1-15] has pointed out that the authors used the very loose Oxford criteria for CFS, requiring mild fatigue, but the incidence of CFS is ten times less if the Center for Disease Control (CDC) rigorous criteria are used. Thus Sharpe et al had not demonstrated the efficacy of CBT in a population who truly had CFS. In December last the Lancet published a paper by Clark et al on predictors of outcome in IAPT but again the dependent variable is of  doubtful validity, changes on PHQ9 and GAD7 in a population whose  diagnostic status is unknown. In fairness in the discussion Clark et al (2017) do note that it is a limitation of their study that they have relied on self-report measures but there is no acknowledgement that their findings are actually unreliable. Doubtless their conclusion that organisational factors effect delivery of an efficacious treatment is true, but this is stating the obvious, if a treatment is found to be efficacious in a randomised controlled trial, unless there is a careful mapping of key elements in the rct e.g reliable diagnosis, ‘gold standard’ assessment, fidelity measures, there will be an inadequate translation from research into routine practice.

My hope for the New Year is objective measures of outcome so that we can truly begin serving clients, now there is a novel idea.

Dr Mike Scott

CBT Researchers Have Abandoned Independent Blind Assesment – Beware of Findings

I have been looking in vain for the last time CBT researchers assessed outcome on the basis of independent blind assessment, which was a cornerstone of the initial randomised controlled trials of CBT.  Current CBT research is more about academic clinicians marketing their wares. Journals such as Behaviour Research and Therapy and Behavioural and Cognitive Psychotherapy and organisations such as BABCP and BPS are happily complicit in this. The message is give a subject a self-report measure to complete, it is less costly than expensive highly trained independent interviewers blinded to treatment, forget about the demand characteristics of a self-report measure ( a wish to please those who have provided a service) and don’t worry if the measure does not accurately reflect the construct under question. My psychiatric colleagues might be forgiven for saying that at least the trials of antidepressants have usually been double blinded, if since the millennium CBT studies have rarely managed to be single blinded, is it time the CBT-centric era ended? But purveyors of other psychotherapies have even more rarely bought into the importance of independent blind assessment.

The overall impact of inattention to independent blind assessment is that the case for pushing CBT is actually not as powerful as the prime movers in the field would have us believe, this may actually be a relief to struggling practitioners. For example Zhu et al (2014) [Shangai Arch Psychiatry, 26, 319-331 examined 12 randomised controlled trials of CBT for generalised anxiety disorder in which there was supposedly independent blind assessment  but in 6 of the 12 studies the main outcome measure was based on the results of a self-reported scale completed by the client (i.e outcome was not actually assessed by the blinded assessor) and concluded that the quality of the evidence supporting the conclusion that CBT was effective for GAD was poor. A meta-analysis of outcome studies  conducted by Cuijpers (2016) World Psychiatry, 15, 245-258 found that using criteria of the Cochrane risk of bias tool only 17% (24 of 144) rct’s of CBT for anxiety and depressive disorders were of high quality. Cuijper et al concluded that CBT ‘is probably effective in the treatment of MDD, GAD, PAD and SAD; that the effects are large when the control condition is waiting list, but small to moderate when it is care-as-usual or pill placebo; and that, because of the small number of high-quality trials, these effects are still
uncertain and should be considered with caution’. Only half the studies had blind assessors and it is not clear whether they were the determinants of outcome or a client completed self-report measure, the study needs further analysis. My impression is that the weakest of studies are those examining guided self-help, computer assisted CBT, (the step 2 interventions in IAPT) yet these interventions are most commonly offered.

Dr Mike Scott