Allegiance bias Archives

Paranoid, But Judged Recovered If Your Conviction of Threat Falls Below 50%

this is the primary outcome measure used in a just published study of CBT for persecutory delusions https://babcpmail.com/AQ0-7G5L5-2KPHRR-4HXKEC-1/c.aspx. But would the typical person suffering from schizophrenia recognise this metric? What if convictions take a variable course and are mood dependent? What is going on here? Unrestrained by such questions Freeman et al (2021) proclaim in their advertisement for the 5 day online course for the Programme:

‘it is the most effective psychological treatment for persecutory delusions. Half of patients have recovery in their persecutory delusion with the Feeling Safe Programme’

‘Recovery’ here has a meaning far removed from common parlance. ‘When I use a word,’ Humpty Dumpty said, in rather a scornful tone, ‘it means just what I choose it to mean – neither more nor less.’ —LEWIS CARROLL, Through the Looking-Glass, 1871. If my conviction about the likelihood of being flooded fell to less than 50%, I would still be wanting to relocate!

Allegiance Bias

Freeman et al (2021) are evaluating their own Feeling Safe Programme but no mention that therefore their study might be prone to allegiance bias. The same therapists administered the Feeling Safe Programme and the comparison Befriending Programme. Given that the therapists knew that the hypothesis was that the former would prove superior to the latter, they are likely to be more enthusiastic about the CBT. Twenty sessions were to be delivered in 6 months in each modality but in the event more sessions were delivered in the CBT. Thus the possibility of allegiance bias amongst the therapists cannot be ruled out. It is therefore not surprising that a statistically significant difference was found between the two arms of the study. But this does not necessarily demonstrate the added benefit of CBT – a further confounding factor is that 71% of those in befriending were on antidepressants compared to 50% in CBT.

Replication Crisis

Freeman et al (2021) make the common cry of all researchers for more research, but there is no mention of the need for independent replication. This latter is particularly important as previous studies have not demonstrated the added value of CBT for persecutory delusions.

Inappropriate Outcome Measure

Clients in CBT were encouraged to take a 6 session module ( the Feeling Safe Module) targetting threat beliefs, how can the latter then be a credible outcome measure? Broader measures such as functioning as I was before I became paranoid or even as I was when I was least paranoid would have been more credible primary outcome measures. Further the secondary outcome measures used were all based on self-report measures, there was no standardised diagnostic interview conducted. Whilst diagnostic labels were affixed at entry into the study ( on what basis is not clear), they were ignored with regards to outcome.

Is The Effect Size Found Meaningful?

The effect size for the primary outcome measure was a Cohen’s d of 0.86, Freeman et al (2021). The effect size for total delusions score on PSYRATS was d=1.2 Freeman et al (2021) celebrate this large effect size as comparable to that found in trials of CBT for anxiety disorders. But in terms of the primary outcome measure the average person undergoing CBT improved by less than one standard deviation compared to the average person who was befriended, this is shown diagrammatically below, does this amount to a real world difference? The economic analysis promised in the pre-trial protocol was not included in the paper, leaving it an open-question as to whether the CBT is worth the added investment.

Eminence-based Rather Than Evidence-based

Advocates of the Feeling Safe Programme, are claiming more than is known, doubtless BABCP and IAPT will seize on it and control how CBT is to be conducted with this population, extending their empire. Well the study was published in Lancet Psychiatry after all? The CBT therapist should be sceptical, but regrettably training courses seem not to equip them for this, I wonder why? Perhaps I am paranoid?

Dr Mike Scott

Spinning Low Intensity CBT and Treatments for Refugees

How seriously should we take claims for the efficacy of an intervention? The ‘Risk of bias” assessment tool developed by the Cochrane Collaboration (2011), see slide, helps distinguish clever marketing from the genuine article. All LiCBT intervention studies fail the Tools requirement for the blinding of outcome assessment, suggesting a high risk of bias.

In LiCBT studies, there are no independent assessments to determine whether those treated were no longer suffering from the primary disorder for which they first presented. LiCBT interventions is the most common treatment modality in the Improving Access to Psychological Therapies (IAPT) service.

But the LiCBT studies are not only subject to this detection bias, but in many instances they are also subject to allegiance bias, with the main author of the study evaluating their own manualised intervention e.g Williams et al (2018) evaluation of the Living Life To The Full Classes doi: 10.1192/bjp.2017.18

But the purveyors of LiCBT interventions do not have a monopoly on proclaiming effectiveness were non exists. A just published study by Kip et al (2020) DOI: 10.1002/cpp.2446 in Clinical Psychology and Psychotherapy claims that ‘psychological interventions can effectively reduce symptoms of both PTSD and depression in adult refugees’. But of the 14 studies on adult refugees only in 4 studies was outcome assessed by a ‘gold standard’ semi structured interview (the CAPS). The method of determining the diagnostic status of the refugees at entry to the study is unclear, but of the 9 questions asked to determine the quality of the included studies the lowest score was for whether the diagnosis was determined by using a semi structured interview. The next lowest score was for the item referring to fidelity checks i.e did the clients actually receive the treatment that it was purported they receive. Five out of six studies on Narrative Exposure Therapy involved at least one author of the published manual on NET. In total half of the trials researchers were involved in utilising manuals whose development they were involved in. There is clearly a pressing need for independent replication of these findings on refugees. There is also a need to higher the methodological bar, as it is impossible to say from this review what proportion of refugees were no longer suffering from PTSD at the end of treatment, nor how long that improvement persisted.

Like IAPT the understandable wish to further dissemination takes precedence over a determination about whether treatment makes a real world difference to clients lives.

Dr Mike Scott