IAPT’s Unreliable Assessment of Competence, Incompetence and Spin

the competence of trainee CBT therapists is routinely assessed using the Cognitive Therapy Rating Scale-Revised (CTRS-R), but a just published study by Roth et al (2019) has shown poor  levels of agreement on the performance of IAPT trainees, using this measure.  The levels of agreement were no better when an alternative measure of competence the University College London CBT Scale was used. On both measures of competence the intra-class correlation coefficients were less than 0.5, indicating poor reliability (on a scale poor, moderate, good, excellent). The UCL Scale is rooted in the competence framework developed by Roth and Pilling (2008) as part of the IAPT programme.

The chaos is underlined by a study conducted by Liness et al (2019), published in Cognitive Therapy and Research which assessed the competence  of IAPT trainees using the CTRS-R with client outcome assessed, mainly with the PHQ9 and GAD7, and no relationship was found, either at the end of training or 12 months later, see link below:

https://www.dropbox.com/s/e26n191ie09sngs/Competence%20and%20Outcome%20IAPT%20no%20relation%202019.pdf?dl=0

But the same set of authors as in the Liness et al (2019) study, have published a further paper in Behavioural and Cognitive Psychotherapy, again of IAPT trainees, evaluated using the CTRS-R. But this time, in the abstract, they reported that ‘CBT competence predicted a small variance in clinical outcome for depression cases’ with no reference to the findings of their other paper!  In the body of their Behavioural and Cognitive Psychotherapy report one discovers that for depression cases the CTRS-R explained 1.3% of the variance in outcome, it is extremely doubtful if this is of any social or clinical significance. There is also a failure to mention in the abstract that CTRS-R did not at all relate to anxiety.  The abstract is dominated by the message that training helped trainees score highly on the CTRS-R, without acknowledging that this might be without meaning. Three of the 6 authors have links to IAPT  and spin is not therefore unexpected. 

Liness et al  Behavioural and Cognitive Psychotherapy (2019), 47, 672–685
doi:10.1017/S1352465819000201

Roth et al  Behavioural and Cognitive Psychotherapy (2019), 47, 736–744 doi:10.1017/S1352465819000316

Roth, A. D. and Pilling, S. (2008). Using an evidence-based methodology to identify the competences required to deliver effective cognitive and behavioural therapy for depression and anxiety disorders. Behavioural and Cognitive Psychotherapy, 36, 129–147. doi: 10.1017/S1352465808004141

 

Dr Mike Scott

 

 

IAPT No Better Than Treatment As Usual

that is the conclusion of a just published study in the British Journal of Psychiatry by Serfaty et al, (2019).  In the study manualised IAPT-delivered CBT was compared to treatment as usual in treating depressive symptoms in people with advanced cancer. These authors concluded: 

‘our results suggest that resources for a relatively costly therapy such as IAPT-delivered CBT should not be considered as a first-line treatment for depression in advanced cancer. Indeed, these  findings raise important questions about the need to further evaluate the use of IAPT for people with comorbid severe illness’

Interestingly the IAPT therapists were all High Intensity Therapists  with mean Cognitive Therapy Rating Scale Score of 47.6  “at the upper end of the ‘proficient range’.” The primary outcome measure was the Beck Depression Inventory II and both the treatment as usual and the CBT group showed a mean reduction of 5 points. Curiously the MINI diagnostic interview was used to assess whether people were in fact depressed and would thereby qualify for the study but it was not re-administered at the end to determine how many were no longer depressed. Nevertheless even using a diagnostic interview trying to determine whether symptoms such as sleep disturbance and fatigue should count as part of a depression or as a feature of the illness   is realistically probably an impossible task.  The appropriate model with severe illnesses is probably support ( both tangible e.g heating allowance and emotional) rather than psychological therapy.

Serfaty et al Effectiveness of cognitive-behavioural therapy for depression in advanced cancer: CanTalk randomised controlled trial British Journal of Psychiatry (2019) Page 1 of 9. doi: 10.1192/bjp.2019.207

Dr Mike Scott

The Treatment of PTSD Has Been Destabilised……

by the advent of stabilisation groups and overvaluing trauma focussed CBT. In the wake of an extreme trauma IAPT clients can be referred to stabilisation groups. Such groups will often meet weekly for 6 weeks and participants are encouraged not to talk about the trauma but rather about its effects. However there is no empirical evidence that such groups make a real world difference. In support of such groups the work of Judith Herman  [ Group Trauma Treatment in Early Recovery (2019) Guilford Press] is often cited, her groups are for those in ‘early recovery’ but there is no specification of what is meant by ‘early’ or from what the person is recovering. IAPT’s assessment process is as vague as Judith Herman’s.

 

Sienna, a Civil Servant had a horrendous rta and after an IAPT telephone assessment was referred to a stabilisation group, she assumed it was for PTSD. The group made no difference to her functioning, nor did the 3 individual sessions of trauma focusssed cbt afterwards. Sienna dropped out of the TFCBT because it was too painful but she never did have PTSD!

 

But the problems in the treatment of PTSD are not confined to IAPT. Although trauma focussed CBT (TFCBT) is the NICE recommended treatment for PTSD, inspection of the randomised controlled trials reveals that on average only one in two people recover. NICE’s guidance can be overvalued, with clinicians continuing to pursue TFCBT when it is clearly not working. With a parallel insistence that they confront the scene of their trauma. Client’s are often more pragmatic thinking that they could get by without re-exposure to the scene, but with the therapist urging the client not to be ‘defeated’. Given the power imbalance the client is unlikely to be able to effectively voice their opinion. There is a pressing need for creative solutions when TFCBT doesn’t work and for a re-examination of the theory on which the latter rests.

I am proposing to run a ‘Getting Back To Me’ workshop next year.

 

Dr Mike Scott