external validity Archives

What NICE says about the efficacy of CBT has been taken as gospel, but Moriana et al (2017) have pointed out that what other similar bodies say is significantly different. The actions of practitioners are micro-managed by august bodies such as NICE (via IAPT), Division 12 (Clinical Psychology American Psychological Association, Cochrane and the Australian Psychological Society, an essentially top down process is in operation. But which, if any should be the determinant?

Rather than arguing about which body has produced the best synthesis of outcome studies the focus should shift to bottom up, asking how does cbt fare in routine practice?

Tolin et al (2015) have suggested that a treatment should only be regarded as effective if there has been a randomised controlled trial of the intervention in routine practice using non-specialist therapists, further the researchers should be independent of those who originally developed the treatment. This has been adopted by the American Psychological Association. An additional requirement should be that the ‘gold standard’ entry requirement for the trial, admission by a standardised diagnostic interview, should also be the primary outcome measure as assessed by independent blind assessors. Only in this way can it be known whether the treatment makes a real world difference i.e it will be known that x% no longer suffer from the disorder at the end of treatment compared to y% in the control condition. Without these diagnostic strictures one ends up with the highly questionable conclusion of Pybis et al (2017) that cbt and counselling are equally effective. Tolin et al (2015) have suggested the external validity criteria have been fulfilled in the case of CBT for OCD, but when we look at other disorders such as trauma focussed cbt for PTSD it is doubtful that it clears such a high methodological bar, for example the supposed replication of Ehers et al CBT for PTSD (2005) by Gillespie et al in Northern Ireland did not involve a standardised diagnostic interview as the primary outcome measure, further there were no independent assessors.

It may be that the struggles of practitioners to achieve performance targets are not so much to do with their deficiencies as inherent in the context within which they are working. Singling out ‘poor performers’ may be unjust in extremis. Pybis et al (2017) concluded that ‘half of all patients (IAPT clients) regardless of type of intervention (counselling or CBT) , did not show reliable improvement’, leaving aside whether the IAPT self-report mesasures they review are at all meaningful, are half the therapists going to be put in the dock?

Ehlers, A et al (2005) Cognitive therapy for PTSD development and evaluation. Behaviour Research and Therapy, 43, 413-431.

Gillespie, K et al (2002) Community based cognitive therapy in the treatment of PTSD following the Omagh bomb. Behaviour Research and Therapy, 40, 345-357.

Moriana, J.A et al (2017) Psychological treatments for mental disorders in adults: A review of the evidence of leading international organizations. Clinical Psychology Review, 54, 29-34

Pybis, J et al (2017) The comparative effectiveness and efficiency of cognitive behaviour therapy and generic counselling in the treatment of depression: evdence from the 2nd UK National Audit of psychological therapies. BMC Psychiatry, 17:215

Tolin, D.F et al (2015) Empirically supported treatment: recommendations for a new model. Clinical Psychology Science and Practice, 22, 317-338.

Dr Mike Scott

Stressed Because You Are Asked To Do What Is Not Possible?

‘I am not IAPT’s employee of the month’, ‘For all to view, I am nowhere near the top of the league table for recovery rates’ – just two of the voices of stressed therapists that I have heard in recent months. Unfortunately challenging the Organisational zeitgeist that generates such demoralisation is likely to be seen as a further sign of ‘inadequacy’.

But the American Psychological Association, Clinical Psychology Division, have refined the criteria for an evidence based treatment in such a way, that it could add ‘grist to the mill’ of those who wish to take issue with what they are being asked to do. The new criteria contain the added requirement that there must be evidence of effectiveness in at least one study, in a nonresearch setting using non-academic therapists and also evidence of effectiveness on functional impairment and not just symptom improvement. [The new requirement is based on the Tolin et al (2015) paper Empirically supported treatment: recommendations for a new model. Clinical Psychology Science and Practice, 22, 317-338]. With this added criteria many of the CBT treatments for health anxiety, psychosis and long term physical health problems and computerised cbt would not clear the raised bar for evidence based treatment. Even an accepted treatment such as exposure and response prevention for OCD has just one routine practice study and this was without a control condition, Tolin still strongly recommended the treatment but suggested that there did need to be further evidence.

Pencil in ‘and CBT’ next to ‘antibiotics’ in the poster below and display in Staff and ? Waiting Rooms for the coming Winter

Often times a client has considerable comorbidity, I have just seen a person with ptsd, depression, binge eating disorder, panic disorder with mild agoraphobic avoidance and ?body dysmorphic disorder following an rta 4 years ago. No randomised controlled trial has ever been conducted on a population with such extensive comorbidity. There has to be proper acknowledgement of the challenges such a client presents for a therapist, it is insufficient to label a client simply as ‘complex’ because this can easily be seen by the powers that be as a ‘cop out’, the only true defence is a comprehensive reliable assessment.

The bottom line may be to challenge, where you can, whether there is evidence of effectiveness for this specific type of client in the routine context in which you are working – a reality check.

Dr Mike Scott