when IAPT’s data is re-examined, independently, startlingly different conclusions emerge. Barry McInnes writes:
Well done Barry
Dr Mike Scott
if there was no IAPT the outcome for treatment of primary care clients would be just the same. In 2006 (before IAPT) Mullin et al examined the effects of counselling/therapy in more than 11,000 clients and concluded that between 5 and 6 clients out of every 10 met the criterion for recovery. These authors used the same criterion with regard to the reliable change index as used by IAPT, but used the CORE-OM self-report measure rather than the PHQ9/GAD7. If anything the Mullin et al (2006) results are slightly better than IAPT’s claimed 50% recovery.
Economists evaluate the worth of a service by comparing it with its non-existence (the appropriate counterfactual), the Mullin et al (2006) study suggests that at the very least there is no added benefit to IAPT.
Thanks to Barry McInnes for alerting me to the Mullin et al (2006) study
Clinical Commissioning groups need to be made aware of this. If psychological therapists were employed by the GP practice there would be obvious accountability to the GPs. At present accountability is solely to a QUANGO with its’ own agenda. It is a scandal that the National Audit Office has not published the results of its enquiry into IAPT. Perhaps a collusion of Quangos.
Dr Mike Scott
at a workshop I gave in Liverpool last Friday, there was much interest in this metric for evaluating the effectiveness of a psychological service. The workshop was titled ‘Group CBT…Yes…But’ (and was also the innaugral meeting of the BABCP Group CBT Special Interest Group), and I reflected that none of the studies of classes, such as ‘Stress Control’ or the ‘Five Areas Approach’ had independent assessors asking people whether as a result of the intervention they were back to their old self, much less whether they remained as their old self for say at least 8 weeks. Yet they are promoted as the first line of treatment in services such as IAPT. Further were groups (as opposed to classes) are run they are often for targets such as ‘low self-esteem’ or ‘destabilisation groups’ with for which there is no evidence at all of real world outcomes. I think a key feature of the workshop for many people was making a sharp distinction between the evidence base for classes as opposed to groups, for depression and the anxiety disorders. The powerpoint presentation for the workshop can be accessed below:
I also suggested that the case for transdiagnostic approaches is, at the very least, not proven. Nevertheless I fear managers will attempt to play a numbers game with regards to groups blurring the distinction between them and classes. With, as suggested in a role play we did, a therapist trying to sell a ‘stress class’ to a client over the telephone, the latter could have had depression, PTSD, body dysmorphic disorder (or some combination there of) or even an adjustment disorder. The therapist herself with insufficient time to make a formulation becoming a candidate for a stress class in her own right!
Dr Mike Scott
this is the title of a just published book by Farhad Dalal, [ London, Routledge] it is a scathing critique of IAPT and its’ ‘managerialism’. Staff too afraid to speak out publicly, a concern primarily for operational matters: numbers and waiting lists. The author reviews randomised controlled trials to try and ascertain the proportion of people who actually get better. Dalal rightly sees IAPT’s claim to a 50% recovery rate as preposterous and its’ criterion for treatment attending at least 2 sessions as unbelievable.
Curiously, Dalal appears not to have heard of my study that showed a recovery rate of just 10%.:
It was possible for me to invalidate IAPT’s claims precisely because I used the DSM criteria that Dalal decries. Unfortunately Dalal, as a group analyst would create a psychological therapy service without any evidence base.
In 2014 Ehlers, Clark et al published a ‘gold standard’ randomised controlled trial of the treatment of PTSD
what makes it ‘gold standard’ was that a) assessment was conducted using a standardised semi-structured interview of high reliability b) outcome was assessed independently of treating clinician using the standardised initial interview, making it possible to specify what proportion of people were no longer suffering from the disorder c) there was a follow up to determine if treatment gains were maintained d) there was a credible attention control condition, so that it was possible to determine whether there was something specific in the treatment that made a difference.
Set against this ‘gold standard’ there is to my knowledge no study of any low intensity therapy that has met the above criterion. Quite simply low intensity interventions do not have a reliable evidence base. Further routine practice can also be assessed using criteria a) b) and c) but this has never been done in relation to IAPT except by myself in a limited context.
This ‘Tsunami’ may destroy everything in its’ path, but we have to know what would constitute the building of an evidence based mental health system see Towards a Mental Health System That Works (2017) Scott London: Routledg: https://www.amazon.co.uk/Towards-Mental-Health-System-Works/dp/1138932965/ref=sr_1_2?ie=UTF8&qid=1550698217&sr=8-2&keywords=Towards+a+Mental
BBC News Video critique of IAPT here: https://vimeo.com/316124732
Dr Mike Scott
this is the title of a Workshop I’m delivering on Feb 22nd in Liverpool, it is also the inaugral meeting the BABCP Group CBT Special Interest Group. Whilst the workshop is full you can join in the discussion on Group CBT simply by posting a reply to this post. There are also other posts on group CBT on this blog just type in ‘group CBT’ in the search box.
Manuals for depression and the anxiety disorders, assessment protocols etc from Simply Effective Group Cognitive Behaviour Therapy (2009) London: Routledge are freely available by clicking the link below:
The questions to be addressed at the workshop include:
‘Groups are a rarity compared to individual therapy, despite the fact that barely more than the tip of the iceberg of clients are likely to be offered therapy in the forseeable future, Why is this? Is changing attitudes to the running of groups likely to be sufficient to ensure wider dissemination of group therapy?
Do you believe you have got the skills necessary for running a group? What are they and how do you know if you have got them? How can you get the skills?
Do you believe running a group would make a worthwhile difference? What outcomes constitute a real world difference? How would I know if marketing is outstripping evidence?
What group treatment works for whom? What about transdiagnostic groups? How transdiagnostic can you go? What is the minimum dose of group CBT? What happens if you don’t ensure full recovery?
What are the organisational obstacles and plusses?’
Do join the SIG by contacting Nicola, email@example.com
Dr Mike Scott
the comments of an IAPT worker when he was told by management to make sure that the client completed the PHQ9/GAD7 face to face at the 6th session. Normally the client completes the measures before each of the first 5 sessions at home. Face to face completion introduces a demand effect, politeness, wanting to please the therapist, lowering scores so that IAPT’s results look better. Matters are compounded for him by only being allowed to provide usually 6 sessions in high intensity. Where is the evidence base for 6 sessions doing anything?’
This behaviour echoes IAPT’s, misrepresentation of its’ own outcome data. A more realistic and independent picture has been painted by Barry McInnes
Clients are waiting less time than ever to access IAPT services, and recovery rates have reached their highest ever level at 50.8%. So why am I not throwing my hat in the air? In a nutshell, the astonishingly high levels of attrition. With less than one in five that are referred and one in four that enter therapy achieving recovery, what is the experience of those that IAPT is serving less well?
Below is a graphic which shows clients at key stages of their journey through the Improving Access to Psychological Therapies (IAPT) programme in the year 2017 – 18. It’s an update of the figures that I provided for 2016 – 17 in a previous blog. As was the case then some of the performance data for 2017 – 18 show an improvement over the previous year. I also said previously that some aspects of performance start from an already worryingly low baseline. Having looked at the latest data I see grounds only for modest optimism.
Once again, the main story behind the numbers is one of extraordinary levels of attrition at each stage of the journey. The detail follows in subsequent sections.
First, the headlines. In the year 2017 – 18, the report and supporting datafile show that:A
an increase of 54,293 on the previous yearA
This is up one and a half percentage points on the previous year, and for the first time exceeding the target of 50%. (definitions for recovery and caseness are shown in the panel below).A
exceeding their respective targets of 75% and 95%
Headlines aside, there’s a story behind the numbers which I’ve broken down into four chapters. They are:
30% of referrals don’t enter therapy.
45% of clients that enter therapy don’t complete.
53% of clients don’t achieve recovery
51% of clients achieve recovery. 49% do not.
In each chapter I’ll show data for 2015 – 16, 2016 – 17 and 2017- 18.
The journey starts with all referrals for the three years. The table below shows those referrals as 100%. Subsequent stages in the process, and the proportion of clients referred that still remain are also shown. In 2017 – 18, 70% of those referred entered therapy. This is defined as having one or more sessions of therapy. The proportions in both of the previous two years were broadly the same.
As we progress through their journey we can see an emerging story of attrition. By the end of therapy, only 39% of clients that were referred now remain (down from 41% in 2016-17) Those that reach recovery represent just 18% of the total referred . That’s less than one in five clients.
Why did only seven in ten referrals enter therapy? I can only speculate, but in doing this it’s important to note that nearly nine in ten referrals were seen within six weeks. Hence length of wait may not be the most important factor. Otherwise, they may no longer have needed therapy, their referral may not have been appropriate, or they may simply not have liked what was offered to them.
We begin this chapter with all clients that entered therapy. That is, all those that had one or more therapy sessions. In 2017 – 18, 55% of those entering therapy are recorded as ending it. This figure is down 4% from 2016 – 17. Whichever year we’re talking about, however, the fact that more than four in ten clients do not complete should be a concern.
Following the journey through, we can see that just 26% of those entering therapy achieved recovery. In other words, clients entering therapy appear to stand just over a one in four chance of recovering.
Why are only 55% of clients recorded as ending therapy? Again, it’s hard to know. In my experience, however, clients more commonly drop out because they feel that therapy isn’t working for them for some reason. Often, they simply disappear without the opportunity to explore this with their therapist, and make adjustments that may better serve their needs.
Here we start with all the clients that are recorded as finishing a course of treatment. The first point to note is that not all clients were at a case level of symptoms or distress at the outset. No matter how much improvement they make, therefore, they cannot achieve recovery. In the 2017 – 18 population, 93% of clients were at case level, and could potentially recover. This is the same as in the previous year.
Those reaching recovery represent 47% of all those that finished treatment. For 2017 – 18, this is an increase of 1% over the previous year.
The final leg of the journey, starting with clients who finished treatment that were at case level at the point they started. In this population we have all the clients who have the potential to achieve recovery.
In 2017 – 18, 51% of clients achieved recovery, of those that finished therapy and were at case level at the start. To be precise, the figure given in the IAPT report is 50.8% (against 49.3% for 2016-17).
The figure for those achieving recovery for 2016 – 17 is 1.5% higher than in the previous year. This is the first time that the recovery rate has exceeded the target of 50% that was established for the IAPT programme.
While this improvement is clearly welcome, however, it remains the case that almost as many clients do not recover as do. At nearly 51:49, the odds of recovering or not recovering are almost even.
Using the tools that accompany the IAPT annual report, it’s possible to explore local and regional performance. Sadly, the datasheet which houses all of the raw data is now structured in such a way that it’s no longer possible to get a single improvement rate for each Clinical Commissioning Group (CCG) area and easily filter those to see the range of performance.
What I am loving, however, in this new reporting structure, is the IAPT Interactive Dashboard, which you’ll find at the foot of the Resources section of the summary page. It’s the tool from which the image of the interactive map (main blog image at the top) is taken, and also the screenshot below.
The interactive tool allows you to filter performance data at a local level, either by hovering over the map or entering the CCG or commissioning region, for example. In the image above right, I’ve selected two CCG areas towards the lower and higher ends of the improvement range. If you’re interested in finding the performance for your local area I’d encourage you to go and have a play.
In summary, then, there is some good news and some bad news. The good news is that clients are able to access services more rapidly than before and that for those that end therapy, their chances of recovery are more than one in two for the first time.
The bad news is that attrition is alive and well. 30% of referrals do not enter therapy. Only 39% of referrals reach the end of therapy, and only 18% achieve recovery. In that context, I find it hard to find too much to celebrate.
I concluded the previous blog with a simple five-point plan that will assist any therapy service, IAPT included, to improve on its current performance. Rather than repeat myself here’s a link that will take you straight here.
I’d love to hear your thoughts on what you’ve read, whether you have a personal connection with IAPT or otherwise. Leave a comment below and please let us have your thoughts.
We thrive on feedback, so please let us know what you think about what you’ve read in the comment section below. Only the name you use to identify yourself will be shown publicly. Thank you!
If you haven’t already, subcribe to our regular (approximately fortnightly) bulletin for information about our latest blog and other news. You’ll find details here:
Well done Barry
Dr Mike Scott