IAPT’s Training Fails But CBT Can Make a Real World Difference

three papers just published in the journal Cognitive Therapy and Research, tell contrasting stories: two are by leading lights in IAPT assessing the competence of trainees, in neither study did they demonstrate any real world outcome. By contrast in a study by Perrin et al (2019) of individual CBT for children (aged 10 to 18) suffering from generalised anxiety disorder 80% no longer had GAD by the end of 10 sessions of treatment compared to 0% in the waiting list.   These impressive results were maintained at 3 month follow up.

IAPT could learn from the Perrin et al (2019) study in that client’s diagnostic status was assessed using  a standardised diagnostic interview  and again at the end of treatment using blind assessors, further therapists followed an evidence based protocol for the identified disorder. Whilst it is costly to make such rigorous assessments and IAPT might fear having to explain to Clinical Commissioning Groups the necessary change in modus operandi, IAPT might then at last make a socially significant difference.

IAPT has been provisionally scheduled to be the focus of presentations on BBC TV and Radio on Wednesday, November 13th. 

Perrin et al (2019) Cognitive Therapy and Research (2019) 43:1051–1064
https://doi.org/10.1007/s10608-019-10020-3

Liness et al (2019) Cognitive Therapy and Research (2019) 43:959–970
https://doi.org/10.1007/s10608-019-10024-z

Liness et al (2019) Cognitive Therapy and Research (2019) 43:631–641
https://doi.org/10.1007/s10608-018-9987-5

Dr Mike Scott

National Audit Office Failed To Audit Improving Access To Psychological Therapies Service

 

because it was ‘too busy’. In response to a freedom of information request, from Liverpool, Consultant Psychologist, Dr Mike Scott, the NAO said on November 1st 2019 that amongst its’ reasons for curtailment of its’ investigation were Brexit, the collapse of Carillion and spending increases on generic medicines.  Further the cost of its’ incomplete investigation in 2017-2018 was £74,000. But Dr Scott comments that the reasons that prompted the investigation still remain. He adds that the IAPT service has cost the taxpayer over £3 billion in the last decade with no independent audit of outcome. Clinical Commissioning Groups have simply taken at face value IAPT’s marking of its’ own homework – whither accountability? The NAO response is a yet further illustration that despite official assurances mental health is at the bottom of the agenda. Is it beyond the political parties to go beyond the rhetoric on mental health at the forthcoming general election and commit to an independent inquiry as to how IAPT client’s actually fare? 

In his submission to the NAO Dr Scott pointed out that IAPT had never been subjected to independent audit using the ‘gold standard’ methodology that has been used to assess the effectiveness of a drug. His own published research see link https://journals.sagepub.com/doi/10.1177/1359105318755264 has suggested that only the tip of the iceberg of IAPT client’s recover much less than the 50% claimed by the Organisation. The Journal of Health Psychology also published 3  commentary papers and a rebuttal paper by Dr Scott.  He suggests  that Clinical Commissioning Groups should in the short term refuse to fund the low intensity interventions (guided self- help, computerised cognitive behaviour therapy and educational classes)  that the majority of IAPT clients receive and for which the evidence base is particularly weak, in favour of funding the face to face psychological therapies and for the long term insist that they will be guided by an appropriate independent audit.

Dr Mike Scott

 

 

 

 

 

Expansion Into Long Term Conditions By IAPT Is Quackery

so challenge Clinical Commissioning Groups on the value for money – no better than homeopathy. Studies of CBT  for long term conditions (LTCs) show either no effect, see Serfaty et al study (2019) on cancer https://www.cambridge.org/core/journals/the-british-journal-of-psychiatry/article/effectiveness-of-cognitivebehavioural-therapy-for-depression-in-advanced-cancer-cantalk-randomised-controlled-trial/E9264C516634EC7BC3FF9E80B551A8C5

and/or rely on a self report measure of questionable real world significance, see the Everitt et al  (2019) study of irritable bowel syndrome https://www.thelancet.com/journals/langas/article/PIIS2468-1253(19)30243-2/fulltext In the Everitt et al (2019) study outcome was assessed primarily by the patient’s completion of a 5 item measure of the severity of IBS (IBS-SSS) rather than a clinician independent of the study asking the IBS-Adequate Relief  question ‘ since… have you had adequate relief of your IBS’. Usually the IBS-AR uses the time frame of the past seven days but in the context of assessing CBT it could be since entering the study for a control group or since cbt for those having cbt.  The correlation between the IBS-AR and IBS-SSS though significant is small see Passos et al (2009) http://nrs.harvard.edu/urn-3:HUL.InstRepos:35859644

The authors of the Everitt et al (2019) study appear not to realise  that use of a self-report measure as the primary outcome measure introduces a demand effect for clients undergoing cbt, they don’t want to feel that they have wasted their time. Further the Passos et al (2009) study showed that the IBS-AR is much less subject to fluctuation than the IBS-SSS. Arguably the IBS-AR is of much greater social significance, addressing whether an intervention makes a real world difference. 

It is worrying that Everitt et al (2019) opine:

‘Offering both web-CBT and telephone-CBT in NHS services such as Improving Access to Psychological Therapy could allow many patients to gain substantial benefits with web-CBT with minimal therapist input while allowing a step-up approach to telephone-CBT for those needing additional
support’

IAPT will surely jump on this to justify empire building and likely ignore the caution of Serfaty et al (2019) 

‘our results suggest that resources for a relatively costly therapy such as IAPT-delivered CBT should not be considered as a first-line treatment for depression in advanced cancer. Indeed, these  findings raise important questions about the need to further evaluate the use of IAPT for people with comorbid severe illness’

If as seems likely Clinical Commisioning Groups fund IAPT’s expansion into LTCs they should be asked to justify this expenditure in the abscence of any empirical base.  

Dr Mike Scott

Only The Client Knows Whether Psychological Treatment Has Made a Clinically Relevant Difference

trouble is nobody asks them! When was the last time you remember a client being asked ‘are you back to your usual self with the treatment you have had’? Organisations, such as IAPT have their own metric, a decrease on a psychometric test and in secondary care psychiatrists will opine ‘seems a bit brighter to day, increase…’. These ‘metrics’ ensure the survival of the Organisation, but have no demonstrated relationship to loss of diagnostic status as assessed by a clinician independent of the service provider.

In a study by Stegenga et al (2012) see link below depressed patients were followed up over 3 years whether there depression took a chronic (17%), fluctuating (40%) or remitting course (43%) course they all showed decreases in PHQ9 scores throughout the study and without any psychological intervention. The only exception was a worsening of PHQ9 score at 6 months for the chronic subgroup. Similarly a 12 year study of anxious patients Bruce et al (2005) showed they were only suffering from their anxiety disorder 80% of the time. Thus finding a decreased psychometric test score per se does not mean anything.

Bruce et al (2005) linkhttps://www.dropbox.com/s/9powmto8miw60a2/Natural%20recovery%20in%20Social%20Phobia%20Panic%20Disporder%20and%20Generalised%20Anxiety%20Disorder.pdf?dl=0

Stegenga et al (2012) linkhttps://www.dropbox.com/s/k0x2fm0ds01no0k/natural%20course%20of%20depression%20stegenga%202012.pdf?dl=0

Organisations and Clinical Commissioning Groups much prefer to talk about operational matters, numbers and waiting lists and show no interest or expertise in reliably assessing clinically relevant outcomes. But it is not just these bodies, the leading journals have for the past decade predominantly published papers on the efficacy of psychological interventions with no insistence that there should have been blind independent assessment. Instead self-report measures have ruled with little awareness that their completion is subject to demand effects and the measures often bear no obvious relationship to the construct under examination.

It is difficult to escape the conclusion that clients are largely fodder for the Organisations. A problem that will not be resolved by increased funding for mental health services albeit that this is clearly needed or by atypical clients as tokens on mental health bodies. The fundamental problem is a lack of respect/reverence for clients.

Dr Mike Scott

Clinical Commissioning Groups Need To Know What Actually Happens Behind IAPT’s Closed Doors

this can be achieved by asking local GPs to ask patients about their experience and crucially to determine what proportion of patients returned to normal functioning after referral to IAPT.

Most IAPT clients receive low intensity CBT, with only 20% recovering, half of whom relapse in a year [ Ali et al (2017)]. Only 10% of LICBT patients are stepped up to high intensity. Independent assessment suggests the overall recovery rate in IAPT is just 15%.[ Scott (2018)] https://www.dropbox.com/s/flvxtq2jyhmn6i1/IAPT%20The%20Need%20for%20Radical%20Reform.pdf?dl=0

Results Show IAPT To Be No Better Than Pre-existing Services

A study from 2006 profiled the improvement rates of 32 primary care counselling services using the CORE Outcome Measure. (CORE-OM). The mean level of reliable improvement (including clients that also recovered)  was 72%. Across IAPT, the reliable improvement figure was 66%. But services can be re-organised to transform IAPT Scott (2018)

https://www.dropbox.com/s/zhr1fkg71aqvno0/Transforming%20IAPT.pdf?dl=0

The Failure To Inspect

CCG’s and the National Audit Office show a conspicuous lack of interest in what is happening behind the closed doors of IAPT, preferring to take the Organisations marketing at face value. IAPT appears not to be accountable to the Care Quality Commission. But the CQC’s failure to effectively monitor institutions catering for those with learning difficulties and autism has unearthed a scandal, and instils little confidence in a critical appraisal of IAPT anytime soon.

An Illustration Of The Travails of a Low Intensity IAPT Recipient

Ted’s case illustrates the dire quality of service, he met IAPT in 2014, the records stated that he had been a worrier all his life, but no diagnosis was made. He was no better after 18 months of low intensity cbt. A lost soul:

Initially Ted was directed to a Psychological Wellbeing Practitioner and computerised CBT, Beating the Blues. Ted is recorded as finding the sessions helpful. At the end of LICBT it is recorded that

‘he would prefer not to access cbt again as good understanding of how his negative thoughts impact his behaviour regularly reads his previous cbt notes but implementation does not improve mood’ his psychometric test results are shown below, ‘his billboard’:

    PHQ9GAD7  
Feb 14   10   14  
 March 14 8   7
  May 14 5   9
  July 16 21 15
  August 16 20   18
     
     

At the end of his low intensity journey, there was again no assessment of his diagnostic status and he was understandably not enthusiastic about further CBT. It seems likely that few people are stepped up from low intensity to high intensity because cbt is at best seen as having limited utility.

Ali et al (2017) How durable is the effect of low intensity CBT for depression and anxiety? Remission and relapse in a longitudinal cohort study Behaviour Research and Therapy 94 (2017) 1-8

Dr Mike Scott

Clinical Commissioning Groups, IAPT’s Fairy Godmother

bestowing their munificence without any audit by GPs of local benefit, at a cost nationally of billions of pounds. Yet it should be a simple matter for any GP to interrogate the practice database of IAPT ‘beneficiaries’ and ask the patient the basic question ‘are you back to your usual self since seeing IAPT’? and to further determine whether recovery is stable and reliable by asking ‘for how long have you been back to your usual self?’ Then to integrate the responses with any recent record of functioning in the record of Consultations. Such data can then be presented to the local GP reps on the CCG’s to decide whether the local IAPT is value for money.

CCG’s need to move beyond simple operational matters of numbers of patients seen and waiting times, to a determination of the percentage of people recovering. The randomised controlled trials of cognitive behaviour therapy for depression and the anxiety disorders have suggested a 50% recovery rate when there has been blind assesment of patients. This was the original justification for IAPT. The suspicion is from my independent analysis of 90 IAPT cases that in routine practice the recovery rate is about 10% see link below

https://www.dropbox.com/s/flvxtq2jyhmn6i1/IAPT%20The%20Need%20for%20Radical%20Reform.pdf?dl=0

However when IAPT marks its’ own homework it miraculously comes up with a 50% recovery rate and has seduced CCGs with its own data. The response of most GPs to this is ‘give us a break, but I am nevertheless grateful for a respite from the patient if they are seeing someone else, so I can get on with my core tasks’. We need to move on to a point where GPs are to a degree advocates for their patients, if they don’t do it no one else will. Without such advocacy mental health patients become not just Cinderellas compared to patients with physical problems but confined to their own personal asylum.

Image result for clinical commissioning groups

It is perfectly possible transform IAPT so that it properly translates the findings of rcts into routine practice, see my trio of Simply Effective Cognitive Behaviour Therapy books published by Routledge and my last book Towards a Mental Health System that Works (2017) London; Routledge. But we need to wake up and smell the coffee.

Dr Mike Scott

IAPT – Improving Access to Placebo Therapies

There is no evidence that IAPT’s psychological interventions are better than placebo and the organisation ought to be renamed Improving Access to Placebo Therapies. This calls into question the unswerving devotion of Clinical Commissioning Groups (CCG’s) and the British Association of Behavioural and Cognitive Psychotherapy (BABCP) towards IAPT.

Expectations exert a powerful influence on any psychological therapy, yet in no IAPT study or analysis of its’ own data, has there been a comparison of the IAPT intervention, with that of a group who expected to get better with a particular intervention. There is no reason to believe that IAPT’s results exceed that of a placebo.

IAPT claims to follow NICE Guidelines in delivering evidence-based treatments (ebts) for psychological disorders. But as it takes no steps to reliably identify disorder/s thus there can be no certainty that an ebt is being used that matches the debility.

GPs’ Cognitive Dispositions To Respond Promotes IAPT

Seeing a GP is a common first step along the IAPT pathway, this of itself is likely to increase expectations that something can be done about the presenting problem. The patient then invests time and energy in the said IAPT intervention, at the end of that period he/she does not want to think they have wasted their efforts. Particularly so if the therapist has been ‘nice’, there is a desire to please him/her but this does not mean that they have met criteria for recovery as defined by NICE, i.e they would no longer be eligible to enter a randomised controlled trial for the disorder from which they were originally suffering.

GP’s might be glad of the placebo effect in that it gives them a brief respite from the patient. But because a placebo does not address the mechanism involved in the generation of a disorder, difficulties are ongoing.

It is easy for GP’s to convince themselves that the IAPT interventions are making a difference because in fact, at least for the anxiety disorders, patients naturally only suffer from a condition for 80% of the time.

Thus a GP can doubtless see a post IAPT client in a good state, the vividnes of this experience (availability heuristic) then gives a mistaken impression of how likely this sequence of events is likely to be and the improvement is attributed to IAPT’s efforts (mis-attribution bias), unfortunately the next time a post IAPT patient is encountered in a good state this is seen as confirmation of their believe (confirmatory bias) in the value of the service. Such GPs may unfortunately play a major part in the CCG’s leading to the perpetuation of a failed service.

Resources

  1. Placebo response, Boot et al (2013) click link below: https://www.dropbox.com/s/fnmuv4t6imdcsug/Placebos%20Boot%20et%20al%202013.pdf?dl=0
  2. Not always got a disorder Bruce et al (2005) click link below

https://www.dropbox.com/s/9powmto8miw60a2/Natural%20recovery%20in%20Social%20Phobia%20Panic%20Disporder%20and%20Generalised%20Anxiety%20Disorder.pdf?dl=0

3. Information processing biases see link below

https://www.dropbox.com/sh/66o4qo8ru8sairz/AABxU_IeXeEcaNOqeEYBgGNOa?dl=0

Dr Mike Scott

IAPT – The Need For A Product Recall

In response to David Clark’s blog ‘IAPT at 10’ on the NHS England website, I wrote: ‘If NHS England invited the manufacturer of a pharmaceutical to review the growth and successes of its’ drug over the last decade eyebrows would be raised. Yet this is precisely what has happened in asking David Clark to comment on his baby (IAPT) with whom he has an ongoing commitment and financial arrangement. In terms of publication bias his piece is off the scale.

No Independent Replication

There has never been independent replication of IAPT’s claim to 50% recovery. My own work, which is wholly independent of IAPT and was published in the Journal of Health Psychology   last year (see link below)  suggests a 10% recovery rate.

https://www.dropbox.com/s/flvxtq2jyhmn6i1/IAPT%20The%20Need%20for%20Radical%20Reform.pdf?dl=0

Questionnaires Rather Than An Independently Administered Standardised Diagnostic Interview

IAPT relies on questionnaires completed by clients with the full knowledge of the treating clinician, introducing a ‘demand’ element into the proceedings. Further there is in IAPT’s procedures no way of knowing that the questionnaire/s are tapping the disorder/s that are germane to the client. 

No Evidence of An Added Value To IAPT When Compared With Findings Before Its’ Inception

The changes in questionnaire scores observed in IAPT clients are no different to those observed on self-report measures administered to clients going through counselling before the advent of IAPT. The Mullin (2006) findings (see link below) are the appropriate counterfactual and indicate no added value to IAPT.

https://www.dropbox.com/s/8a4qv5r13rotkyy/Appropriate%20Counterfactual%20Mullin%202006.pdf?dl=0

Clients present for therapy at their worst and some improvement with time would inevitably be visible on a questionnaire, IAPT has provided no evidence that clients given simply attention would not have shown the same changes to those observed.

The Jettisoning of Evaluation Guidelines

Entry into Pharmaceutical/Psychological Studies is governed by the administration of a standardised diagnostic interview. Outcome is determined by blind re-administration of the interview at the end of treatment and follow up. In line with this, an international team of Experts [Guidi et al (2018) see link below] have developed evaluation guidelines stipulating the need for blind independent assessment of psychological interventions. All IAPT generated studies have breached these guidelines.

https://www.dropbox.com/s/hizta38yqm4lfh3/Methodological%20Recommendations%20for%20Trials%20of%20Psychological%20Interventions.pdf?dl=0

Countries that do not look at psychological interventions through the lens of such evaluation guidelines will be taken in by IAPT’s marketing prowess. Unfortunately many such countries have shown such gullibility in the last decade.

Failure to Engage and Treat Clients

IAPT loudly proclaims the very large number of clients that it makes contact with but this is meaningless when their trajectory is considered. Half of those referred to or referring themselves to IAPT   have less than 2 treatment sessions.  The mean number of sessions attended for those who have 2 or more sessions is 6, there is no NICE approved treatment for a psychological disorder that requires just 6 sessions.  It is scarcely credible that IAPT is providing an evidence based treatment on any scale. There is an an independent re-analysis of the IAPT data in the link below

http://therapymeetsnumbers.com/is-iapt-too-big-to-fail/

A Failure of Governance

IAPT is essentially a QUANGO dependent on NHS England, and committed to expansion but without any observance of evaluation guidelines.  NHS England has taken IAPT’s claims at face value, as a consequence Clinical Commissioning Groups focus only on operational matter, numbers, waiting times etc with no focus on clinical matters in their interactions with IAPT. The National Audit Office conducted an inquiry into IAPT but has failed to publish its’ results. There has been a gross failure of governance by public bodies and their representatives.

Only The Voice Of IAPT’s Hierarchy Is Listened To

There has been no attempt by public bodies to independently seek the views of consumers of IAPT services. However an IAPT teacher, Jason Roscoe has publicly made a blistering attack on the service, see link below

https://www.dropbox.com/s/myz53dyn8zqhj13/Has%20IAPT%20become%20a%20bit%20like%20Frankenstein.docx?dl=0

He reflects ‘the gap between what the literature advises and what management allow seems to be widening leaving the patients as the ones who are being given sub-therapeutic, watered-down CBT’ and adds ‘The result? A revolving door where patients return in quick succession for multiple episodes of treatment with a different therapist each time…..not only this IAPT also seems to be making its own workers ill with reports of compassion fatigue and burnout not uncommon’.

The views of the 90 IAPT clients I examined were almost wholly negative and indicated the need to transform IAPT see link below

https://www.dropbox.com/s/zhr1fkg71aqvno0/Transforming%20IAPT.pdf?dl=0

IAPT The Need For Product Recall

There are such serious doubts about what IAPT has delivered over the last decade, that if it were a piece of machinery the product would have been recalled. A decade ago I wrote a book on how CBT can be delivered, with fidelity to evidence based treatment protocols, [Scott (2009) Simply Effective Cognitive Behaviour Therapy, London: Routledge], there is a pressing need to review such provision. In private communication with David Clark I have acknowledged that my approach would make the assessment process more costly. However the evidence of the past decade is that it is not possible to make a real world difference to client’s lives without closely following the procedures involved in randomised controlled trials of CBT. Departure from reliable assessment, diagnosis, advice/treatment results in a failure to translate efficacious treatments to routine practice’.

Unfortunately NHS England only permits upto 1000 character comments on their invited blogs, so essentially only the 1st paragraph of this blog will likely appear.

Dr Mike Scott

Where IAPT Has Never Happened, No Evidence Of Worse Outcome

Clinical Commissioning Groups (CCG’s) should consider why other parts of the UK have not followed England’s lead on IAPT, after more than a decade. Wales, Scotland and Northern Ireland  have remained unimpressed by IAPT’s groundbreaking claims and have not followed suit.  In Wales almost 40% of people surveyed said ‘yes’ or ‘mostly’ when asked had the services they accessed led to improved mental health and wellbeing  [Gofal (2016) Peoples experiences of primary mental health services in Wales Three Years On].  The results show that the largest proportion of respondents (79%) were offered prescription medication. The proportion of people who felt that they has been offered advice and information was 77%. 21.5% were offered Cognitive Behavioural Therapy, while 32% were offered another form of psychological therapy. 36% were offered a further mental health assessment. 26% were referred to another service and 17% were signposted to another service. Just 12% were offered physical exercise, 10% were offered books on prescription and 3% were offered befriending. If you have a mental health problem in Wales it is not obviously worth the trip across the border to an English IAPT service.

There are undoubtedly serious problems with mental health services across the UK, but these are no less in England despite IAPT. .

Dr Mike Scott

BBC Investigates IAPT – A Stepping Stone To Accountability

On Friday the BBC TV broadcast an interview with me in which I said IAPT’s recovery rate was just 15%, this contrasts with IAPT’s claim of a 50% recovery rate, arrived at by marking their own  homework. Further I noted that using IAPT’s own data there is a threefold  difference in recovery rate by geographic area, if there were such differences in outcome with heart bypass surgery it would be thought that something was seriously amiss.  It is great that the subject of IAPT has finally come under public scrutiny but Clinical Commissioning Groups, MPs, and professional bodies have to put IAPT in the dock and question whether the £1bn expenditure on it has been justified. That there has been no funded independent assessment is an ongoing scandal.

The BBC Investigation  can be accessed using the  following link

http://www.bbc.co.uk/news/health-45895541

interestingly IAPT focussed the BBC’s attention on the 1 in 7 geographic areas that fail to reach its’ 50% yardstick, rather than that only 1 in 7 of its’ clients overall recover.

 

Dr Mike Scott