Unnecessary Treatment Is The Rule In IAPT – Due Diligence?

 

The UK Government, Improving Access to Psychological Therapies (IAP) only uses psychometric test screening measures  to assess clients, most commonly the  PHQ9 ( a measure of the severity of depression) and GAD7 (a measure of the severity of generalised anxiety disorder), but other measures are advised for other disorders, such as the PCL-5 for PTSD. A study by Zimmerman and Matia (2001) [The Psychiatric Diagnostic Screening Questionnaire: development, reliability and validity. Comprehensive psychiatry, 42(3), 175–189. https://doi.org/10.1053/comp.2001.23126 ] showed that questionnaire measures that reflect DSM criteria have a roughly 90% sensitivity across major depressive disorder, PTSD, panic disorder, social phobia and GAD, i.e it correctly identifies 9 out of 10 of those who do have one of these disorders. But it identifies only about 60% (specificity) of those who do not have the disorder and for GAD only 50%.  However many more people do not have a particular disorder than have one, leading to unnecessary treatment for many. The National Audit Office should take note of this and re-instate its’ investigation, where is the due diligence with regards to IAPT? £4billion has been given to IAPT!

Depression

In the Zimmerman  and Mattia (2001) study 47.9% of the psychiatric outpatients had major depression. Assuming psychiatric outpatients are a reasonable approximation to the IAPT population, then in a sample of 100 patients approx. 50 would have depression and 50 would not. Of the 50 with depression, 45 would have been correctly identified and treated. However of the 50 who did not have depression only, 30 would have been correctly identified leaving 20 as false positives, candidates for inapropriate treatment. Thus roughly for every two depressed cases appropriately treated one would be inappropriately treated. For depression the appropriate/inappropriate ratio is 2/1 – pretty wasteful.

Generalised Anxiety Disorder

In the Zimmerman Mattia Study 17.5% pf the psychiatric outpatients  had GAD. Thus in a sample of 100 patients approx. 18 would have GAD, of whom 16 would have been correctly identified and treated. But 82 would not have GAD but 50% of them would have been regarded as having GAD meaning that 41 would have been inappropiately treated. Thus for GAD the appropriate/inappropriate ratio is 16/41, so that for every one GAD client treated appropriately 2-3 others are treated inappropriately.

Post-traumatic Stress Disorder

In the Zimmerman and Mattia study 10.5% of the psychiatric outpatients had PTSD. Thus in a sample of 100 clients approx. 11 would have PTSD with 9 being correctly classified and treated. However 89 would not have PTSD of these 62% (55) were correctly classified, meaning that 34 were false positives. Thus the ratio of appropriately treated/ inappropriately treated is approximately 1/4 , for every one treated appropriately 4 are treated inappropriately.

IAPT’s Preposterous Claim On Recovery

Given the ubiquity of unnecessary treatment in IAPT, its’ claim of a 50% recovery rate [IAPT Manual (2019)] is preposterous.  I found a 10% recovery rate Scott (2018) https://doi.org/10.1177%2F1359105318755264, which is much more likely if a body relies simply on a screening instrument.

The Need To Translate Research Methodology Into Routine Practice

Ehlers et al. Trials (2020) 21:355 https://doi.org/10.1186/s13063-020-4176-8 have used the PDSQ to screen for cases of PTSD in their study of therapist assisted treatment for the condition, but have followed the screen up by using a standardised semi-structured interview the SCID to then diagnose PTSD. In this study they have kept a screen in its place and not allowed it free rein as in IAPT.  The IAPT Manual p25 states ‘To ensure that all relevant problems are identified, it is recommended that assessments include systematic screening for each of the conditions that IAPT treats. Standardised commercial screening questionnaire that cover the full range of problems and that can be completed by people before they attend an assessment can be considered ‘ and cites the  PDSQ as an example. But sole use of any screening instrument is very wasteful.

Ehlers et al (2020) have sought to establish whether no more than 4 hours therapist time can make a real world difference to PTSD sufferers lives, a consummation devoutly to be wished, these authors could be well employed helping IAPT get its’ own house in order.

 

Dr Mike Scott

Following NICE Guidance On Covid Treatment Threatens To Overwhelm Mental Health Services

Yesterday NICE issued guidance on the management of Covid post 12 weeks (long term) https://www.nice.org.uk/guidance/ng188 and recommends that those with mild anxiety or mild depression are referred to mental health services, with severe cases of anxiety/depression referred to psychiatrists. IAPT (Improving Access to Psychological Therapies) has already been conducting webinars for its’ Step 3 staff, within which concerns were expressed about possibly overwhelming services and the pathologising of normality. Despite this further webinars are planned for the low intensity (Step 2) staff. Buoyed by its’ success in attracting monies for psychological therapies for long-term conditions (LTCs), such as chronic pain, irritable bowel syndrome, IAPT sees an opportunity to extend its’ reach to those affected by Covid. Those with long term Covid are likely to suffer the same fate of those with Chronic Fatigue Syndrome of not being really listened to. 

Given that according to NICE the most common features of long term Covid are fatigue, ‘brain fog’ and breathlessness, and that ‘symptoms of anxiety and depression’  are presented as possible symptoms of Covid at any stage, how is it possible to make an additional diagnosis of anxiety and depression? With the exception of the few, Covid patients who may be suicidal the distinction between the physical and psychological symptoms is fraught with difficulties. One response is to ignore the distinction, ignore the science and claim that all with Covid need a psychological therapist, but there is no scientific evidence for this – albeit that it suits the purposes of service providers to make such a claim. If you were not feeling ‘mildly anxious or depressed’ when you contract Covid that is probably very worrying!

An editorial in the British Medical Journal http://dx.doi.org/10.1136/bmj.m4425 bemoans the medico-political contexts that has hampered scientists expressing their concerns over the evidence base for handling Covid. But such a medico-political context has operated for years with regard to IAPT. There has been no independent evidence that IAPT’s work with sufferer’s from LTC’s has led to the resolution of accompanying psychological disorders. There has been no comparison with an active placebo or with the fate of LTC sufferers before the advent of IAPT.  The National Audit Office was allowed to suspend its’ investigation of IAPT in 2017, with no check on the appropriateness of having spent £4 billion of the public purse on the Service. Matters have been compounded by the BABCP’s (the lead organisation for cbt) unwavering support for IAPT and the British Psychological Society’s endorsement of IAPT training. Despite any evidence that the competence of therapists trained relates to client outcome Liness et al (2019) https://www.dropbox.com/s/e26n191ie09sngs/Competence%20and%20Outcome%20IAPT%20no%20relation%202019.pdf?dl=0.

2021 can only get better, one needs hope, I think that this is the message of Christmas.

Dr Mike Scott

IAPT’s Black Hole – Accountability

I recently asked the National Audit Office to restart it’s investigation into IAPT. I am expecting their reply in the next week or two. There has been no independent scrutiny of IAPT. They have been answerable only to Clinical Commissioning Groups, which have consisted largely of GPs and allowed IAPT to mark its’ own homework.

But the accountability gap also extends downwards, where is the evidence that front line staff or clients have been consulted or involved in decision making?  Most recently IAPT has offered webinars, for its staff on helping those with long term COVID.   There is a tacit assumption that this will be within the expertise of IAPT therapists just as helping those with long term physical conditions such as irritable bowel syndrome. But the IAPT staff working with LTCs were never consulted, before this new foray. Client’s with LTCs were never asked whether they were back to their old selves (or best functioning) before this proposed further extension of IAPT’s empire.  

In the forthcoming issue of the British Journal of Clinical Psychology I have challenged IAPT’s account of its ‘performance’ see ‘Ensuring IAPT Does What It Says On The Tin’ https://doi.org/10.1111/bjc.12264. There is a reply in rebuttal see ‘The costs and benefits of practice-based evidence: correcting some misunderstandings about the 10-year meta-analysis of IAPT studies’ https://doi.org/10.1111/bjc.12268 that reveals a breathtaking level of conflict of interests. IAPT and its’ fellow travellers should be held to account. But importantly they also need to account to their therapists and clients. [ The original IAPT paper is available at https://doi.org/10.1111/bjc.12259]

 

Dr Mike Scott

£4 billion Spent On IAPT and National Audit Office Says ‘Value For Money’ Is Not Our Concern

and we  have ‘no plans to revisit work’. This was the National Audit Office’s response to a Freedom of Information Request to my colleague Joan Crawford on February 18th 2020 http://FOI-1298.  The NAO’s response continues our ‘purpose was to establish the relevant facts’ with regards to waiting times. The NAO adds it was these concerns expressed by an NHS staff member that first led to the NAO inquiry.  Curiously the NAO  then says  it was planned to describe the responsibilities of different health sector bodies  ‘in assuring and overseeing the accuracy and integrity of the reported data’ but the NAO has never addressed the ‘integrity’ of the data. The NAO has failed to tell anyone that the unit of analysis in IAPT’s own data set, the proportion of people who recover, is suspect. Whilst IAPT claims a 50% recovery rate my own independent analysis submitted to the NAO suggests it is just 10%.  The NAO curtailed its’ investigation in June 2018 because of ‘changing priorities’, leaving its’ findings unpublished and duties undischarged. There is a pressing need for independent audit. 

On the same day as the FOI response, the Government announced that it will spend £2.6 billion over 6 years on flood defences and that every £1 spent will save £8. However the UK Government has failed to ensure that the NAO furnishes it with similar data. Without such data there is no real world accountability.

 

Dr Mike Scott

DWP Scores Claimant With Multiple Diagnoses As Zero Impaired And She Loses Benefits

and suffers a depressive reaction. I was preparing a desktop report for Ms X just as a National Audit Office (NAO) Report into the DWPs procedures was announced. It revealed that the DWP were investigating 69 cases of suicide following cessation of Personal Independence Payments (PIPs). The NAO observed that the true extent of suicide in this context is unknown. It is time to put the mental sequelae of DWPs decisions on the agenda.  

Extensive documentation on Ms X reveals recurrent depressive disorder, autism and adult ADHD, together with years of contact with secondary care mental health services. Despite this the DWP assessor indicated that she had 0 problems communicating and interacting with others! Reading his letter of justification he relied entirely on his perception of how she presented at interview.  He gave a total Summary score of 0 which is simply preposterous, whether or not she had sufficient points to meet the PIPs criteria. I have written to the DWP for a review of the case.  I have also suggested that not only should suicides be subjected to an Internal Process Review but all Claimants who are judged to have scored 0. Such a change in PIP score is near miraculous as people are awarded PIPs initially because of enduring functional impairment. But IPR’s are not open to public scrutiny and the NAO pointed out that it is not known whether such reviews have led to any change in practice: There is a need for transparency, I await with interest the DWP’s response to my letter.

Unfortunately the DWP’s assessor has adopted the style of  most mental health professionals, reliance primarily on a single source of data (IAPT on self report measures of doubtful relevance ) or the clinicians take on the client’s story with an open ended interview.  This results in missed diagnosis, mistreatment and misleading statements about the client’s diagnostic status.

Dr Mike Scott

 

 

 

 

 

 

Audit of Secondary Care Psychological Therapies Fails Clients

The National Clinical Audit of Anxiety and Depression (NCAAD) has just been published https://www.rcpsych.ac.uk/members/your-monthly-enewsletter/january-2020-enewsletter/anxiety-and-depression-report?dm_i=3S89,13323,2H3J22,3SCFB,1 but it is impossible to gauge from it what proportion of those with anxiety or depression recovered with psychological treatment. There was no reliable methodology employed to determine what constituted a ‘case’ of anxiety or depression and there was no independent evaluation of outcome. 

No evidence is provided that psychological therapy made a real world difference to client’s lives.  The authors reported that 75% of service users agreed that their therapy helped them to cope with their difficulties, with 88% agreeing they were treated with empathy, dignity and kindness.  The average number of treatment sessions attended was 13. Having made such a time investment clients  are unlikely to be critical of the service they received particularly, as was usually the case, the therapist was judged a nice person. 

The report opines that 65% of service users were receiving a type of therapy in line with NICE Guidance for their disorder. But given that diagnostic status was not reliably determined there can be no certainty that an appropriate NICE protocol was used. There is nothing in the report to indicate that treatment records were reviewed (or capable of review) in such a way as to determine matching treatment targets, strategies and disorder. This makes one sceptical of the authors claim that the main intervention was CBT, it is alleged CBT. With just over a half completing the planned number of sessions. With a further 1 in 3 people receiving a type of treatment that was not NICE compliant even by the standards of the authors of the report.  

The authors call for an increased use of psychometric tests (no test was used in more than 15% of cases) and a reduction of waiting times (almost half waited over 18 weeks). Doubtless these are laudable aims but of themselves are unlikely to make any real world difference to client’s lives.

There is a legitimation of current practice, with implicit claims for more funding and better training, all horribly reminiscent of the failed IAPT service.  The National Audit Office needs to not only re-ignite its’ inquiry into IAPT but also determine whether secondary care psychological therapy is value for money – the NCAAD provides no evidence of the latter.

Dr Mike Scott

 

NHS Improvement Reveals Hole in IAPT’s Bucket

but the NHS has taken no steps to stop the haemorrhaging of clients and money.  A quarter (25.3%) of IAPT’s expenditure in 2017-2018, £75.58 million, was devoted to clients who were either not assessed or were not put in any care cluster (groups of diagnoses), according to the National Schedule of Reference costs see link https://www.dropbox.com/s/3xlu6tipaeguk2c/FOI%20IAPT%20data.xlsx?dl=0

which I acquired through a Freedom of Information Request. Curiously only 15.2% of IAPT’s expenditure, £45.68 million in 2017-2018 went on common mental health problems of low severity, whilst £176.86 million (58.99%) was spent on non-psychotic disorder.

For those with common mental health problems of low severity there were 214,863 high intensity contacts compared to 378,617 low intensity contacts. Thus low severity often appears to necessitate high intensity contacts, this raises questions about the reliability of ‘common mental mental health problems of low severity’ category.

For non-psychotic disorders 1.075 million high intensity contacts were delivered and almost as many low intensity contacts, 0.876 million. Assuming that the those categorised as a having a non psychotic disorder are more functionally impaired than those with common mental health problems of low severity, why are they having so many low intensity contacts? Sticking plasters for serious injury! It is time for the National Audit Office to restart its’ inquiry. 

 

Dr Mike Scott

 

National Audit Office Failed To Audit Improving Access To Psychological Therapies Service

 

because it was ‘too busy’. In response to a freedom of information request, from Liverpool, Consultant Psychologist, Dr Mike Scott, the NAO said on November 1st 2019 that amongst its’ reasons for curtailment of its’ investigation were Brexit, the collapse of Carillion and spending increases on generic medicines.  Further the cost of its’ incomplete investigation in 2017-2018 was £74,000. But Dr Scott comments that the reasons that prompted the investigation still remain. He adds that the IAPT service has cost the taxpayer over £3 billion in the last decade with no independent audit of outcome. Clinical Commissioning Groups have simply taken at face value IAPT’s marking of its’ own homework – whither accountability? The NAO response is a yet further illustration that despite official assurances mental health is at the bottom of the agenda. Is it beyond the political parties to go beyond the rhetoric on mental health at the forthcoming general election and commit to an independent inquiry as to how IAPT client’s actually fare? 

In his submission to the NAO Dr Scott pointed out that IAPT had never been subjected to independent audit using the ‘gold standard’ methodology that has been used to assess the effectiveness of a drug. His own published research see link https://journals.sagepub.com/doi/10.1177/1359105318755264 has suggested that only the tip of the iceberg of IAPT client’s recover much less than the 50% claimed by the Organisation. The Journal of Health Psychology also published 3  commentary papers and a rebuttal paper by Dr Scott.  He suggests  that Clinical Commissioning Groups should in the short term refuse to fund the low intensity interventions (guided self- help, computerised cognitive behaviour therapy and educational classes)  that the majority of IAPT clients receive and for which the evidence base is particularly weak, in favour of funding the face to face psychological therapies and for the long term insist that they will be guided by an appropriate independent audit.

Dr Mike Scott

 

 

 

 

 

IAPT’s Failure To Engage

the IAPT Annual Report (2018)/2019] see link below, reveals that a third (31.2%) of new referrals drop out before treatment and approximately two thirds (61.1%) do not complete a course of treatment (using IAPT’s liberal definition of treatment as attending 2 or more session) with almost a third (29.54 %)  attending only one treatment session.

https://www.dropbox.com/s/hwn9ncuuyds8qfa/IAPT%20Annual%20Report%202018-2019.pdf?dl=0

IAPT’s disengagement is illustrated by Jock’s records which revealed that at age 6 he had behaviour problems and threatened to stab himself.  By age 14 he was diagnosed with oppositional defiance disorder and was short tempered. At age 19  he was diagnosed as having an anxiety state low mood drinking 10 units in a binge once or twice a fortnight  and cannabis 2-3 times a week. Despite his extensive history he was assessed by IAPT and assigned to a step 2 (low intensity) workshop, unsurprisingly he DNA’d. Two years later he is referred to them again for depression and unsurprisinly he does not respond to their opt in letter. Five years later the GP notes that he is struggling with an online CBT course has had to enlist his father to help because he is not computer literate. Then after a major negative life event he develops a depressive psychosis. Had IAPT bothered to listen this troubled soul of longstanding, the results could have been very different.

Institutional Disengagement

Engagement difficulties are built into the fabric of IAPT. Daniel consulted his GP 2 years after a major trauma and was found to have PTSD and depression and was promised a referral to IAPT. 4 weeks later he was prescribed an increase in medication and a different GP gave him IAPT’s telephone number to ring. Daniel was furious, he felt that he had explained that his mood was very up and down and that he could not be relied on to ring them. His interpretation of the organisational setup was that no one was really interested. This perception was likely to be compounded if and when he underwent a telephone assessment as had already had lots of acrimonious telephone conversations with the housing Dept and DWP since his trauma.   

It is surely time f or the Care Quality Commission and the National Audit Office to take note of the near universal disengagement of clients, voting with their feet, and institute an independent review of IAPT to determine what if any real world difference it makes. There is considerable media interest in these failings.

Dr Mike Scott

 

 

IAPT – The Need For A Product Recall

In response to David Clark’s blog ‘IAPT at 10’ on the NHS England website, I wrote: ‘If NHS England invited the manufacturer of a pharmaceutical to review the growth and successes of its’ drug over the last decade eyebrows would be raised. Yet this is precisely what has happened in asking David Clark to comment on his baby (IAPT) with whom he has an ongoing commitment and financial arrangement. In terms of publication bias his piece is off the scale.

No Independent Replication

There has never been independent replication of IAPT’s claim to 50% recovery. My own work, which is wholly independent of IAPT and was published in the Journal of Health Psychology   last year (see link below)  suggests a 10% recovery rate.

https://www.dropbox.com/s/flvxtq2jyhmn6i1/IAPT%20The%20Need%20for%20Radical%20Reform.pdf?dl=0

Questionnaires Rather Than An Independently Administered Standardised Diagnostic Interview

IAPT relies on questionnaires completed by clients with the full knowledge of the treating clinician, introducing a ‘demand’ element into the proceedings. Further there is in IAPT’s procedures no way of knowing that the questionnaire/s are tapping the disorder/s that are germane to the client. 

No Evidence of An Added Value To IAPT When Compared With Findings Before Its’ Inception

The changes in questionnaire scores observed in IAPT clients are no different to those observed on self-report measures administered to clients going through counselling before the advent of IAPT. The Mullin (2006) findings (see link below) are the appropriate counterfactual and indicate no added value to IAPT.

https://www.dropbox.com/s/8a4qv5r13rotkyy/Appropriate%20Counterfactual%20Mullin%202006.pdf?dl=0

Clients present for therapy at their worst and some improvement with time would inevitably be visible on a questionnaire, IAPT has provided no evidence that clients given simply attention would not have shown the same changes to those observed.

The Jettisoning of Evaluation Guidelines

Entry into Pharmaceutical/Psychological Studies is governed by the administration of a standardised diagnostic interview. Outcome is determined by blind re-administration of the interview at the end of treatment and follow up. In line with this, an international team of Experts [Guidi et al (2018) see link below] have developed evaluation guidelines stipulating the need for blind independent assessment of psychological interventions. All IAPT generated studies have breached these guidelines.

https://www.dropbox.com/s/hizta38yqm4lfh3/Methodological%20Recommendations%20for%20Trials%20of%20Psychological%20Interventions.pdf?dl=0

Countries that do not look at psychological interventions through the lens of such evaluation guidelines will be taken in by IAPT’s marketing prowess. Unfortunately many such countries have shown such gullibility in the last decade.

Failure to Engage and Treat Clients

IAPT loudly proclaims the very large number of clients that it makes contact with but this is meaningless when their trajectory is considered. Half of those referred to or referring themselves to IAPT   have less than 2 treatment sessions.  The mean number of sessions attended for those who have 2 or more sessions is 6, there is no NICE approved treatment for a psychological disorder that requires just 6 sessions.  It is scarcely credible that IAPT is providing an evidence based treatment on any scale. There is an an independent re-analysis of the IAPT data in the link below

http://therapymeetsnumbers.com/is-iapt-too-big-to-fail/

A Failure of Governance

IAPT is essentially a QUANGO dependent on NHS England, and committed to expansion but without any observance of evaluation guidelines.  NHS England has taken IAPT’s claims at face value, as a consequence Clinical Commissioning Groups focus only on operational matter, numbers, waiting times etc with no focus on clinical matters in their interactions with IAPT. The National Audit Office conducted an inquiry into IAPT but has failed to publish its’ results. There has been a gross failure of governance by public bodies and their representatives.

Only The Voice Of IAPT’s Hierarchy Is Listened To

There has been no attempt by public bodies to independently seek the views of consumers of IAPT services. However an IAPT teacher, Jason Roscoe has publicly made a blistering attack on the service, see link below

https://www.dropbox.com/s/myz53dyn8zqhj13/Has%20IAPT%20become%20a%20bit%20like%20Frankenstein.docx?dl=0

He reflects ‘the gap between what the literature advises and what management allow seems to be widening leaving the patients as the ones who are being given sub-therapeutic, watered-down CBT’ and adds ‘The result? A revolving door where patients return in quick succession for multiple episodes of treatment with a different therapist each time…..not only this IAPT also seems to be making its own workers ill with reports of compassion fatigue and burnout not uncommon’.

The views of the 90 IAPT clients I examined were almost wholly negative and indicated the need to transform IAPT see link below

https://www.dropbox.com/s/zhr1fkg71aqvno0/Transforming%20IAPT.pdf?dl=0

IAPT The Need For Product Recall

There are such serious doubts about what IAPT has delivered over the last decade, that if it were a piece of machinery the product would have been recalled. A decade ago I wrote a book on how CBT can be delivered, with fidelity to evidence based treatment protocols, [Scott (2009) Simply Effective Cognitive Behaviour Therapy, London: Routledge], there is a pressing need to review such provision. In private communication with David Clark I have acknowledged that my approach would make the assessment process more costly. However the evidence of the past decade is that it is not possible to make a real world difference to client’s lives without closely following the procedures involved in randomised controlled trials of CBT. Departure from reliable assessment, diagnosis, advice/treatment results in a failure to translate efficacious treatments to routine practice’.

Unfortunately NHS England only permits upto 1000 character comments on their invited blogs, so essentially only the 1st paragraph of this blog will likely appear.

Dr Mike Scott