Categories
BABCP Response - NICE Consultation January 2022

The UK’s IAPT Service Is An Abject Failure

 

from ‘Mad In America’ https://www.madinamerica.com/2022/06/uk-iapt-abject-failure/

 

Rescued man in boat looking on shipwreck

Dr Michael J Scott Ph.D

In 2012, an editorial in the prestigious journal Nature claimed that the UK’s IAPT Service is “world-beating”—meaning that the service is the world’s best for treating mental health concerns. Now that 10 years have passed, it’s clear that this was not true. Despite the hype, the IAPT is hardly “world-beating.” In fact, it is a doubtful model for other countries to follow.

The IAPT (“Improving Access to Psychological Therapies”) was created in 2008 and fully funded by the UK government (though restricted to provision only in England). The program claims to provide evidence-based psychological treatment, most commonly in the form of cognitive behavioural therapy. It was initially focused on patients with depression and anxiety disorder diagnoses, but its scope has been gradually expanded to include the psychological treatment of long-term physical conditions such as chronic fatigue syndrome.

Patients either self-refer (at no cost) or are referred to the IAPT by their primary care physician/general practitioner. Clients are initially assessed during a 30-minute telephone conversation by a Psychological Wellbeing Practitioner (PWP) who is not an accredited therapist and is not trained to diagnose. IAPT clients initially receive low-intensity CBT (six hours or less of therapy contact) delivered by a PWP. If they are deemed not to have responded to this, on the basis of a psychometric test, they can be “stepped up” to high-intensity CBT delivered by an accredited psychological therapist.

How Well Is the IAPT Doing?

Proponents of the IAPT cite its outcomes for the clients who complete their treatment sessions—but over half of IAPT clients don’t even attend two sessions. The response of the IAPT to this massive disengagement with the service—clients “voting with their feet,” as it were—is currently to offer training to its staff to “streamline” matters by encouraging them to use a computer algorithm to decide which clients go where. This training, which was held on June 22, 2022, involves expert advice from the CEO of Limbic, an artificial intelligence Company. Thus, not only are IAPT staff to be distracted by clients having to complete psychometric tests at each encounter, but now additionally by the administration of the computer algorithm.

Yet Limbic claims it is “an AI assistant for clinical assessments in IAPT—improving access, reducing costs and freeing up staff time.” Where is the independent evidence for this? What clients really want is to be listened to, but there is little chance of this any time soon. Conquering disengagement in this manner is a forlorn hope. 

IAPT’s 2019 manual states that IAPT staff do not make a diagnosis. However, the treatment recommendations of NICE (the UK’s National Institute for Health and Clinical Excellence) are explicitly tied to a taxonomy of specific treatments for identified disorders (largely, diagnosis-specific variations on CBT). IAPT legitimises itself by a claim to be NICE compliant, but how could they be if they don’t make diagnoses? How can they follow NICE’s specific treatment guidelines without first assessing which condition they are treating?

Whilst the IAPT’s claim to be NICE-compliant might be music to the ears of NHS England and clinical commissioning groups, it is simply not true. No independent fidelity checks have ever have been made on IAPT’s ministrations. IAPT allegedly provides specific CBT protocols for different disorders, but it has provided no evidence that it does this.

In my examination of the physician’s records of 90 IAPT clients, I found no evidence of compliance with CBT protocols.IAPT operates simply as a business with superb marketing.

In no independent audit of IAPT, have clients been asked “are you back to your usual self since this treatment?” This is a goal that is meaningful to the client—with the possible supplementary question, “for how long have you been back to your usual self?” In papers published by IAPT staff, no such real-world outcome measures have ever been used. Instead, reliance has been placed on changes in psychometric test scores. Yet without a control group, it’s impossible to know whether these changes would have occurred with passage of time anyway. After all, people present to psychological treatment at their worst, and at any other time, they are likely to score better. In statistics, this principle is known as regression to the mean, and it is vital to account for this effect.

Further, the client knows his/her therapist sees their completed questionnaire, pushing them to respond more positively. A client may also not want to endorse negative responses on the questionnaire because it would mean acknowledging that they have wasted their time in therapy. These are known issues with self-report measures that are commonly cited as important limitations in research studies.

Finally, the questionnaire results also give no indication of whether any improvement has lasted for a period that the client would regard as meaningful, such as eight weeks or (hopefully) more.

In an era of personalised medicine, it is essential that the voice of the client is heard. It is an almost ubiquitous failure of studies to employ a primary outcome measure that a client can identify with—known as patient-centred outcomes. Perhaps the nearest proxy is loss of diagnostic status (full recovery) determined by an independent assessor. The burden of proof rests on providers of services to demonstrate that their treatments are making an important difference to a client’s life in a way that is clearly recognisable to the client. IAPT has not cleared this evidential bar.

Use of an algorithm is not incompatible with a clinician operating on a knowledge base of what works for whom. But it has to be first demonstrated that the former constitutes added value over the latter. The burden of proof is on AI advocates to demonstrate its relevance in a particular context. This cannot be done by a company with a vested interest in AI.

It is perfectly possible that using two systems, an algorithm and clinical judgement, might create confusion. Should an IAPT client not fare well (for example, if a client dies by suicide) when treated under such a dual system, could IAPT convince a court, on the balance of probability (the UK benchmark in personal injury cases), that its approach was evidence-based? After all, an AI approach is not something that NICE has recommended—so it flies in the face of the guidelines. Traditionally, courts are swayed by eminence-based evidence (such as the word of authority figures like psychiatrists), but they have become more aware (at least in the UK) of the perils of unbridled clinical judgement.

Inadequacies of IAPT’s Chosen Metric

IAPT’s chosen metric is a change in score on two psychometric tests. Clients are deemed to have responded if on the PHQ-9 (a measure of the severity of depression) their test score has gone from above 10 to below 10 with treatment, with a difference of at least 6 points, and if on the GAD 7 (a measure of the severity of generalised anxiety disorder) their score has gone from above 8 to below 8 with treatment, with a difference of at least 4 points. This approach has numerous deficiencies:

  • It is not known whether the completer of these two measures was actually suffering from depression or generalised anxiety disorder, because IAPT clinicians do not make diagnoses. Further, it is not known whether either of these “conditions” was the disorder that was causing most impairment in functioning.
  • When test results are observable by the treating clinician, clients may wish to be polite and relay a lower score, particularly as they do not want to feel that they have wasted their time in therapy.
  • No attention control comparator: Changes in test scores are meaningless unless there is a comparator group who have received the same attention and a credible rationale. It’s unclear whether the therapy resulted in the improvement, or if the client would have improved with a “placebo” therapy (simply receiving professional “attention” without CBT techniques).
  • Regression to the mean: Clients typically present at their worst, so with the mere passage of time they will score more modestly.
  • The vagaries of a minimally important difference (MID) in a score: MIDS are established by statistical comparisons between an “ill” and “well” group, but they do not necessarily indicate clinically important difference. They also fail to address the client’s perception and there is no personalisation of treatment outcome. Clients are given no voice.

Haemorrhaging

Last year about a third of referrals (30.1%) to the IAPT service did not access it. Of those who accessed the service over a third (38%) had just one treatment session. Thus, by the start of the IAPT race over half of people (57%) have not engaged in treatment (as defined by IAPT’s metric of attending two or more sessions). If these results applied to a physiotherapy service it would raise serious doubts about the suitability of the service! Why then does the UK National Audit Office studiously avoid scrutiny of IAPT, which costs the taxpayer over £1 billion a year?

Sub-Therapeutic Dose and Treatment Outcomes

Of those who engaged in IAPT, the average number of sessions was 7.5, which is well below the 12 to 20 sessions that NICE recommends for depression and anxiety disorders (IAPT’s primary targets). Are we to believe that IAPT’s clinicians are so skilled that they can achieve recovery with only half the dosage of therapy as that delivered by therapists in the randomised controlled trials of CBT for depression and the anxiety disorders?

The benchmark set by trials of therapy as a treatment for these conditions is that at least 50% must “recover” (no longer meet criteria for a diagnosis). But there is no evidence that IAPT has achieved this. My own research suggests that only the tip of the iceberg recover.

In my capacity as an Expert Witness to the Court, I examined 90 IAPT clients who had been through the service either before or after the personal injury. Only 9.2% of subjects lost their and diagnostic status, as assessed using the “gold standard” SCID interview. It mattered not whether subjects were assessed before or after their personal injury. (These findings are in need of replication by clinicians independent of IAPT with a non-litigant population.)

Diagnostic Creep

IAPT appears not to so much follow the data, but to follow funding opportunities. It has branched out into territory were angels fear to tread: Medically Unexplained Symptoms (MUS). MUS is an umbrella term embracing conditions as diverse as chronic fatigue syndrome, fibromyalgia, and irritable bowel syndrome. IAPT’s contention is that exaggerated negative beliefs about symptoms, and maladaptive behaviours (e.g., avoidance), play a pivotal role in the maintenance of symptoms in these conditions. The therapeutic task, as they see it, is to then to modify these cognitions and behaviours.

What happens if a client protests that they have valid medical symptoms? Well, IAPT clinicians are taught not to openly disagree and not to say ‘it is all in your mind’. But to nevertheless continue to focuss on the ‘exaggerations in beliefs and avoidance behaviours’, so much for honesty.

Along with Keith Geraghty from the University of Manchester, I have published a critique of this sojourn. We identified a series of seven core problems and failings of the IAPT, including an unproven treatment rationale, a weak and contested evidence-base, biases in treatment promotion, exaggeration of recovery claims, under-reporting of drop-out rates, and a significant risk of misdiagnosis and inappropriate treatment. We concluded that psychotherapy should not become the default option when patients have “medically unexplained symptoms.”

The term “medically unexplained symptoms” should be written in lowercase to avoid conveying the impression that a meaningful homogenous entity is being described. The term should not enter the taxonomy of disorders. Importantly, even the DSM-5 has shunned recourse to the term. But unfortunately, IAPT only pays lip service to the standard diagnostic criteria.

Big Pharma and IAPT

Whilst some large pharmaceutical companies have greatly served the public good with regard to COVID, their performance with regard to psychotropic drugs is much less impressive. Researchers like Read and Moncrieff pin their hopes on the efficacy of CBT demonstrated in randomised controlled trials, but make no mention that CBT as delivered in routine practice bears little relationship to the protocols utilised in RCTs. Just as the claims of pharmaceutical companies require critical appraisal so too does the UK IAPT service.

 In an article in the British Journal of Clinical Psychology, I explained what the IAPT needs to do to get back on track. I noted that in their published papers, IAPT staff do not declare their allegiances and indeed state that they have no conflict of interest! This dishonesty needs to change. I suggested that there is a need for IAPT to prove their case using the standards of proof that are required in other areas of medicine.  To this effect, they cannot simply claim that they deliver evidence-based treatment—there has to be documented evidence for it that would be persuasive to an independent observer.  The IAPT service has to demonstrate that it provides an added value over someone attending an advice centre or counselling service. Yet the IAPT has yet to demonstrate that it makes an important enough difference to clients’ lives that they would recognise it.

This catalogue of omissions may create a sense of déjà vu for those who have followed the machinations of Big Pharma with regards to mental health. Unfortunately, there has only been a deafening silence from the IAPT regarding my critique.

 

 

 

Categories
BABCP Response - NICE Consultation January 2022

The IAPT Double Myth of Economic and Clinical Utility

 

The impetus for the Improving Access to Psychological Therapies (IAPT) service was derived from a) economic considerations and b) an assertion that the positive results of randomised controlled trials of, primarily CBT, for depression and the anxiety disorders would be realised in routine practise.

 

 

A Money-Saver?

In 2006 the Centre for Economic Performance stated that “the total loss of output due to depression and chronic anxiety is some £12 billion a year-1% of our total national income ”. The contention was that investment in psychological therapy would pay for itself by a reduction in such costs. But currently IAPT costs over £1 billion a year, where is the evidence of a substantial reduction in the loss of output? Where is the evidence that IAPT constitutes a no-cost talk therapy?

 

Poor Performance at the Coal-Face

In the randomised controlled trials on average 50% of clients lost their diagnostic status as assessed by independent blind assessors. But no such unbiased assessors have ever gauged the impact of IAPT’s ministrations. IAPT has always marked its own homework. Rather than the claimed recovery rate of 50%, the best available evidence suggests that only the tip of the iceberg recover, Scott (2018).

Each myth means that the other is not carefully examined and IAPT advocates can deftly switch the focus from one to the other under critical scrutiny – a politician’s dream.

 

Psychological Therapy – a history of exaggerated claims

T.S Eliot wrote ‘Humankind cannot bear very much reality’ this applies particularly to looking at the effects of mental heath treatment. In 1751 the scientist and Quaker, Benjamin Franklin petitioned the Pennsylvania colonial assembly for funds to build a hospital on the grounds that ‘it has been found , by the experience of many years, that above two thirds of the Mad People received into the Bethlem Hospital in England  and there treated properly , have been perfectly cured’. He was reiterating claims made in published books by English doctors. Fast forward over 260 years, to 2012 and an editorial in the prestigous journal Nature declares IAPT ‘represents a world-beating standard thanks to the scale of its implementation and the validation of its treatments’ (p. 473)’.  A decade later, NHS England echoes this declaring ‘the Improving Access to Psychological Therapies (IAPT) programme began in 2008 and has transformed the treatment of adult anxiety disorders and depression in England. IAPT is widely-recognised as the most ambitious programme of talking therapies in the world and in the past year alone more than one million people accessed IAPT services for help to overcome their depression and anxiety, and better manage their mental health’ and recommends the IAPT Manual (2021). In 2019, Pickersgill examined the proliferation of IAPT by canvassing the views of professionals and professional bodies, noting that IAPT fellow-travellers were in the ascendancy. But in this evangelisation for the in vogue psychological interventions nobody has asked the consumer or considered the operation of vested interests.

The Absence of Open Discussion

Psychological disorders are ubiquitous and can negatively impact the course of coexisting physical conditions. Since the days of Benjamin Franklin, UK data on mental health treatment, has been used to foster the belief that UK treatments are a ‘world beater’. But independent evidence to support this contention is lacking. There is not just a gap between the psychological treatments delivered in randomised controlled trials and what comes to pass in routine psychological services, such as the Improving Access to Psychological Therapies Programme (IAPT) service, but a chasm. Efforts to have a meaningful debate on the issue have been met with a deafening silence. In the silence, the scope of psychological treatments has gradually been expanded, beyond the initial focus of depression and the anxiety disorders to include patients with long term physical conditions – a psychological imperialism. The power-holders definition of the outcome of routine psychological treatment reigns.

 

Dr Mike Scott

Nature (2012) Editorial: Therapy deficit. Nature 489(7417): 473–474.

Pickersgill M. (2019). Access, accountability, and the proliferation of psychological therapy: On the introduction of the IAPT initiative and the transformation of mental healthcare. Social studies of science49(4), 627–650. https://doi.org/10.1177/0306312719834070

 

 

 

 

 

 

Categories
BABCP Response - NICE Consultation January 2022

Low Intensity CBT, ‘Penny Wise & £ Foolish’

Psychological Wellbeing Practitioners (PWPs) deliver the smallest dose of psychological interventions (low intensity CBT), less than 6 hours of contact per client (Shafran 2021). Making it cheaper than high intensity CBT. But there is little evidence that the PWPs ministrations make a difference the client would recognise. There are no randomised controlled trials of high or moderate quality (Cochrane Grade) that attest to low intensity CBTs efficacy. The PWPs are not psychological therapists, as such, most IAPT clients do not receive psychological therapy. The name IAPT, Improving Access to Psychological Therapies is therefore an example of doublethink. Here we have a classic example of false economy.

Low intensity CBT is intended to be the first step for those suffering from depression and the anxiety disorders, with PTSD and OCD clients going straight to high intensity interventions. Most clients first encounter low intensity CBT, should they not respond they are placed on a waiting list for high intensity CBT. In practice comparatively few, about 10%, are stepped up,  but with wide regional variations.It is not so much stepped care as stopped care.

Implementation of the ‘stepped care’ model costs the taxpayer of over a £1bn a year. But there has been no independent evaluation of the package or its components. IAPT is the first ANGO (an autonomous non- governmental agency) funded by the government, as opposed to the intended and understood QUANGO – a quasi autonomous non-governmental agency. Perhaps the National Audit Office and MPs might care to explain why there has been this failure of governance?

 

Dr Mike Scott

Categories
BABCP Response - NICE Consultation January 2022

Why Fund The IAPT Business?

 

‘Two thirds of GPs providing specialist mental health support beyond their competence’ this was the headline in Pulse, May 9th 2022. This has been brought about by NHS pressures. With 38% of consultations having a mental health element compared to 25% pre-Covid. But there is no evidence the patients have fared less well than if they had been referred to the Government’s Improving Access to Psychological Therapies (IAPT) business (or secondary care mental health services).   In similar vein there is no evidence that those attending the Citizens Advice Bureau with mental health problems do any less well than those attending IAPT [http://www.cbtwatch.com/no-evidence-that-the-improving-access-to-psychological-therapies-iapt-service-does-any-better-than-contact-with-the-citizens-advice-bureaux-cabx]. It appears that IAPT is no better than an attention placebo.

Ideally IAPT would have been subjected to a randomised controlled trial in which clients were alternately assigned to the services ministrations and to  a credible placebo intervention. With outcome gauged by blind assessors, using a standardised reliable diagnostic interview. But no such study has been forthcoming or seems likely to happen anytime soon. Though less than ideal comparisons can be made with the trajectory of attendees of GPs and Citizens Advice Bureaus.

The burden of proof is on IAPT to demonstrate that its’ staff have a competence beyond that of GP’s and Citizens Advice Bureau Workers, that makes a real world difference to client outcome. My own research [Scott (2018) https://doi.org/10.1177%2F1359105318755264] suggests that only the tip of the iceberg of IAPT clients recover .

GPs acknowledge the limits of their competence, IAPT staff do not, at least publicly. Unfortunately nobody holds them to account, they are a law unto themselves. We continue to throw away over a £1 billion a year on IAPT, with the National Audit Office, NHS England and Clinical Commissioning  Groups showing a radical apathy about the matter.

Dr Mike Scott