The previous Health Minister, Dr Therese Coffey was asked 7 questions about the effectiveness of routine psychological therapy for depression and the anxiety disorders. Only 1 of the 7 questions was answered and that, only partially. The Department of Health advised that NHS England were better placed to answer my questions and so I put the queries to them. I wrote about this in my blog for Mad In America. Since then NHS England have replied, or rather not replied, suggesting that I put the questions to my local Integrated Care Board (ICB’s are the recent replacement for Clinical Commissioning Groups). My questions obviously relate to national policy and not local difficulties, it is completely disingenuous of NHS England to reply in this way. I have now asked them for a considered response to my questions.
For the sake of completion I have reproduced the questions below:
1.The Government Improving Access to Psychological Therapies (IAPT) Service is experimenting with public, direct access to a Psychological Wellbeing Practitioner. But PWPs are not trained in diagnostics nor are they qualified therapists. Why then are they being given this gatekeeping role?
2. The IAPT service has cost billions of pounds, since its’ inception in 2008. Why, then has there been no independent audit of the service?
3. With regards to physical health the Government is funding Community Diagnostic Centres, with regards to mental health why is there no facility for reliable diagnosis in IAPT?
4. With regards to mental health there is no evidence that those availing themselves of IAPT fare any better than those attending the Citizens Advice Bureaux? What then is the added value of funding IAPT?
5. How is the experiment of making PWPs gatekeepers being evaluated and who decided on the criteria?
6. IAPTs claimed recovery rate of 50% has not been independently verified. The independent evidence of an Expert Witness to the Court [Scott (2021) British Journal of Clinical Psychology] suggests that in fact only the tip of the iceberg recover. Is this not grounds for a publicly funded independent audit?
7. How do we know IAPT is value for money?
It seems likely that only politician-like responses will be forthcoming. Will nobody grasp the nettle?
A 24% response rate combining the two, and manual driven psychotherapy conferred no added benefit Bartova et al (2021). In the podcast from Mad In America these findings are set against a 31% placebo response rate. Further no evidence that the interventions altered the course of a disorder, which is the prime objective of treatments for physical disorders. Rather the focus was on symptomatic relief. Articles covered in podcast include:
Pies and Dawson (2022) have today taken up the cudgel to attack the findings of Moncrieff et al (2002) that were the springboard for the podcast. But they are disingenuous in claiming that no one of academic credibility has ever suggested that low serotonin causes depression. For decades, at least in the UK this has been the dominant message given to patients, with the implication that they need antidepressants to restore the chemical imbalance. Pies and Dawson (2022) have recourse to a biopsychosocial model which posits interactions of thoughts, feelings, social factors and biology, in which will be found some biological factor that is of key importance in the development of depression and through which antidepressants will be found to work. But given the track record to date this seems unlikely and provides little basis for current pharmacological practice with the exception of the use of lithium.
In the Bartova et al (2021) study the therapists claimed that they were adhering to a manual driven psychotherapy protocol, but no fidelity checks were made. A similar scenario to the claim made by IAPT in the UK that it delivers CBT, but without any independent corroboration. It is I believe the case that CBT can make a real world difference for depression and the anxiety disorders if appropriately delivered.
from ‘Mad In America’ https://www.madinamerica.com/2022/06/uk-iapt-abject-failure/
Rescued man in boat looking on shipwreck
Dr Michael J Scott Ph.D
In 2012, an editorial in the prestigious journal Nature claimed that the UK’s IAPT Service is “world-beating”—meaning that the service is the world’s best for treating mental health concerns. Now that 10 years have passed, it’s clear that this was not true. Despite the hype, the IAPT is hardly “world-beating.” In fact, it is a doubtful model for other countries to follow.
The IAPT (“Improving Access to Psychological Therapies”) was created in 2008 and fully funded by the UK government (though restricted to provision only in England). The program claims to provide evidence-based psychological treatment, most commonly in the form of cognitive behavioural therapy. It was initially focused on patients with depression and anxiety disorder diagnoses, but its scope has been gradually expanded to include the psychological treatment of long-term physical conditions such as chronic fatigue syndrome.
Patients either self-refer (at no cost) or are referred to the IAPT by their primary care physician/general practitioner. Clients are initially assessed during a 30-minute telephone conversation by a Psychological Wellbeing Practitioner (PWP) who is not an accredited therapist and is not trained to diagnose. IAPT clients initially receive low-intensity CBT (six hours or less of therapy contact) delivered by a PWP. If they are deemed not to have responded to this, on the basis of a psychometric test, they can be “stepped up” to high-intensity CBT delivered by an accredited psychological therapist.
How Well Is the IAPT Doing?
Proponents of the IAPT cite its outcomes for the clients who complete their treatment sessions—but over half of IAPT clients don’t even attend two sessions. The response of the IAPT to this massive disengagement with the service—clients “voting with their feet,” as it were—is currently to offer training to its staff to “streamline” matters by encouraging them to use a computer algorithm to decide which clients go where. This training, which was held on June 22, 2022, involves expert advice from the CEO of Limbic, an artificial intelligence Company. Thus, not only are IAPT staff to be distracted by clients having to complete psychometric tests at each encounter, but now additionally by the administration of the computer algorithm.
Yet Limbic claims it is “an AI assistant for clinical assessments in IAPT—improving access, reducing costs and freeing up staff time.” Where is the independent evidence for this? What clients really want is to be listened to, but there is little chance of this any time soon. Conquering disengagement in this manner is a forlorn hope.
IAPT’s 2019 manual states that IAPT staff do not make a diagnosis. However, the treatment recommendations of NICE (the UK’s National Institute for Health and Clinical Excellence) are explicitly tied to a taxonomy of specific treatments for identified disorders (largely, diagnosis-specific variations on CBT). IAPT legitimises itself by a claim to be NICE compliant, but how could they be if they don’t make diagnoses? How can they follow NICE’s specific treatment guidelines without first assessing which condition they are treating?
Whilst the IAPT’s claim to be NICE-compliant might be music to the ears of NHS England and clinical commissioning groups, it is simply not true. No independent fidelity checks have ever have been made on IAPT’s ministrations. IAPT allegedly provides specific CBT protocols for different disorders, but it has provided no evidence that it does this.
In no independent audit of IAPT, have clients been asked “are you back to your usual self since this treatment?” This is a goal that is meaningful to the client—with the possible supplementary question, “for how long have you been back to your usual self?” In papers published by IAPT staff, no such real-world outcome measures have ever been used. Instead, reliance has been placed on changes in psychometric test scores. Yet without a control group, it’s impossible to know whether these changes would have occurred with passage of time anyway. After all, people present to psychological treatment at their worst, and at any other time, they are likely to score better. In statistics, this principle is known as regression to the mean, and it is vital to account for this effect.
Further, the client knows his/her therapist sees their completed questionnaire, pushing them to respond more positively. A client may also not want to endorse negative responses on the questionnaire because it would mean acknowledging that they have wasted their time in therapy. These are known issues with self-report measures that are commonly cited as important limitations in research studies.
Finally, the questionnaire results also give no indication of whether any improvement has lasted for a period that the client would regard as meaningful, such as eight weeks or (hopefully) more.
In an era of personalised medicine, it is essential that the voice of the client is heard. It is an almost ubiquitous failure of studies to employ a primary outcome measure that a client can identify with—known as patient-centred outcomes. Perhaps the nearest proxy is loss of diagnostic status (full recovery) determined by an independent assessor. The burden of proof rests on providers of services to demonstrate that their treatments are making an important difference to a client’s life in a way that is clearly recognisable to the client. IAPT has not cleared this evidential bar.
Use of an algorithm is not incompatible with a clinician operating on a knowledge base of what works for whom. But it has to be first demonstrated that the former constitutes added value over the latter. The burden of proof is on AI advocates to demonstrate its relevance in a particular context. This cannot be done by a company with a vested interest in AI.
It is perfectly possible that using two systems, an algorithm and clinical judgement, might create confusion. Should an IAPT client not fare well (for example, if a client dies by suicide) when treated under such a dual system, could IAPT convince a court, on the balance of probability (the UK benchmark in personal injury cases), that its approach was evidence-based? After all, an AI approach is not something that NICE has recommended—so it flies in the face of the guidelines. Traditionally, courts are swayed by eminence-based evidence (such as the word of authority figures like psychiatrists), but they have become more aware (at least in the UK) of the perils of unbridled clinical judgement.
Inadequacies of IAPT’s Chosen Metric
IAPT’s chosen metric is a change in score on two psychometric tests. Clients are deemed to have responded if on the PHQ-9 (a measure of the severity of depression) their test score has gone from above 10 to below 10 with treatment, with a difference of at least 6 points, and if on the GAD 7 (a measure of the severity of generalised anxiety disorder) their score has gone from above 8 to below 8 with treatment, with a difference of at least 4 points. This approach has numerous deficiencies:
It is not known whether the completer of these two measures was actually suffering from depression or generalised anxiety disorder, because IAPT clinicians do not make diagnoses. Further, it is not known whether either of these “conditions” was the disorder that was causing most impairment in functioning.
When test results are observable by the treating clinician, clients may wish to be polite and relay a lower score, particularly as they do not want to feel that they have wasted their time in therapy.
No attention control comparator: Changes in test scores are meaningless unless there is a comparator group who have received the same attention and a credible rationale. It’s unclear whether the therapy resulted in the improvement, or if the client would have improved with a “placebo” therapy (simply receiving professional “attention” without CBT techniques).
Regression to the mean: Clients typically present at their worst, so with the mere passage of time they will score more modestly.
The vagaries of a minimally important difference (MID) in a score: MIDS are established by statistical comparisons between an “ill” and “well” group, but they do not necessarily indicate clinically important difference. They also fail to address the client’s perception and there is no personalisation of treatment outcome. Clients are given no voice.
Haemorrhaging
Last year about a third of referrals (30.1%) to the IAPT service did not access it. Of those who accessed the service over a third (38%) had just one treatment session. Thus, by the start of the IAPT race over half of people (57%) have not engaged in treatment (as defined by IAPT’s metric of attending two or more sessions). If these results applied to a physiotherapy service it would raise serious doubts about the suitability of the service! Why then does the UK National Audit Office studiously avoid scrutiny of IAPT, which costs the taxpayer over £1 billion a year?
Sub-Therapeutic Dose and Treatment Outcomes
Of those who engaged in IAPT, the average number of sessions was 7.5, which is well below the 12 to 20 sessions that NICE recommends for depression and anxiety disorders (IAPT’s primary targets). Are we to believe that IAPT’s clinicians are so skilled that they can achieve recovery with only half the dosage of therapy as that delivered by therapists in the randomised controlled trials of CBT for depression and the anxiety disorders?
The benchmark set by trials of therapy as a treatment for these conditions is that at least 50% must “recover” (no longer meet criteria for a diagnosis). But there is no evidence that IAPT has achieved this. My own research suggests that only the tip of the iceberg recover.
In my capacity as an Expert Witness to the Court, I examined 90 IAPT clients who had been through the service either before or after the personal injury. Only 9.2% of subjects lost their and diagnostic status, as assessed using the “gold standard” SCID interview. It mattered not whether subjects were assessed before or after their personal injury. (These findings are in need of replication by clinicians independent of IAPT with a non-litigant population.)
Diagnostic Creep
IAPT appears not to so much follow the data, but to follow funding opportunities. It has branched out into territory were angels fear to tread: Medically Unexplained Symptoms (MUS). MUS is an umbrella term embracing conditions as diverse as chronic fatigue syndrome, fibromyalgia, and irritable bowel syndrome. IAPT’s contention is that exaggerated negative beliefs about symptoms, and maladaptive behaviours (e.g., avoidance), play a pivotal role in the maintenance of symptoms in these conditions. The therapeutic task, as they see it, is to then to modify these cognitions and behaviours.
What happens if a client protests that they have valid medical symptoms? Well, IAPT clinicians are taught not to openly disagree and not to say ‘it is all in your mind’. But to nevertheless continue to focuss on the ‘exaggerations in beliefs and avoidance behaviours’, so much for honesty.
Along with Keith Geraghty from the University of Manchester, I have published a critique of this sojourn. We identified a series of seven core problems and failings of the IAPT, including an unproven treatment rationale, a weak and contested evidence-base, biases in treatment promotion, exaggeration of recovery claims, under-reporting of drop-out rates, and a significant risk of misdiagnosis and inappropriate treatment. We concluded that psychotherapy should not become the default option when patients have “medically unexplained symptoms.”
The term “medically unexplained symptoms” should be written in lowercase to avoid conveying the impression that a meaningful homogenous entity is being described. The term should not enter the taxonomy of disorders. Importantly, even the DSM-5 has shunned recourse to the term. But unfortunately, IAPT only pays lip service to the standard diagnostic criteria.
Big Pharma and IAPT
Whilst some large pharmaceutical companies have greatly served the public good with regard to COVID, their performance with regard to psychotropic drugs is much less impressive. Researchers like Read and Moncrieff pin their hopes on the efficacy of CBT demonstrated in randomised controlled trials, but make no mention that CBT as delivered in routine practice bears little relationship to the protocols utilised in RCTs. Just as the claims of pharmaceutical companies require critical appraisal so too does the UK IAPT service.
In an article in the British Journal of Clinical Psychology, I explained what the IAPT needs to do to get back on track. I noted that in their published papers, IAPT staff do not declare their allegiances and indeed state that they have no conflict of interest! This dishonesty needs to change. I suggested that there is a need for IAPT to prove their case using the standards of proof that are required in other areas of medicine. To this effect, they cannot simply claim that they deliver evidence-based treatment—there has to be documented evidence for it that would be persuasive to an independent observer. The IAPT service has to demonstrate that it provides an added value over someone attending an advice centre or counselling service. Yet the IAPT has yet to demonstrate that it makes an important enough difference to clients’ lives that they would recognise it.
This catalogue of omissions may create a sense of déjà vu for those who have followed the machinations of Big Pharma with regards to mental health. Unfortunately, there has only been a deafening silence from the IAPT regarding my critique.
