IAPT’s Below Intensity CBT – Rhetoric vs Reality

 

The Improving Access to Psychological Therapies (IAPT) Service clients receive on average 6 sessions (amongst those who complete 2 or more sessions), compared to the typical 13  sessions in the 29 evidence based, ‘gold standard’, randomised controlled trials, considered by OST (2008) doi:10.1016/j.brat.2007.12.005. The discrepancy in the dosage of treatment creates a suspicion that, in routine practice, clients receive a sub-therapeutic dose of treatment.

In practice the Service haemorrhages clients, with 60% of all referrals not completing treatment in 2017-2018,  Moller et al (2019) https://doi.org/10.1186/s12888-019-2235-z.  In this period a third (35%) of clients were given an unspecified diagnosis making the delivery of a NICE approved evidence based treatment impossible. The IAPT services claim to a 50% recovery rate, comparable to that in randomised controlled trials, is preposterous in this context.

Service providers, such as IAPT, are necessarily opportunistic and overstate the power of a simple and cheap mode of service delivery, thereby doing wonders for their power and bank balance. But clients are the losers, particularly those from disadvantaged backgrounds, Moller et al (2019) https://doi.org/10.1186/s12888-019-2235-z.

The rhetoric was that the Service would pay for itself because of the positive effects on employment but Moller et al (2019) have found no evidence of this https://doi.org/10.1186/s12888-019-2235-z.Employed: Start of treatment 316,604; end of treatment, 302,746; Unemployed and seeking work: Start of Treatment, 54,580; End of Treatment 49,803; Long term sick or disabled or in receipt of benefits; Start of Treatment, 43,275; End of Treatment, 43,671. Using IAPT’s own data for 2017-2018,  there is a 17% difference in recovery rate depending on whether the client was from the most deprived area , with a 41.0% recovery rate compared to a 58.1% recovery rate for the least deprived area.

 

Dr Mike Scott

 

 

The Mistreatment of IAPT Clients – The Smoking Gun

What right has the Improving Access to Psychological Therapies (IAPT) Service to routinely label each client with a diagnostic code (ICD of the International Classification of Disorders, World Health Organisation) when the Organisation’s Manual states that it does not do diagnosis. Fearful of litigation, it states that its diagnoses should not be used for medico-legal purposes, but as the code is the determinant of treatment, IAPT should be in the dock!

The Improving Access to Psychological Therapies (IAPT) service screens clients for treatment using the PHQ-9 but a study published in the British Medical Journal by Brooke Levis et al last year http://dx.doi.org/10.1136/bmj.l1476  indicates that half the  deemed depression cases  have been incorrectly diagnosed. 

In high quality randomised controlled trials of the treatment for depression all clients admitted have been diagnosed as having depression according to a ‘gold standard’ diagnostic interview such as the SCID. The recovery rate in the rcts is 50%. But IAPT claims that it approaches the recovery rate of rcts. This is preposterous! Consider 100 IAPT cases which score above the PHQ-9 cut off of greater or equal to 10. One half of them i.e 50 will not actually have depression and therefore cannot recover from the disorder. Of the other half, 50 cases, if the IAPT clinicians were as good as in the rcts 25 would recover. Thus the maximum possible recovery rate for depression in IAPT is 25% and this is assuming its clinicians are as good as the highly trained clinicians in rcts. More plausibly the recovery rate for depression in IAPT is the 14.9% I found in my independent study of IAPT, http://DOI: 10.1177/1359105318755264 https://doi.org/10.1177/135910531875 using the SCID.

In primary care 22% of patients score over 10 on the PHQ9, so what are the treatment implications for the likely 3 out of 4 IAPT clients who score below 10? For these the PHQ-9 offers no direction.

But IAPT has its’ own answer, IAPT Manual, p 24  (2019), a) come up with a problem descriptor then choose an ICD 10 that that ‘matches’ the descriptor and  then b) a NICE treatment that matches the ICD 10 code.  Consider an IAPT client who reports that they are feeling emotionally numb at work, detached from others and fatigued after little exercise. The therapist could plump for either depression, burnout, chronic fatigue syndrome or the effects of COVID-19, with no guidance as the appropriate label! 

Using IAPT system Delgadillo et al (2020) http://dx.doi.org/10.1037/ccp0000507 classified over 40% of clients as having ‘Affective Disorder’  and over 20%  as having a ‘mixed disorder’. But there are no randomised controlled trials for ‘affective disorder’ or ‘mixed’, so that for 60% of IAPT’s clients there cannot be an appeal to an evidence based treatment (i.e one based on a randomised controlled trial). Considering again a sample of 100 IAPT clients who score less than 10 on the PHQ9 60 of them will have been labelled with a disorder for which there can be no evidence based treatment, this leaves 40 clients who in principle could be treated  with an evidence based treatment. Again assuming that for this population of 40 that allegedly covered GAD (10-12%), panic disorder (4-6%), social anxiety disorder (4-6%), specific phobia (0.5-1.0%), OCD (4-5%), PTSD (6-8%) and other (2-3%) there was an overall recovery rate of 50% only 20% of the allegedly ‘non-depressed’ clients would recover. This 20% would have to regarded as an upper limit because it assumes the IAPT therapist would be as skilled as the highly trained therapists involved in the rcts for anxiety disorders. A more realistic estimate of recovery for the IAPT ‘anxious clients’ would be the 14.2% found in my study of IAPT clients http://DOI: 10.1177/1359105318755264 https://doi.org/10.1177/135910531875

The other metric employed  by IAPT is the GAD-7, a measure of the severity of depression, but as according to IAPT it has only been relevant to one in 10 of its service users, any effect of the treatment of this disorder will only effect the above picture minimally. Assuming a 50% recovery the effect will be even less and less still when one compares the training of therapists in GAD acts with the training of the routine IAPT therapist.

IAPTS sole reliance on psychometric tests and fudge has backfired badly, but it is the client who suffers most, with therapists suffering from the recoil.  

Dr Mike Scott

 

 

 

Number Theatre and Routine Mental Health

the National Institute for Health Research has just published a review of studies of the psychological treatment of Medically Unexplained Symptoms (MUS) https://doi.org/10.3310/hta24460 [Leaviss J, Davis S, Ren S, Hamilton J, Scope A, Booth A, et al. Behavioural modification interventions for medically unexplained symptoms in primary care: systematic reviews and economic evaluation. Health Technol Assess 2020;24(46)] but in all studies the primary outcome measure was an improvement of symptoms on some psychometric test. No categorical measure was used such as no longer suffering from a ‘disorder’ such as fibromyalgia, irritable bowel syndrome or chronic fatigue syndrome post treatment. Likewise the Improving Access to Psychological Treatment (IAPT) markets its success on a change in score on psychometric tests the PHQ-9 and GAD-7. Further whether or not an IAPT clinician is to be subjected to a formal review of competence is based on a change of score on these measures. No categorical measure is used such as the proportion of cases of depression, panic disorder, generalised anxiety disorder etc that have lost their diagnostic status. Sir David Spiegelhalter the Statistician has coined the term ‘number theatre’ to describe the way in which the UK Government has promulgated statistics in relation to the Pandemic, but this drama been playing for years in the mental health arena.  I am reminded of a line from a song somewhere, ‘I am more than a number in a little red book’, although intended for a very different context, it seems particularly apt for IAPT.

Lies
Damned Lies
and
IAPT
 

Number theatre in the mental health field has it seems been driven by the desire of psychologists to colonise. It is a reaction against the categorical labels employed by psychiatry. But the truth of the matter is both are needed simultaneously. To take a medical example, if I have a heart problem I need to know what the problem is but also my blood pressure today.

IAPT will topple because it pivots on psychometric tests. Inspection of of its’ main pillar, the PHQ-9 exposes a crumbling structure:

  1. Client’s judgement of their functioning does not match changes on the PHQ-9 https://doi.org/10.3310/hta24460. Thus an IAPT therapist might report to his supervisor the ‘improvement’ on his/her clients score on the PHQ-9 and at the same time report that the latter said they are ‘the same old’. The overall judgement of the client is likely to be dismissed in favour of the alleged ‘moving towards recovery’ or ‘recovery’ on the PHQ-9.
  2. In the initial validation study of the PHQ-9  by Kroenke and Spitzer it was not validated against a ‘gold standard’ that it was sufficiently different to to make it an acceptable diagnostic aid according to the AMSTAR
  3. The findings of the progenitors of the PHQ-9 Kroenke and Spitzer were not replicated by independent researchers using a ‘gold standard’ diagnostic interview  such as the SCID.
  4. The diagnostic accuracy of an instrument depends very much on the prevalence of the disorder in which it was first evaluated. In the case of the PHQ-9 psychiatric outpatients in the United States. There is no reliable evidence (as assessed by a standardised diagnostic interview)  on the prevalence of disorders amongst those attending IAPT (which include both self referrers and GP referrals).  Thus the clinical utility of the PHQ-9 in this context is unknown.
  5. The PHQ-9 is purportedly a measure of the severity of depression, but there is poor concordance between it and alternative measures of the severity such as the HAD i.e a person would be in a different category of severity depending on which measure is used.

5. The use of a psychometric test with a summary score assumes that each of the items (9 in the case of the PHQ-9) contribute equally to the total score. But this is implausible an item about suicidal ideation (item  9 on the PHQ-9) is likely to  be more significant than an item about fatigue. 

6. Two patients on the PHQ-9 could have the same score, but arising from one patient endorsing all intermediate scores whilst the second endorses several items at the highest score. The same score but arguably a quite different meaning.

7. The PHQ-9 assumes that is the frequency of a symptom  that is the determinant of severity rather than the intensity. 

8. Unless the mechanism by which a PHQ9 score is changed is known it cannot determined that an evidence based treatment was in fact used. Thus those getting a supposed ‘result’ may be more at fault than those acknowledging none response, the latter may simply be more honest. 

These considerations on the PHQ-9 may not be prohibitive of its use, if employed in the context of a standardised diagnostic interview that has established the person has depression. But such an interview would likely also yield the presence of one or more coexisting disorders. The trajectory of these additional disorders would have to be tracked by other psychometric tests that are pertinent to the disorder. The idea that the  PHQ-9 can stand alone as judge and jury on a client’s mental health is absurd.

However politicians, public health bodies and clinical commissioning groups like to be told that there is a simple solution to a problem and that they can make a difference by implementing the chosen solution. Enter stage right IAPT proclaiming ‘give the PHQ9 reduce it below 10, job done and woe betide any clinician who does not manage this routinely’. As an encore IAPT uses numbers e.g throughput of clients, waiting lists to placate politicians and funders.  Exhaustion, numbing and detachment [burnout] are an inevitable consequence of these working conditions. No amount of self-reflection as advocated by Psychological Wellbeing Practitioner in the current issue of CBT Today, is going to make a real world difference. It is a shame that CBT Today has become IAPT’s comic.

Dr Mike Scott

 

 

 

 

The Mismatch Between Clients Global Judgements and Changes on Psychometric Test

casts doubt on the wisdom of the Improving Access to Psychological Therapies IAPT) services sole reliance on test results.  A recent study by Hobbs et al (2020) https://doi.org/10.1017/S0033291719003878 compared subjects global judgements on improvement/deterioration with changes on the PHQ9 and found poor agreement. For those who reported ‘feeling worse’ PHQ9 scores showed no change or improvement for 76% of them.  Further for those who reported ‘feeling a lot worse’  for 81% of them the PHQ9 showed no change or an improvement.  

Hobbs et al (2020) conclude that test results tell only part of the client’s story and there is a need for clinical assessment.

The interview assessments in IAPT go no further than the open ended interviews that are the first part of all standardised diagnostic interviews. Open-ended interviews furnish at most differential diagnoses and highlight candidate disorders for further investigation.  Further inquiry is then made of all of the symptoms in a diagnostic set ( controlling for information variance) and thresholds are used to determine whether a symptom is present  at a level that would constitute functional impairment (controlling for criterion variance).  Reliable diagnosis makes it possible to determine which evidence based treatments are likely to be appropriate in a particular case, bearing in mind the client’s social context and cognitive capacities.

Ultimately global judgements have a more real world feel than changes on psychometric test. A client knows whether they are back to their old selves post treatment and whether they would regard this as enduring. Similarly a client with a lifelong history will now whether they are back to their ‘best functioning’ post treatment and whether it is just one more ‘flash in the pan’. Similarly an independent assessor of a Service can make g global judgement (using a standardised diagnostic interview) whether the person has lost their diagnostic status post treatment and whether there is evidence of permanence. Psychometric tests have been grossly overvalued by IAPT for quality control purposes, but they are fantastic for marketing to the unwary.

Tests Misleading for Diagnostic Acccuracy

It is common for advocates of psychometric tests to quote high sensitivities/ specificities of the order off 80% for instruments such as the PHQ9. But this does not mean that using the PHQ9 on all clients coming through the IAPT door that using its 10 or greater cut off 4 out of 5 clients will be correctly classed.  Tests are validated in a particular context, thus if the proportion of clients with diagnosed disorder (using a diagnostic interview)  differs  by context then so to will the appropriate cut offs and sensitivities/specificities.  Further a psychometric test does not indicate what other disorders are present nor which is the primary disorder. Treatment that fails to address comorbidity is likely to fall short and comorbidity is the norm not the exception. 

Dr Mike Scott

PROMS – Track and Trace for Mental Health Without Knowing What Is Being Tracked

a just published study in the British Medical Journal  https://doi.org/10.1136/bmj.m3313 has found that  ‘There is insufficient evidence and mostly of low quality, that routine monitoring  with PROMS (Patient reported outcome measures) … leads to improvement in outcomes’.  Of the 5 studies reviewed one was of the Improving Access to Psychological Therapies (IAPT) Service in which the PHQ-9 and GAD-7 self report measures were used. 

Strangely the authors of the study Kendrick and Maund (2020) are surprised by the negative findings. It seems not to have occurred to them, that if it is not known with any certainty what the patients were suffering from in the first place then using the most available psychometric test to measure outcome is unlikely to yield any positive findings. In none of the studies was a standardised diagnostic interview used to establish diagnosis and determine any accompanying diagnostic comorbidity.  Thus it cannot be reliably known which is the outcome measure of primary interest, and should becomes the established yardstick before treatment begins and what secondary analyses should be declared in advance. This is akin to the need to pre-register how the results of a randomised controlled trial are going to be analysed rather than going on a post hoc fishing expedition highlighting some positive finding or other to justify a service.

Last Night of The PROMS?

The use of PROMS appears to be fuelled by the need to quickly process patients, using surrogate outcome measures. Rather than taking the time to properly listen to them and use a real world outcome measure such as loss of diagnostic status for say 8 weeks, as assessed by an independent evaluator using a standardised diagnostic interview. Psychometric tests completed for the benefit of a treating clinician are subject to demand characteristics, including wanting to please the therapist and not wanting to feel time has been wasted in engaging in psychological therapy. These concerns are amplified when tests are administered (as in IAPT) on a weekly basis and clients can easily remember their last score.

For all the deficiencies of track and trace over COVID-19,  the target is at least not a ‘fuzzy’ , rendering the process meaningless. Ironically since the demise of Public Health England Baroness Dido Harding is in charge the Covid-19-19 Track and trace. I e-mailed her asking if she was also going to assume responsibility for IAPT but have had no reply. Any QUANGO such as IAPT is likely to rejoice at the absence of accountability but to the detriment of the public. There has to be clarity about exactly who IAPT is accountable to now.

Monitoring Is Necessary But  Never Sufficient 

Just as monitoring the spread of the coronavirus is critical to triggering some preventative measures, it is likely going to be insufficient until there is an evidence based treatment protocol including a vaccine and treatment of the effected. So to only an informed monitoring of mental health problems can highlight appropriate treatment interventions. Monitoring by itself is descriptive rather than prescriptive. Unfortunately there is nothing in the Kendrick and Maund (2020) approach that is likely to make it reliably prescriptive, making their proposed developments in monitoring rather pointless.

Dr Mike Scott

 

Is Evidence Based Treatment Possible Without Evidence Based Assessment?

‘no’, this is the take home message from a just published study by Moses et al in the Journal of Anxiety Disorders https://www.sciencedirect.com/science/article/pii/S0887618520300931. An evidence based assessment includes a diagnostic interview, as well as a clinical interview and psychometric tests. Moses et al (2020) summarise the literature that the inclusion of a diagnostic interview improves outcome, by minimising missed diagnosis and misdiagnosis. These authors bemoan their finding that only a small minority of Australian psychologists use a diagnostic interview, but the position is even worse in the UK, as the largest provider of services the Improving Access to Psychological Therapies (IAPT) explicitly excludes the making of diagnosis/diagnostic interviews.   IAPT cannot improve access to evidence based psychological therapies because it does not operate the admission gate of an evidence based assessment.

The absence of an EBA leads to a revolving door, demoralising clients in search of a credible explanation of their difficulties. An EBA is a necessary part of evidence based practice (EBP) in that it highlights candidate evidence supported treatments (ESTs). But clinical judgement is still required to ascertain whether there is a sufficient match between client and the subjects in the EST. Most ESTs have admitted clients to the study with a limited range of comorbid disorders and have not been cognitively impaired, or suffering debilitating pain. Further the clients in the EST have been in a safe environment. 

 

Dr Mike Scott

IAPT and BABCP Duck Key Questions

‘what proportion of IAPT clients have maintained recovery from the primary disorder for which they first presented?’ . The Improving Access To Psychological Treatments (IAPT) Service prides itself on its’  large comprehensive database, as if this was somehow a guarantor of the effectiveness of the service.  But it is not possible to interrogate this database to determine the  extent of restoring clients to their normal functioning, as they don’t do diagnosis.

Not only don’t they do diagnosis, they refuse to share a platform with anyone known to be critical of them.  To date IAPT has not published written rebuttals of its’ critics charges. IAPT uses the muscle of the British Association of Cognitive and Behavioural Psychotherapies (BABCP) when challenged. Later this month the BABCP has its Annual Conference. I have had no indication from the President Elect as to how they are going to address my concerns over conflicts of interest and editorial freedom, but I do know that pride of place is to be given to IAPT’s leading light. BABCP is IAPT’s apologist. It might better spend its’ time investigating why the IAPT documentation indicates that its therapists, who are invariably BABCP members, make it up as they go along, sprinkling their notes with CBT terms, without any evidence of fidelity to an evidence based protocol for anything.

Dr Mike Scott

 

 

National Institute for Health Protection to Control IAPT?

in a blog written just before the demise of Public Health England I noted  the’Breathtaking Naivety of Public Health England On Mental Health’, https://wp.me/p8O4Fm-2HI. My hope is that its’ replacement the National Institute of Health Protection (NIHP)  will question why £4billion of the taxpayers money has been spent on the Improving Access to Psychological Therapies (IAPT) Programme, without any publicly funded independent evaluation of the service. My own independent finding was that only 10% of  those going through the IAPT service recover and that the public are very dissatisfied https://connection.sagepub.com/blog/psychology/2018/02/07/on-sage-insight-improving-access-to-psychological-therapies-iapt-the-need-for-radical-reform/ ,. By contrast IAPT claims a 50%  recovery rate, but my just published paper in the British Journal of Clinical Psychology, https://onlinelibrary.wiley.com/doi/10.1111/bjc.12264#.XzwEMhZvXuk.email  casts serious doubts on the Services claim.

I have written to Baroness Harding of Winscombe, Dido Harding, the head of NIHP  to clarify whether the NIHP is indeed going to be the monitor of IAPT’s performance and if not who is? I have also stressed that no agency, including IAPT, should be allowed to mark its’ own homework.   It is imperative that a the metric for gauging the effectiveness of a service is one that the general public would recognise as meaningful, such as being independently assessed as no longer suffering from the disorder that they first presented with, as opposed to a surrogate measure, such as a change of score on a psychometric test completed in the presence of the therapist.

As MPs resume sitting in Parliament it is critical to ask who will now be in charge of ensuring IAPT does what it says on the tin and how will this QUANGO be made accountable?

Dr Mike Scott

The Improving Access To Psychological Therapies (IAPT) Programme Is Spreading Into Prisons – Why?

 

The July/August 2020 Issue of the Psychologist has a one page advert from the Forward Trust recruiting IAPT workers and others for positions in  five prisons. The Service Development Manager of the Trust speaking to the Psychologist said  ‘Many of our clients present with complex issues that would preclude then from community IAPT’. Which raises the interesting question of what body of evidence are they to call upon in dealing with these complex cases. Doubtless the Trust has done valuable work in for example giving out self-help leaflets to help prisoners manage the Covid crisis and facilitating connections with family. But such work was done hitherto by probation officers. The Trust was founded 30 years ago to support people in prison with drug problems. Where is the added value of an IAPT input?

IAPT has already published data showing clients with personality disorders do less well in community IAPT.  But IAPT clinicians have no way of reliably identifying clients with personality disorders. Further there is no evidence that they can faithfully administer a treatment protocol for personality disorder.  It seems that this is yet another example of IAPT’s expansionism,  matching its’ foray into treating long term  physical conditions that are medically unexplained. The lack of demonstrated evidence seems not to bother the service, it will likely proceed by running workshops of alleged best practice. It takes it for granted that its’ expansion is an obvious good. This is actually incredibly arrogant, demeaning of pre-existing services.

 

British Journal of Clinical Psychology Commentary and Rebuttal Of IAPT Paper

the Journal yesterday published my critique, ‘Ensuring IAPT Does What It Says On The Tin’ https://onlinelibrary.wiley.com/doi/10.1111/bjc.12264#.XzwEMhZvXuk.email of the recent IAPT ( Improving Access to Psychological Therapies) paper, by Wakefield et al (2020).

£4bn has been spent on IAPT without publicly funded independent audit. This is a scandal when the best-evidence is that only 10% of those using the service recover. There is no evidence that the Service makes a real world difference to clients’ lives, returning them to their old selves/no longer suffering from the disorder that they first presented with for a significant period. The claimed 50% recovery rate by the service is absurd.  Not only has the now defunct Public Health England  mishandled the pandemic, but it has had a matching performance on mental health. It is too early to judge whether the newly formed Health Protection Board will grasp the nettle of mental health. But I doubt that it will until there is open professional discussion that the present IAPT service is not fit for purpose. It will likely need the involvement of politicians to ensure radical reform of IAPT and that mental health is not again kicked into the long grass.

Dr Mike Scott