Number Theatre and Routine Mental Health

the National Institute for Health Research has just published a review of studies of the psychological treatment of Medically Unexplained Symptoms (MUS) https://doi.org/10.3310/hta24460 [Leaviss J, Davis S, Ren S, Hamilton J, Scope A, Booth A, et al. Behavioural modification interventions for medically unexplained symptoms in primary care: systematic reviews and economic evaluation. Health Technol Assess 2020;24(46)] but in all studies the primary outcome measure was an improvement of symptoms on some psychometric test. No categorical measure was used such as no longer suffering from a ‘disorder’ such as fibromyalgia, irritable bowel syndrome or chronic fatigue syndrome post treatment. Likewise the Improving Access to Psychological Treatment (IAPT) markets its success on a change in score on psychometric tests the PHQ-9 and GAD-7. Further whether or not an IAPT clinician is to be subjected to a formal review of competence is based on a change of score on these measures. No categorical measure is used such as the proportion of cases of depression, panic disorder, generalised anxiety disorder etc that have lost their diagnostic status. Sir David Spiegelhalter the Statistician has coined the term ‘number theatre’ to describe the way in which the UK Government has promulgated statistics in relation to the Pandemic, but this drama been playing for years in the mental health arena.  I am reminded of a line from a song somewhere, ‘I am more than a number in a little red book’, although intended for a very different context, it seems particularly apt for IAPT.

Lies
Damned Lies
and
IAPT
 

Number theatre in the mental health field has it seems been driven by the desire of psychologists to colonise. It is a reaction against the categorical labels employed by psychiatry. But the truth of the matter is both are needed simultaneously. To take a medical example, if I have a heart problem I need to know what the problem is but also my blood pressure today.

IAPT will topple because it pivots on psychometric tests. Inspection of of its’ main pillar, the PHQ-9 exposes a crumbling structure:

  1. Client’s judgement of their functioning does not match changes on the PHQ-9 https://doi.org/10.3310/hta24460. Thus an IAPT therapist might report to his supervisor the ‘improvement’ on his/her clients score on the PHQ-9 and at the same time report that the latter said they are ‘the same old’. The overall judgement of the client is likely to be dismissed in favour of the alleged ‘moving towards recovery’ or ‘recovery’ on the PHQ-9.
  2. In the initial validation study of the PHQ-9  by Kroenke and Spitzer it was not validated against a ‘gold standard’ that it was sufficiently different to to make it an acceptable diagnostic aid according to the AMSTAR
  3. The findings of the progenitors of the PHQ-9 Kroenke and Spitzer were not replicated by independent researchers using a ‘gold standard’ diagnostic interview  such as the SCID.
  4. The diagnostic accuracy of an instrument depends very much on the prevalence of the disorder in which it was first evaluated. In the case of the PHQ-9 psychiatric outpatients in the United States. There is no reliable evidence (as assessed by a standardised diagnostic interview)  on the prevalence of disorders amongst those attending IAPT (which include both self referrers and GP referrals).  Thus the clinical utility of the PHQ-9 in this context is unknown.
  5. The PHQ-9 is purportedly a measure of the severity of depression, but there is poor concordance between it and alternative measures of the severity such as the HAD i.e a person would be in a different category of severity depending on which measure is used.

5. The use of a psychometric test with a summary score assumes that each of the items (9 in the case of the PHQ-9) contribute equally to the total score. But this is implausible an item about suicidal ideation (item  9 on the PHQ-9) is likely to  be more significant than an item about fatigue. 

6. Two patients on the PHQ-9 could have the same score, but arising from one patient endorsing all intermediate scores whilst the second endorses several items at the highest score. The same score but arguably a quite different meaning.

7. The PHQ-9 assumes that is the frequency of a symptom  that is the determinant of severity rather than the intensity. 

8. Unless the mechanism by which a PHQ9 score is changed is known it cannot determined that an evidence based treatment was in fact used. Thus those getting a supposed ‘result’ may be more at fault than those acknowledging none response, the latter may simply be more honest. 

These considerations on the PHQ-9 may not be prohibitive of its use, if employed in the context of a standardised diagnostic interview that has established the person has depression. But such an interview would likely also yield the presence of one or more coexisting disorders. The trajectory of these additional disorders would have to be tracked by other psychometric tests that are pertinent to the disorder. The idea that the  PHQ-9 can stand alone as judge and jury on a client’s mental health is absurd.

However politicians, public health bodies and clinical commissioning groups like to be told that there is a simple solution to a problem and that they can make a difference by implementing the chosen solution. Enter stage right IAPT proclaiming ‘give the PHQ9 reduce it below 10, job done and woe betide any clinician who does not manage this routinely’. As an encore IAPT uses numbers e.g throughput of clients, waiting lists to placate politicians and funders.  Exhaustion, numbing and detachment [burnout] are an inevitable consequence of these working conditions. No amount of self-reflection as advocated by Psychological Wellbeing Practitioner in the current issue of CBT Today, is going to make a real world difference. It is a shame that CBT Today has become IAPT’s comic.

Dr Mike Scott

 

 

 

 

The Cost of IAPT Is At Least Five Times Greater Than Claimed

The British Medical Journal has just published the following letter of mine online with the above title:

‘Six years ago a News headline in the BMJ proclaimed ‘Increasing access to psychological therapies will cost NHS nothing’ BMJ 2012; 344 doi: https://doi.org/10.1136/bmj.e4250, citing a report of Lord Layard  of the Mental Health Policy Group of the Centre for Economic Performance http://cep.lse.ac.uk/_new/research/mentalhealth/default.asp, that claimed ‘after an average of 10 sessions half the people with anxiety conditions will recover, most of them permanently, and half the patients with depression will recover’ .  Far from being substantiated an independent assessment by Scott (2018), http://journals.sagepub.com/doi/pdf/10.1177/1359105318755264, using a standardised diagnostic interview, suggest a 10% recovery rate. This represents a five-fold increase of the cost of treatment per cured person.

The progenitors of IAPT, Clark and Layard in their book Thrive (2015) claim that the cost of treatment in IAPT is £650 per person, for people having attended 2 or more treatment sessions.  This leaves out of account the 40% of its clients who attend only one treatment session [IAPT (2018)] and the costs of the initial assessments which totalled £92 million in 2016-2017, with total costs of £367,219,192 in that period.  This means that the true cost of IAPT is at least 5 times greater than alleged, all without any government funded independent audit. Further average session attendance for those ‘treated’ in IAPT is 6.6 [IAPT (2018)] not the average of 10 sessions that Lord Layard deemed necessary, so that the average patient in fact receives a sub-therapeutic  dose of treatment.

In 2012 Lord Layard claimed ‘the average improvement in physical symptoms is so great that the resulting savings on NHS physical care outweigh the cost of the psychological therapy’. This claim remains unproven and what limited evidence is available points in the opposite direction. How do Clinical Commissioning Groups justify paying such inflated sums? how can they be sure another agency could not achieve the same for less? how do they know that GPs simply tracking clients with depression and anxiety disorders would not achieve the same outcomes? NHS England should surely advise CCG’s to ask searching questions and organise a long overdue government funded independent audit of IAPT focusing on real world outcomes, such as loss of diagnostic status..

BMJ (2012) ;344:e4250 Increasing access to psychological therapies will cost NHS nothing, says report

Clark, D.M and Layard, R (2015) Thrive: The Power of Evidence-Based Psychological Therapies London: Penguin.

IAPT (2018) Psychological Therapies: Annual report on the use of IAPT services England, 2016-17 Data Tables. NHS Digital: Community and Mental Health Team.

Mental  Health Policy Group of the Centre for Economic Performance (2012) How mental health loses out in the NHS.   http://cep.lse.ac.uk/_new/research/mentalhealth/default.asp.

Scott, M.J (2018) IAPT: The Need for Radical Reform. The Journal of Health Psychology, 23, 1136-1147.

 

Dr Mike Scott

Shifting The Focus To The Client Being Well

Conferences, government agencies such as IAPT and recent research papers, gloss over the proportion of clients who are well at the end of CBT (remission), preferring instead to talk of the number of people who have responded (response) to an intervention (proportion of people whose score reduced by greater than x%). Springer et al (2018) call for a shift in focus to the real world outcome of remission ‘remission should be the ultimate goal of treatment’. Chasteningly they point out  that the remission rate for CBT across the anxiety  disorders is just 50%.

Interestingly the results of Springer et al’s meta-analysis showed that the remission results were poorer when there was no blind evaluator. This may be important in evaluating IAPT’s performance because they have had no independent evaluator! Further the results in the Springer analysis were poorer still when there was comorbid conditions such as depression and/or subtance use disorder, suggesting that all a client’s disorders need tackling not just the primary anxiety disorder. GAD and PTSD did better than OCD and SAD with panic disorder in between.

Clients want to be well again not just reduce their score on a psychometric test that some clinician deems acceptable for their own reasons. Losing their diagnostic status should be a necessary condition  for assessing outcome, albeit that arguably it also ought to be complimented with reduction below a certain cut-off on a psychometric test.

 

Springer, K.S., Levy, H.C and Tolin, D.F (2018) Remission in CBT for adult anxiety disorders. A meta-analysis. Clinical Psychology Review, published online ahead of print

Please cite this article as: Springer, K.S., Clinical Psychology Review (2018), https://doi.org/10.1016/j.cpr.2018.03.002

Dr Mike Scott