PROMS – Track and Trace for Mental Health Without Knowing What Is Being Tracked

a just published study in the British Medical Journal  https://doi.org/10.1136/bmj.m3313 has found that  ‘There is insufficient evidence and mostly of low quality, that routine monitoring  with PROMS (Patient reported outcome measures) … leads to improvement in outcomes’.  Of the 5 studies reviewed one was of the Improving Access to Psychological Therapies (IAPT) Service in which the PHQ-9 and GAD-7 self report measures were used. 

Strangely the authors of the study Kendrick and Maund (2020) are surprised by the negative findings. It seems not to have occurred to them, that if it is not known with any certainty what the patients were suffering from in the first place then using the most available psychometric test to measure outcome is unlikely to yield any positive findings. In none of the studies was a standardised diagnostic interview used to establish diagnosis and determine any accompanying diagnostic comorbidity.  Thus it cannot be reliably known which is the outcome measure of primary interest, and should becomes the established yardstick before treatment begins and what secondary analyses should be declared in advance. This is akin to the need to pre-register how the results of a randomised controlled trial are going to be analysed rather than going on a post hoc fishing expedition highlighting some positive finding or other to justify a service.

Last Night of The PROMS?

The use of PROMS appears to be fuelled by the need to quickly process patients, using surrogate outcome measures. Rather than taking the time to properly listen to them and use a real world outcome measure such as loss of diagnostic status for say 8 weeks, as assessed by an independent evaluator using a standardised diagnostic interview. Psychometric tests completed for the benefit of a treating clinician are subject to demand characteristics, including wanting to please the therapist and not wanting to feel time has been wasted in engaging in psychological therapy. These concerns are amplified when tests are administered (as in IAPT) on a weekly basis and clients can easily remember their last score.

For all the deficiencies of track and trace over COVID-19,  the target is at least not a ‘fuzzy’ , rendering the process meaningless. Ironically since the demise of Public Health England Baroness Dido Harding is in charge the Covid-19-19 Track and trace. I e-mailed her asking if she was also going to assume responsibility for IAPT but have had no reply. Any QUANGO such as IAPT is likely to rejoice at the absence of accountability but to the detriment of the public. There has to be clarity about exactly who IAPT is accountable to now.

Monitoring Is Necessary But  Never Sufficient 

Just as monitoring the spread of the coronavirus is critical to triggering some preventative measures, it is likely going to be insufficient until there is an evidence based treatment protocol including a vaccine and treatment of the effected. So to only an informed monitoring of mental health problems can highlight appropriate treatment interventions. Monitoring by itself is descriptive rather than prescriptive. Unfortunately there is nothing in the Kendrick and Maund (2020) approach that is likely to make it reliably prescriptive, making their proposed developments in monitoring rather pointless.

Dr Mike Scott

 

The Cost of IAPT Is At Least Five Times Greater Than Claimed

The British Medical Journal has just published the following letter of mine online with the above title:

‘Six years ago a News headline in the BMJ proclaimed ‘Increasing access to psychological therapies will cost NHS nothing’ BMJ 2012; 344 doi: https://doi.org/10.1136/bmj.e4250, citing a report of Lord Layard  of the Mental Health Policy Group of the Centre for Economic Performance http://cep.lse.ac.uk/_new/research/mentalhealth/default.asp, that claimed ‘after an average of 10 sessions half the people with anxiety conditions will recover, most of them permanently, and half the patients with depression will recover’ .  Far from being substantiated an independent assessment by Scott (2018), http://journals.sagepub.com/doi/pdf/10.1177/1359105318755264, using a standardised diagnostic interview, suggest a 10% recovery rate. This represents a five-fold increase of the cost of treatment per cured person.

The progenitors of IAPT, Clark and Layard in their book Thrive (2015) claim that the cost of treatment in IAPT is £650 per person, for people having attended 2 or more treatment sessions.  This leaves out of account the 40% of its clients who attend only one treatment session [IAPT (2018)] and the costs of the initial assessments which totalled £92 million in 2016-2017, with total costs of £367,219,192 in that period.  This means that the true cost of IAPT is at least 5 times greater than alleged, all without any government funded independent audit. Further average session attendance for those ‘treated’ in IAPT is 6.6 [IAPT (2018)] not the average of 10 sessions that Lord Layard deemed necessary, so that the average patient in fact receives a sub-therapeutic  dose of treatment.

In 2012 Lord Layard claimed ‘the average improvement in physical symptoms is so great that the resulting savings on NHS physical care outweigh the cost of the psychological therapy’. This claim remains unproven and what limited evidence is available points in the opposite direction. How do Clinical Commissioning Groups justify paying such inflated sums? how can they be sure another agency could not achieve the same for less? how do they know that GPs simply tracking clients with depression and anxiety disorders would not achieve the same outcomes? NHS England should surely advise CCG’s to ask searching questions and organise a long overdue government funded independent audit of IAPT focusing on real world outcomes, such as loss of diagnostic status..

BMJ (2012) ;344:e4250 Increasing access to psychological therapies will cost NHS nothing, says report

Clark, D.M and Layard, R (2015) Thrive: The Power of Evidence-Based Psychological Therapies London: Penguin.

IAPT (2018) Psychological Therapies: Annual report on the use of IAPT services England, 2016-17 Data Tables. NHS Digital: Community and Mental Health Team.

Mental  Health Policy Group of the Centre for Economic Performance (2012) How mental health loses out in the NHS.   http://cep.lse.ac.uk/_new/research/mentalhealth/default.asp.

Scott, M.J (2018) IAPT: The Need for Radical Reform. The Journal of Health Psychology, 23, 1136-1147.

 

Dr Mike Scott