Month: December 2023

Categories
l

The Myth That A Positive Psychometric Test Result Means Something

when administered, in the customary fashion, in isolation. It doesn’t, it is likely to be a red herring, about as useful as contending that because a person has a scar on their face they are likely to be a criminal. 

Service providers such as NHS Talking Therapies and Anxiety UK have adopted the routine use of the PHQ-9 and GAD-7 to chart the trajectory of Client’s symptoms. But have done so without first reliably clarifying which population the completer of the test is from. The particular completed questionnaire could have been handed in yesterday by a man in a red suit! There needs to be a fit between the domain of test and an identified disorder. The Service providers do not assess whether the chosen test is ‘fit for purpose’.

Psychometric tests are given credence if they have high sensitivity, (identifying a large proportion, typically 90%, of those with a disorder) and high specificity ( identifying a large proportion, typically, 90% of those without the disorder). But such impressive figures are only relevant if the population being addressed is known, This is quite unlike the routine clinical situation, in which the client has not been reliably categorised. The metric that is really important for a clinician is the positive predictive power of the test Monaghan (2021) positive predictive value reflects the proportion of subjects with a positive test result who truly have the outcome of interest.  But this is usually no better than chance. 

Only a minority of clients are likely to have are likely to have a particular disorder. There are many more people without a disorder than with a disorder. This means that using a test, even with a high specificity, the number of false positives may be several times greater than true positives. Resulting in widespread inappropriate treatment.

Harrison, A. G. & Edwards, M. J. (2023) assessed the the accuracy of 7 psychometric measures of ADHD and found that they were no more accurate than flipping a coin. The ability of self-report methods to accurately diagnose attention deficit hyperactivity disorder: A systematic review. Journal of Attention Disorders, 27(12), 1343-1359. https://doi.org/10.1177/10870547231177470 (Link)

Dr Mike Scott

Categories
l

Why Would You Believe In NHS Talking Therapies/Father Christmas?

A recent paper by Ost et al (2023) proclaims the good news that psychological therapy for the anxiety disorders can be effectively delivered in routine practice. But that is not to say that this ever happens as a matter of course. It only happens in the special circumstance were researchers take their wares to a non-academic site and deliver treatments as per protocol. Crucially, Ost et al (2023) make no comment on the external validity of their findings. To have raised such matters amongst CBT practitioners, would it seems be akin to debating the existence of Santa in front of the children!

It is the case that the results of effectiveness studies are comparable to those in efficacy trials, with a typically 50% remission rate. Leading Ost et al (2023) to advocate the wider dissemination of CBT. The findings and recommendations are likely to gladden the heart of NHS Talking Therapies Service providers, providing fuel for expansion. But it is an arbitrary inference to take the effectiveness studies as an endorsement of what happens in routine practice in the UK. 

 

Children grow out of a belief in Father Christmas, adults it seems can create their own fantasy at the expense of other adults and children. But it would be more truly Christmas all-year round if contacts with client’s began with:

‘ In your own good time, tell me what has been getting to you’

Christmas is about hope. Have a good Christmas.

Dr Mike Scott

Categories
l

‘I just can’t give you the time of day’

This is a potted summary  of an Editorial in this month’s British Journal of Psychiatry,  by Consultant Psychiatrist, Dr Martin Deahl, reflecting on his 38-year career, as he nears retirement. But the same could be said by psychological therapy practitioners. He makes the point that really listening is a necessary ingredient in any psychotherapeutic intervention. Pre-NHS Talking Therapies/IAPT this was taken as axiomatic by all mental health professionals, but it was quickly abandoned by the Service. Drew et al’s (2021) and Faija et al’s (2022) studies have evaluated transcripts of  treatment sessions and discovered a machine-like process, were priority is given to client’s completion of psychometric tests, which do not inform treatment. Whichever direction one looks, it is difficult to escape the conclusion that mental health clients have been seriously failed. 

NHS Talking Therapies has improved access to psychological therapies but without the therapist being present in any meaningful way. Only a significant minority of clients attend two or more treatment sessions, investing time and energy in the therapeutic enterprise. Most are appreciative of the therapist’s efforts and do not want to believe that they have wasted their energies. But few believe that they are either back to their old-self post-treatment or functioning at their best ever. There is a conspiracy of silence about treatment not being personalised. Client’s are kept in the dark that they are missing out. 

Recently I met a lady who had had a very bad car accident, she underwent a telephone assessment with NHS Talking Therapies and signed up for a 6 session group programme for PTSD. There were 20 people in the group and discussion of personal traumas was forbidden. When I asked her about her experience she said it was ‘helpful’ but there was no real-world change in her functioning. She said that she was given a trauma related psychometric test at the beginning and end. Her score improved and they suggested, on this basis,  that she didn’t really need to come back for individual therapy but could if she wished. She didn’t take up the option. Her treatment has been a con:

  • she did not know that there is no empirical evidence that a 6 week group programme for PTSD is evidence-based.
  • Nor did she know that she was given a treatment for something she might not have.
  • She was deprived of informed consent. 

There is a pandemic of presenteeism amongst psychological therapists with many unaware that they are carriers, passing their ‘best practice’ on to others.

Dr Mike Scott