The Myth That A Positive Psychometric Test Result Means Something

when administered, in the customary fashion, in isolation. It doesn’t, it is likely to be a red herring, about as useful as contending that because a person has a scar on their face they are likely to be a criminal. 

Service providers such as NHS Talking Therapies and Anxiety UK have adopted the routine use of the PHQ-9 and GAD-7 to chart the trajectory of Client’s symptoms. But have done so without first reliably clarifying which population the completer of the test is from. The particular completed questionnaire could have been handed in yesterday by a man in a red suit! There needs to be a fit between the domain of test and an identified disorder. The Service providers do not assess whether the chosen test is ‘fit for purpose’.

Psychometric tests are given credence if they have high sensitivity, (identifying a large proportion, typically 90%, of those with a disorder) and high specificity ( identifying a large proportion, typically, 90% of those without the disorder). But such impressive figures are only relevant if the population being addressed is known, This is quite unlike the routine clinical situation, in which the client has not been reliably categorised. The metric that is really important for a clinician is the positive predictive power of the test Monaghan (2021) positive predictive value reflects the proportion of subjects with a positive test result who truly have the outcome of interest.  But this is usually no better than chance. 

Only a minority of clients are likely to have are likely to have a particular disorder. There are many more people without a disorder than with a disorder. This means that using a test, even with a high specificity, the number of false positives may be several times greater than true positives. Resulting in widespread inappropriate treatment.

Harrison, A. G. & Edwards, M. J. (2023) assessed the the accuracy of 7 psychometric measures of ADHD and found that they were no more accurate than flipping a coin. The ability of self-report methods to accurately diagnose attention deficit hyperactivity disorder: A systematic review. Journal of Attention Disorders, 27(12), 1343-1359. (Link)

Dr Mike Scott