The Mismatch Between Clients Global Judgements and Changes on Psychometric Test

casts doubt on the wisdom of the Improving Access to Psychological Therapies IAPT) services sole reliance on test results.  A recent study by Hobbs et al (2020) https://doi.org/10.1017/S0033291719003878 compared subjects global judgements on improvement/deterioration with changes on the PHQ9 and found poor agreement. For those who reported ‘feeling worse’ PHQ9 scores showed no change or improvement for 76% of them.  Further for those who reported ‘feeling a lot worse’  for 81% of them the PHQ9 showed no change or an improvement.  

Hobbs et al (2020) conclude that test results tell only part of the client’s story and there is a need for clinical assessment.

The interview assessments in IAPT go no further than the open ended interviews that are the first part of all standardised diagnostic interviews. Open-ended interviews furnish at most differential diagnoses and highlight candidate disorders for further investigation.  Further inquiry is then made of all of the symptoms in a diagnostic set ( controlling for information variance) and thresholds are used to determine whether a symptom is present  at a level that would constitute functional impairment (controlling for criterion variance).  Reliable diagnosis makes it possible to determine which evidence based treatments are likely to be appropriate in a particular case, bearing in mind the client’s social context and cognitive capacities.

Ultimately global judgements have a more real world feel than changes on psychometric test. A client knows whether they are back to their old selves post treatment and whether they would regard this as enduring. Similarly a client with a lifelong history will now whether they are back to their ‘best functioning’ post treatment and whether it is just one more ‘flash in the pan’. Similarly an independent assessor of a Service can make g global judgement (using a standardised diagnostic interview) whether the person has lost their diagnostic status post treatment and whether there is evidence of permanence. Psychometric tests have been grossly overvalued by IAPT for quality control purposes, but they are fantastic for marketing to the unwary.

Tests Misleading for Diagnostic Acccuracy

It is common for advocates of psychometric tests to quote high sensitivities/ specificities of the order off 80% for instruments such as the PHQ9. But this does not mean that using the PHQ9 on all clients coming through the IAPT door that using its 10 or greater cut off 4 out of 5 clients will be correctly classed.  Tests are validated in a particular context, thus if the proportion of clients with diagnosed disorder (using a diagnostic interview)  differs  by context then so to will the appropriate cut offs and sensitivities/specificities.  Further a psychometric test does not indicate what other disorders are present nor which is the primary disorder. Treatment that fails to address comorbidity is likely to fall short and comorbidity is the norm not the exception.