Tag: Abuse of Psychometric tests

Categories
l

IAPT’s Researchers Failure to see the Elephant in the Room

 a meaningless outcome measure.

The January issue of the American Journal of Psychiatry, contains a paper by Delgadillo et al https://jamanetwork.com/journals/jama/fullarticle/10.1001/jamapsychiatry.2021.3539?utm_campaign=articlePDF%26utm_medium=articlePDFlink%26utm_source=articlePDF%26utm_content=jamapsychiatry.2021.3539 claiming that 52.3% of those routed along a stratified treatment pathway showed a reliable and clinically significant improvement (RCSI), compared to 45.1% along the Improving Access to Psychological Therapies (IAPT) usual stepped care pathway. The additional cost of the stratified pathway was £104.5 per patient, representing the additional time devoted to a patient  to determine the data needed to put into an algorithim and determine whether low intensity should be first or high intensity first. The authors concluded that for this additional sum there was approximately a 7% increase in the probability of RCSI. But at no point do the authors question the validity of adopting IAPT’s self-report metric for outcome.

 

There has never been independent assessment of treatment outcome in IAPT. Further there is no evidence that the changes in IAPT self-report measures represent an added value over the comparable changes when counselling was employed pre-IAPT. There are a number of reasons why there would be improvements on self-report measures as treatment progresses that have little to do with therapeutic effectiveness including: A) regression to the mean, as patients tend to present initially at their worst B) the test results are a focus in therapy, creating a demand issue for the patient and C) patients, understandably, do not wish to feel they’ve wasted their time.

IAPT ignores the fact that the context in which a psychometric test is conducted is crucial. Used in isolation, they are a mirage of the client’s real-world concerns. The RCSI is a perfectly reasonable outcome measure if used in a controlled trial in which the diagnostic status of the patient has been assessed with a standardised reliable interview, at the beginning and at a minimum post-treatment. But in the Delgadillio et al (2022) study, as in all IAPT studies, no diagnosis is made using in gold-standard semi-structured interviews. The population addressed lacks specificity, the only boundary for entry into the study was a PHQ9 score greater than 10, making replication highly problematic. The title of the Delgadillio et al (2022) study suggests that focus was on depression but there can be no certainty that this is actually that case.

 

The Delgadillio et al (2022) study does not address whether a patient would see the apparent difference in outcome between stratified approach and a step approach as a difference that matters. It is impossible to gauge from study what proportion of patients lost their diagnostic status along the differing trajectories. The self-report measures used by refer to functioning in the previous two weeks, patients typically have their treatment’s terminated when their score falls below a threshold of 10 on the PHQ9. But anxious and depressed patients experience waxing and waning of symptoms so that a reliable outcome must specify the duration of recovery, for example eight weeks. The supposed recoveries in IAPT could often be flashes in the pan.

Dr Mike Scott

Reference

Delgadillo J, Ali S, Fleck K, et al. Stratified Care vs Stepped Care for Depression: A Cluster Randomized Clinical Trial. JAMA Psychiatry. 2022;79(2):101-108. doi:10.1001/jamapsychiatry.2021.3539

 

485 words

 

 

Categories
l

IAPT And The Abuse of Psychometric Tests

A person who consents to a psychometric test has a right to a full explanation of its purposes. I have not met an IAPT client who has been given such an explanation. IAPT employees see it as a requirement of the organisation for ‘audit’, but this is not an explanation. Informed consent means that it has to be explained what would be the consequences of not taking the test, this never happens. It is important that tests are only given that are relevant to the purposes of evaluation (not to do so probably breaches data protection legislation). But in administering say the PHQ-9 the IAPT worker does not know whether this is pertinent to whatever the client is suffering from e.g OCD or PTSD (as there is no reliable standardised diagnostic interview). Further the client isn’t informed of the purpose to which the test result will be put, e.g it will be used by IAPT in such a way that any positive change on it greater than 6 will be publicised as indicating the difference the Organisation makes. It is not explained that the PHQ-9 was developed with funding from Pfizer, the drug company who would clearly benefit from the overidentification of depression. Further the PHQ-9 was extracted from the Prime-MD interview, taken out of this context its’ meaning is questionable.

Psychologists wield power in IAPT, they know or at least should know about the appropriate use of psychometric tests e.g if they are administered weekly the person can remember their last response thus biasing scoring. If on their watch they are allowing others to misuse them then this may be a matter for the HCPC and for some also the University body that employs them. Psychologists and Universities can not be complicit in a Government Quango marketing itself.

Dr Mike Scott