Tag: evidence based assessment

Categories
l

Is Evidence Based Treatment Possible Without Evidence Based Assessment?

‘no’, this is the take home message from a just published study by Moses et al in the Journal of Anxiety Disorders https://www.sciencedirect.com/science/article/pii/S0887618520300931. An evidence based assessment includes a diagnostic interview, as well as a clinical interview and psychometric tests. Moses et al (2020) summarise the literature that the inclusion of a diagnostic interview improves outcome, by minimising missed diagnosis and misdiagnosis. These authors bemoan their finding that only a small minority of Australian psychologists use a diagnostic interview, but the position is even worse in the UK, as the largest provider of services the Improving Access to Psychological Therapies (IAPT) explicitly excludes the making of diagnosis/diagnostic interviews.   IAPT cannot improve access to evidence based psychological therapies because it does not operate the admission gate of an evidence based assessment.

The absence of an EBA leads to a revolving door, demoralising clients in search of a credible explanation of their difficulties. An EBA is a necessary part of evidence based practice (EBP) in that it highlights candidate evidence supported treatments (ESTs). But clinical judgement is still required to ascertain whether there is a sufficient match between client and the subjects in the EST. Most ESTs have admitted clients to the study with a limited range of comorbid disorders and have not been cognitively impaired, or suffering debilitating pain. Further the clients in the EST have been in a safe environment. 

 

Dr Mike Scott

Categories
l

Psychiatrists and Social Workers Acting With Impunity

A client of mine LC, see news item link , lost access to her 3 children for 5 months because a psychiatrist declared she had an emotional unstable personality disorder. I protested to the Court that this was wholly unfounded, having treated LC succesfully myself with CBT for depression. The appointed Expert Witness agreed with me that LC did not have an emotionally unstable personality disorder. The social workers refused to distribute my report to the agencies involved and were insistent that she attend a non-evidenced based treatment targetted at personality disorder. LC was mandated to attend regular meetings with social workers and other agencies. After legal action against Cumbria County Council they have apologised simply on the basis that there were no grounds for insisting that access had to be supervised.  The link to the News item is below:

http://www.newsandstar.co.uk/news/Cumbria-County-Council-sorry-after-mums-five-month-hell-0b215a18-055c-43e3-8b03-69ff77b5a05e-ds

But there is absolutely nothing to prevent this psychiatrist or any other, from making a diagnosis without careful reference to diagnostic criteria, routinely psychiatrists use criteria in a cavalier manner. There is no systematic enquiry about each of the symptoms that constitute a diagnostic set and no use of published thresholds to determine whether a symptom is present at a clinically significant level. This is no academic matter, the abject misery caused to my client LC for those 5 months was absolutely horrendous. To my knowledge there have been no sanctions against the social workers, they simply took a view of LC, taking their lead from the psychiatrist and refused to consider information that might contradict their conclusions. They showed no understanding of the concept of a personality disorder nor of what constituted evidence-based treatment. These professionals have been left to carry on and wreak havoc wherever, doubtless in good faith.

These matters should be a serious concern for the Royal College of Psychiatrists and the British Association of Social Workers but I have no confidence that these matters will be addressed because they are endemic within those professions.

Legal proceedings are still continuing for human Rights Violations by Cumbria County Council in respect of 2 of the children.

 

Dr Mike Scott

Categories
l

A New Year Resolution – Take a First Step to Evidence Based Assessment

It is likely that for most agencies 80% of the work is in relation to a few disorders, (Pareto’s Law of the ‘Vital Few’) so developing an expertise  at identifying these accurately represents a good investment of time and effort.        First of all it is necessary to accurately audit what is coming in through you agency, this could involve using a screen for the 10 most common disorders, the First Step Questionnaire Revised ( available freely on this blog) and then  ask the further diagnostic questions in Simply Effective CBT  [ Scott (2009)] for any positive screen.  Knowing say the 4 commonest disorders dealt with by your agency it would be comparitively easy to cascade training for the accurate identification of these to front line staff, who have perhaps little training.   At a later date training can be given in say the next 4 most common disorders  and  so on so ultimately there is a comprehensive assessment.  Such rigorous assessment is also the best way of monitoring outcome.

Questionnaires completed by the client are subject to demand characteristics, clients wanting to please the therapist or convince themselves that they haven’t wasted time in attending therapy. Administration of weekly psychometric tests are particularly suspect, as clients can remember their scoring and show reductions which can be mistaken for real world differences.

 

Dr Mike Scott

Categories
l

Gifting Evidence Based Assessment

Evidence based assessment (EBA) is like unwrapping a Christmas present, the first layer is ‘What are we dealing  with (prediction)?, the next what are the options for use, ‘what can we do about it (prescription)? the core is whether the present has made a real world difference ‘ how will we know if we are accomplishing our goals (process)?

 

The three phase approach to EBA, prediction, prescription and process is described (minus the Christmas present!) in December’s Clinical Psychology Science and Practice by Youngstrom et al with an accompanying commentary (including the questions above) by Steven D Hollon.   Youngstrom et al suggest screening for the 10 disorders that most likely comprise 80% of your workload. This can be achieved using the First Step questionnaire from Simply Effective CBT, Scott (2009) London: Routledge an updated free version for 10 disorders is on this blog. Then using a standardised semi-structured interview for positive screens to reliably identify disorders.  Knowing the disorder/s it is possible to predict what is likely to be the best treatment, and prescribe appropriate treatment targets and matching strategies.  Assessment is not just a front end process, assessing whether there has been appropriate process can only be gauged at the end when the assessment is repeated to determine whether the client has got their life back.

 

 

 

In practice many therapists paddle their own canoe, if they stop long enough to really listen they would find that despite the client’s politeness, oftentimes scoring a psychometric test low to please the therapist (weekly administrations have also been found to artificially lower scores), there has not been the real world change.

A very Happy Christmas to Everyone

 

Dr Mike Scott

Categories
l

IAPT Performance Investigation – National Audit Office Request

‘The National Audit Office is currently carrying out an investigation into the performance data of IAPT services. The investigation is focussing on waiting times, but also refers to the reported 50% recovery rates and can accept information about the collection and measurement of data across IAPT outcomes.  

This is a genuine opportunity for us to challenge the data on which mental health service targets are being set.

Many of you have important experience of what is happening in IAPT services that is crucial for the NAO in building an accurate picture of what is going on.

Please submit your evidence to Jenny George Jenny.George@nao.gsi.gov.uk and David Rarity David.Raraty@nao.gsi.gov.uk  who will be writing the report during August. It’s a tight deadline so please submit what you can as soon as possible.

The NAO website is HERE and below is the information provided about the inquiry.

Improving Access to Psychological Therapies performance data

The ‘Improving Access to Psychological Therapies’ (IAPT) programme increases access to National Institute for Health and Care Excellence approved treatment for depression and anxiety disorders. In October 2014, the Department of Health and NHS England jointly published Achieving Better Access to Mental Health Services by 2020. This set new standards for the time people should wait for mental health treatment and the care they should be able to access. In the case of IAPT services, the standards are that 75% of people referred should be treated within six weeks, and 95% within 18 weeks of referral, and that 50% of those who complete treatment will recover. NHS Digital publishes monthly statistics that report performance against these standards. This investigation will establish the facts around how the national statistics are prepared.

This is a really significant opportunity for us to share our experiences of what is going wrong in performance management of services. Please, take the time to contribute to the report’. 

 Thanks to Steve Flatt for alerting me to the above from the ‘Surviving Work’ website
Dr Mike Scott

 
Copyright © 2013 The Resilience Space, All rights reserved. www.theresiliencespace.com
Categories
l

The Silencing of Dissent and IAPT

 

This month’s Behavior Research and Therapy features a paper by Ali et al in which IAPT data on relapse after low intensity (Li) interventions is reviewed, and it is concluded that further attention to relapse prevention may be needed. I submitted a rejoinder essentially saying that Li-interventions have been a false economy and complaining that it had not been declared that Ali headed the Northern IAPT Research network, but it was rejected.

The editor began her letter of explanation with ‘Each of the reviewers is a highly experienced researcher in the area of low-intensity treatments for depression anxiety’.  But that is precisely the problem, researchers in low intensity see no pressing need for independent assessment using a ‘gold standard’ diagnostic interview (unlike their forbearers who conducted the bench-marking studies that gave CBT its’ credibility), although they pay lip service to it.

In practice, low intensity researchers find it ‘reasonable’ to conduct research on outcome solely on the basis of changes in a psychometric test. This strategy enables research to be done on the cheap, produce lots of papers and get brownie points in academia. There is a mutually beneficial groupthink amongst low intensity researchers and the IAPT hierarchy. Low intensity interventions fail an evidence based assessment test with a shameful lowering of the bar of methodological rigour. I will return in future blogs to editors/reviewers scant regard for criterion related validity and the misuse of Jacobsen’s Reliable and Clinically Significant Change Index, an abuse that is rampant in IAPT.

Dr Mike Scott

Categories
Current Psychological Therapy Issues

Evidence Based CBT Joins the ‘Endangered Species’ List

‘The good news is we have successfully amputated your right leg, the bad news is we operated on the wrong leg’.  The moral is that there can be no evidence based CBT treatment without an evidence based assessment (EBA). But EBA’s are increasingly absent in order to ease the research burden, ‘just rely on a psychometric test’, and reduce ‘treatment’ costs.

Emergency medicine operates on the 7 P’s, ‘Proper Pre-planning Prevents Piss-Poor Performance’. Is it seriously proper-pre-planning to operate as IAPT do, with a  typically 15-20 minute telephone conversation, conducted by the most junior member of staff, to be then placed on waiting list of upto 6 months for a treatment of indeterminate appropriateness? The reliability of this ‘pre-planning’ has never been assessed by anyone independent of IAPT. But this has not stopped Clinical Commissioning Groups often ring fencing IAPT monies at the expense of other mental health service providers.   The CCG’s need to be reminded that they ought to be working in an evidence based framework were the results of randomised controlled trials with independent assessment of outcomes hold sway (the top of the pyramid below). 60% of IAPT clients receive low intensity treatment, this is way down at the bottom of the evidence pyramid below, with a high potential for bias, reflected in powerful marketing.

It is time that CCG’s told IAPT that their pre-planning is simply unacceptable, there has to be an evidence based assessment. It is time the National Audit Office asked why are we continuing to fund a Service that has not been independently evaluated using ‘gold standard’, diagnostic assessment procedure. But most of all it is time to listen to those subjected to an inhospitable and unhelpful service. MP’s have to insist they are listened to.

Dr Mike Scott