with recordings from an IAPT (Improving Access to Psychological Treatment) client, IAPT therapist, myself and the British Medical Association. The 37 minute programme will be broadcast on Radio 4 on Tuesday, September 24th at 8.0pm and repeated on Sunday, September 29th at 5.0pm.
hopefully this will be a springboard for the expression of the views of those most effected by IAPT, and will lead to a transformation of the Service.
IAPT shouts eureka at this point, discharges the client and claims the improvement is due to the therapists efforts. It is like declaring that a person who is terminally ill is cured because they have had a good refreshing day gardening. In effect IAPT has gone fishing for flashes in the pan – I make similar points in a forthcoming BBC Radio 4 investigation into IAPT.
But studies of depression and the anxiety disorders e.g Bruce et al (2005) require a 2 month period of symptoms not significantly impairing functioning (see also DSM-5 criteria for recurrence of depression). Bruce et al (2005) point out that anxiety sufferers naturally only have symptoms 80% of the time. With regards to depression Stegenga et al (2012) point out that for 40% of depression sufferers their depression naturally takes a variable course. Without independent assessment of the period for which the person is without significant symptoms talk of remission/ recovery is meaningless. IAPT’s clients have not been assessed using this metric.
Bruce et al (2005) https://www.dropbox.com/s/9powmto8miw60a2/Natural%20recovery%20in%20Social%20Phobia%20Panic%20Disporder%20and%20Generalised%20Anxiety%20Disorder.pdf?dl=0
Stegenga et al (2012) https://www.dropbox.com/s/k0x2fm0ds01no0k/natural%20course%20of%20depression%20stegenga%202012.pdf?dl=0
IAPT’s low intensity CBT should be re-branded ‘below intensity CBT’, as all the methodologically rigorous CBT outcome studies were conducted on full dose CBT. Guided self-help (GSH) interventions were first recommended by a NICE committee in 2007 and 2009 for depression and the anxiety disorders. In its’ wake IAPT enthusiastically adopted GSH such that by 2018, 70% of clients were being given it. But recently therapists have been told not to use the term ‘GSH’ but talk to clients instead of ‘low intensity CBT’. This re-labelling appears to have occurred because of the difficulties of engaging the public in this more obviously cheap option (see previous post).
But NICE did not conduct a systematic review of the outcome literature, rather its’ recommendations were simply the advice of its’ committee. It failed to acknowledge that there were no studies of ‘guided self-help (GSH)’ with a hard outcome measure i.e studies involving an independent blind assessor using a standardised diagnostic interview. Thus there was no evidence that the man/woman in the street would recognise that the GSH had returned them to normal functioning. However the recommendation of NICE was that the low intensity interventions had to be matched to the particular depression or anxiety disorder. But IAPT took what it wanted from the NICE guidance, jettisoned making a diagnosis and proclaimed that appropriate treatment could follow a problem descriptor, without any empirical evidence for the latter. The upshot is that for a decade IAPT clients have largely been subjected to ‘below intensity cbt’.
There has been a decade of ‘the below intensity CBT’ revolution and it has failed. This is not to say that there may not be cheaper effective options for service delivery such as group CBT, but the scope for such interventions is limited to depression and some anxiety disorders and much more methodologically rigorous outcome studies are necessary to confirm its place.
the IAPT Annual Report (2018)/2019] see link below, reveals that a third (31.2%) of new referrals drop out before treatment and approximately two thirds (61.1%) do not complete a course of treatment (using IAPT’s liberal definition of treatment as attending 2 or more session) with almost a third (29.54 %) attending only one treatment session.
IAPT’s disengagement is illustrated by Jock’s records which revealed that at age 6 he had behaviour problems and threatened to stab himself. By age 14 he was diagnosed with oppositional defiance disorder and was short tempered. At age 19 he was diagnosed as having an anxiety state low mood drinking 10 units in a binge once or twice a fortnight and cannabis 2-3 times a week. Despite his extensive history he was assessed by IAPT and assigned to a step 2 (low intensity) workshop, unsurprisingly he DNA’d. Two years later he is referred to them again for depression and unsurprisinly he does not respond to their opt in letter. Five years later the GP notes that he is struggling with an online CBT course has had to enlist his father to help because he is not computer literate. Then after a major negative life event he develops a depressive psychosis. Had IAPT bothered to listen this troubled soul of longstanding, the results could have been very different.
Engagement difficulties are built into the fabric of IAPT. Daniel consulted his GP 2 years after a major trauma and was found to have PTSD and depression and was promised a referral to IAPT. 4 weeks later he was prescribed an increase in medication and a different GP gave him IAPT’s telephone number to ring. Daniel was furious, he felt that he had explained that his mood was very up and down and that he could not be relied on to ring them. His interpretation of the organisational setup was that no one was really interested. This perception was likely to be compounded if and when he underwent a telephone assessment as had already had lots of acrimonious telephone conversations with the housing Dept and DWP since his trauma.
It is surely time f or the Care Quality Commission and the National Audit Office to take note of the near universal disengagement of clients, voting with their feet, and institute an independent review of IAPT to determine what if any real world difference it makes. There is considerable media interest in these failings.
as an External Examiner I found it painful to watch videos of trainees trying to ensure their interview with a client was CTRS (or its’ successor the Revised Cognitive Therapy Rating Scale) compliant. I vividly remember one Course Leader giving a student a score above the competence threshold of 36 (see R-CTRS Manual, link at end of blog ), despite the student barely making eye contact with the client – the latter was busy leaving through the 12 item (each scored 0-6) scale on his lap! Unfortunately institutions were obliged to use it, and students groaned in silence. Before another cohort of trainees endures this rite of passage, those that have the courage to, should consider that the R-CTRS may cause more problems rather than it solves i.e that it is, iatrogenic.
Life Before the CTRS
In the seminal study of CBT for depression by Rush, Beck, Kovacs and Hollon (1977) https://link.springer.com/journal/10608 they did not use the CTRS. Later when Steve Stradling and I conducted a randomised controlled trial comparing individual CBT, group CBT and treatment as usual, [Behavioural Psychotherapy, 18, 1-19] we simply followed Beck’s protocol, p409-411 (1979) Cognitive Therapy of Depression, by Beck, Rush, Shaw and Emery., published by John Wiley and Sons to achieve good results.
The Poor Predictive Value of the R-CTRS
The CTRS has only been evaluated in a sample of depressed clients undergoing cognitive therapy [Shaw et al (1999)] , therapists scores on this did not predict outcome on self-report measures the Beck Depression Inventory or the SCL-90 (a more general measure of psychological distress) however it did predict outcome on the clinician administered Hamilton Depression Scale predicting just 19% of the variance in outcome, but it was the structure parts of the scale (setting of an agenda, pacing, homework) that accounted for this 19% not items measuring socratic dialogue etc. The authors concluded: ‘The results are, however, not as strong or consistent as expected’When the CTRS was first evaluated the results were not compelling’ . I enlarged on this previously in my blog ‘The Mis-Selling of the R-CTRS’ http://www.cbtwatch.com/mis-selling-cognitive-therapy-rating-scale/.
The R-CTRS and IAPT
Since my earlier blog I have more recently blogged ‘Jump Through Our Hoops an Make No Difference To Client Outcome’, http://www.cbtwatch.com/iapt-training-jump-through-our-hoops-and-make-no-difference-to-client-outcome/ in which I noted an IAPT study that had used the R-CTRS to predict outcome and found that there was no relationship to outcome.
Cavalier Usage of The R-CTRS
Studies using the R-CTRS tend to be cavalier. In a study comparing BA and CBT, Richards et al (2016) used the R-CTRS but these authors did not report how this or indeed the competence measure for BA related to outcome. Richards et al said that though both modalities were equally effective in treating depression, but BA was to be preferred because it was cheaper to train therapists in BA. They further claim that the CBT therapists were competent with a mean score of 37.9 on the R-CTRS ( but this score is almost identical to the threshold of 36 in the R-CTRS Manual deemed necessary for a competent therapist) so on this metric half of the CBT therapists were not competent. Thus there has been a meaningless implementation of CBT. Paradoxically it may be that the CBT therapists performance had been made worse by having to use the R-CTRS.
Spinning The R-CTRS
Given the paucity of evidence for the utility of the R-CTRS for depression and possible negative side effects one would expect that it would not have been applied to other disorders. Unfortunately trainees are asked to apply it to whatever the client’s complaint or ‘problem descriptor’ as IAPT would have it. Little wonder that trainees are stressed by its’ usage.
CBT luminaries are spinning the plates furiously this conference season, a paper in next months Behavior Therapy, 50 (2019) 864–885 by clinicians from the University of Sheffield, has an abstract that advocates Group Behavioral Activation for depression as a front line treatment. The abstract also claims a moderate to large effect on depressive symptoms. Most people are unlikely to read further than the abstract, but closer inspection reveals the conclusions are deeply flawed.
In passing the abstract mentions that the standardized mean difference (SMD) between group BA and waiting list was 0.72. This would cause few people to question the findings, but actually it means the results are of doubtful clinical relevance, as it actually means there is less than one standard deviation in outcome between the treated group and the waiting list. Your eyes may already be glazing over at the thought that some stats are on the way, but bear with me. If a group of depressed patients had a mean Beck Depression Inventory Score of 28 at the start of treatment, [assuming that the spread of the results was 7, the standard deviation – taken from the Scott and Stradling (1990) study Behavioural Psychotherapy, 18, 1-19 ] a mean score of 23 at the end of treatment would produce an SMD of 0.71, i.e about the same as in the University of Sheffield analysis. Thus the average person experiencing this change of score is unlikely to feel that they are back to their normal selves, and are likely to view it as part of the normal cycling of mood, influenced by positive events e.g the company/support of fellow sufferers for a time in a group. In none of the Group BA studies was there an independent assessor determining whether clients were still depressed or the permanence of any change. Unsurprisingly the authors found that the Group BA was no better than any other active treatment (i.e controlling for attention and expectation), and make an implicit plea for the Dodo verdict ‘ all therapies are equal and must have prizes’.
In the body of the paper the authors acknowledge that the Group BA studies are of low quality, save one and that analyses were on treatment completers as opposed to the more rigorous intention to treat. But there is no indication anywhere as to what proportion of people recover from depression with any permanence. Yet this did not stop the spin in the abstract! Unfortunately it will likely be music to the ears of IAPT and one can expect Group BA to be soon advocated, particularly as it is contended that BA is easier for therapists to learn than CBT.
In 1990 Steve Stradling and I had published [Behavioural Psychotherapy, 18, 1-19] a study of depressed clients comparing, group CBT, individual CBT and a waiting list condition. For Group CBT the initial mean BDI was 29.0 and end of treatment score was 6.2 whilst for individual treatment the comparable scores were 28.21 and 11.53. However those on the waiting list also improved from 25.89 initially to 20.26 at the end of waiting list. Thus, it is far from clear that the results from the University of Sheffield analysis on Group BA are actually better than those of putting people on a waiting list.
In the August 2015 issue of the Psychologist I wrote:
“In the July issue of the Psychologist you referred to a meta-analysis of 70 CBT studies for depression conducted by Johnsen and Friborg (2015) and opined ‘CBT doesn’t seem to be helping reduce depression symptoms as much today as it used to when it was first developed in the 1970s’. But this conclusion may be premature, inspection of Table One of Johnsen and Friborg’s study shows that from 1977 up to and including the millennium 85% of studies were randomised controlled trials (RCT’s) but from 2001-2014 the comparable figure was 65%. One of the hallmarks of an RCT is blind assessment, using a standardised diagnostic interview. Thus there can be no certainty that populations treated post the millennium are comparable to those before. Johnsen, T. J., & Friborg, O. (2015, May 11). The Effects of Cognitive Behavioral Therapy as an Anti-Depressive Treatment is Falling: A Meta-Analysis. Psychological Bulletin. Advance online publication. http://dx.doi.org/10.1037/bul0000015″ Reliance on weak evidence has become a post-millenium phenomenon.
But spin is not confined to recent CBT studies, Jellison et al (2019) have examined spin in leading journals of psychiatry and in the journal Psychological Medicine, of 116 randomised controlled trials spin was identified in 56% of them, with 21% in the abstract results section and 49.1% in the abstract conclusions section. See link below:
IAPT is camouflaging what most of its clients receive and has eskewed a focus on clinically relevant outcomes. But one of the domains that the CQC assesses services against is whether they are Outcomes-focused. The CQC needs to conduct an inquiry into IAPT.
Guided Self-Help (GSH) has been the diet of 71% of IAPT’s clients, but therapists have now been advised not to mention GSH, because it may be off-putting! But rather to refer instead to ‘low intensity telephone CBT’ . Notwithstanding that NICE has justified its’ support for low intensity CBT on the basis of studies that were termed ‘GSH’. There is a transparency about offering GSH, clients have a right to know what they are letting themselves in for. Informed consent cannot be meaningfully given to a term like ‘low intensity telephone CBT’.
The matter of informed consent is compounded further by IAPT by their failure to inform clients of what clinically relevant outcome he/she can expect. In particular what minimally important difference the client can expect and clearly see as meaningful. Changes on a psychometric test do not qualify as a clinically relevant outcome by contrast a client can clearly understand say an expectation to be back to their usual self.
IAPT’s ‘low intensity telephone CBT’ itself rests on a fault line, studies that found statistical significance between groups e.g computer assisted CBT vs waiting list, but without a) any discussion of the clinical relevance of the findings and b) blind independent assessment of outcome. Dissemination of the low intensity interventions has been promoted on the back of statistical significance rather than clinical relevance. This makes it imperative that the CQC becomes outcomes focused in a transparent way and is not sucked in by IAPT’s self serving surrogates.
there are no limits to IAPT’s ambitions, making failure inevitable. IAPT’s target in practice is, “whatever the client complains of” and treatment is operationalised as “whatever its’ therapists do”, Both focii are so loose that it cannot fulfill it’s promise, like a totalitarian revolution that runs out of steam.
The IAPT Manual published a year ago leaves both targets and treatment ‘fuzzy’, whilst proclaiming a commitment to NICE Guidelines. A target of ‘client complaints’ makes no distinction between ‘ disorder’ and everyday unhappiness/stresses. Yet the treatments advocated by NICE are quite specific to disorders.
At most IAPT staff ask about some symptoms of a disorder, but without coverage of all the symptoms of a disorder. But they are not taught to ask whether a symptom is present at a clinically significant level, i.e whether it is making a real world difference to a client’s life. Only clinically significant symptoms count in DSM. As a result IAPT client’s are typically treated for disorders they don ‘t have, without any fidelity check on compliance with a protocol.
There is tremendous vested interest, financially, emotionally and intellectually in IAPT continuing as it is, marking its’ own homework with applause from BABCP and the BPS.
day workshop September 20th, Liverpool. Details from BABCP website are below:
GROUP CBT SPECIAL INTEREST GROUP
Group CBT – Forward Leap or Backward Somersault?
Led by Dr Mike Scott Friday 20 September 2019 Times: 9.30am to 3.30pm (Registration from 9.00am) Venue: 54 St James Street, Liverpool, L1 0AB
About the workshop: Groups are a way of addressing the commonalities amongst peoples’ difficulties/disorders [Scott (2011)]. They also offer the enticing prospect of a wider dissemination of services. This workshop addresses the question of when is group CBT effective and when is it a pawn in a numbers game? Jo Clifford will present the NICE recommendations on group CBT. Attendees will take part in a simulated stress management group (SMG), to experience the trials and tribulations of being a leader, co-leader and group member. A framework for understanding and managing interactions will be described. The criteria for judging a group as effective will be considered and participants will be asked to deliver a verdict on an SMG group. This then serves as a methodological template for evaluating other group interventions. The therapeutic relationship is a key element of both individual and group CBT but takes a slightly different form in the latter [Whitfield and Scott (2019)]. These differences will be explicated in the workshop. Participants are invited to reflect on their own experiences of groupwork: in this connection Nicola Walker will give a presentation on the side effects of group therapy. Finally, participants will be encouraged to reflect on the scope for implementing group CBT where they are.
Scott, M.J (2011) Simply Effective Group Cognitive Behaviour Therapy London: Routledge Whitfield, G and Scott, M (2019) CBT Delivered in Groups in ‘The Therapeutic Relationship in Cognitive Behavioural Therapy Edited by S Moorey and A Lavender London: Sage publications.
Registration and General Information BABCP Member fee: £50 Non-Member fee: £70 BABCP Member Student: £40 Non-Member Student: £60 Lunch and refreshments will be provided. A CPD certificate for 6 hours will be issued – 50/50 skills and theory Closing date for registrations is Friday 13 September 2019 For any event or booking queries please contact BABCP head office on 0330 320 0851 or email firstname.lastname@example.org For venue enquiries please see website – http://www.thewomensorganisation.org.uk/54stjamesstreet
Timetable 9.00am Registration 9.30am Mike Scott Psychoeducation strengths and limits 9.50am Jo Clifford NICE Recommendations for Group CBT 10.10am Mike Scott Group CBT in practice 10.30am Coffee 10.50am Mike Scott Simulated Group Session of Stress Management. Reflections of participants, leader and co-leader 11.30am Mike Scott Framework for evaluating group skills and organisational context. But do skills relate to outcome? 12.30pm Lunch 1.15pm Nicola Walker Side effects of Group Therapy 1.45pm Nicola Walker Participants’ experiences and views on best practice for group CBT Discussion 2.30pm Coffee 2.50pm Putting it together where I am 3.30pm Close
applying the acid tests of the Cochrane Collaboration Tool and the GRADE Handbook for the quality of randomised controlled trials, studies of low intensity CBT fail to clear the methodological bar. Whilst only high intensity studies for depression and the anxiety disorders make a successful jump. This calls into question IAPT’s penchant for disseminating CBT for everything, with an imprimatur from BABCP, paying travel expenses of upto £100 for special interest group members to attend a pre-conference workshop Revolution in Mental Health Service Delivery: The Evolution of Low Intensity CBT on Tuesday 3rd September.
One of the seven domains highlighted by the Cochrane Collaboration tool for assessing bias is the blinding of outcome assessment. I have been unable to locate one outcome study of low intensity CBT that fulfills this criteria whilst there are a significant minority of studies of high intensity interventions for depression and the anxiety disorders that do.
The GRADE handbook for assessing the quality of trials comments in section 3.4 ‘not infrequently, outcomes most important to patients remain unexplored’, with regards to psychological interventions clients are rarely asked by someone independent of the study whether and if for how long they are back to their usual selves since treatment. Instead most commonly reliance is placed on a surrogate measure a client completed questionnaire, as opposed to an independent clinicians assessment using a standardised diagnostic interview to determine whether there has been a loss diagnostic status.
These concerns are crystallised in a study of CBT for Health Anxiety conducted by Cooper et al (2017), Behavioural and Cognitive Psychotherapy, 2017, 45, 110–123 doi:10.1017/S1352465816000527
whilst 10 of the 13 studies in a meta analysis used the DSM or ICD-10 to determine whether people should be admitted to the meta analysis, in no study was meeting these criteria used as an outcome measure. To be no longer suffering from the identified health anxiety at end of treatment/follow up would have been a client important outcome. Instead the self-report Health Anxiety Questionnaire was used as surrogate. Cooper et al (2017) attempted to rate studies using the Cochrane Collaboration tool using a summary score for the seven domains, but this bore no relation to outcome and as the authors admitted was a questionable procedure. Despite this CBT was claimed to be an effective treatment for health anxiety.
I am afraid I can’t join in the jamboree for IAPT services that takes place at the BABCP annual conference. I doubt that the ‘House of Cards’ will be discussed and it would likely be seen as banned literature on IAPT training courses.