CBT Outcome Studies – An Example of The Problems of Generalisation

The problems of generalising from CBT randomised controlled trials can be illustrated by examining the following just published study from Clark et al (2022):


  1. All patients were referred by an Improving Access to Psychological Therapies (IAPT) Service if they were thought to be suffering from social anxiety disorder(SAD). But following detailed diagnostic assessment only one half were found to be suffering from the disorder. Thus it seems likely that IAPT staff, left to their own devices, would have provided inappropriate treatment for one in two patients.
  2. IAPT staff do not make assessments using standardised diagnostic interviews, so that there can  be no certainty that the results of this, or any other credible randomised controlled trial, would translate into routine practice.
  3. The three treatment  clinicians involved in the trial involved were experienced, well known clinical psychologists, unlike the therapists in routine practice. It is unlikely the latter would achieve the same outcome as the former.
  4. In the study internet delivered CBT and standard CBT were the active comparisons and the results set against a waiting list control condition. But patients on a waiting list do not expect to get better and this comparotor has therefore been termed a nocebo. Given such able clinicians, it is possible that, if they had provided an alternative treatment with a credible rationale e.g ‘managing shyness/better mixing’ with equal attention the results would not have been appreciably different. The chances of this are increased by Clark et al (2022) finding that many of the standard components of the original CBT, such as accompanying the patient to social experiments were found to be redundant.   
  5. The study authors were evaluating, the computerised CBT program that they had developed. But no mention that the study  requires independent replication because of the possibility of allegiance bias. Interestingly the authors report no conflict of interest. 
  6. The study required patients to have internet access and almost two thirds had higher education. The results may not be applicable in forgotten towns.
  7. Patients were required to have SAD as their main problem, but patients typically see themselves as having a range of difficulties and want treatment for all. In this study 30% were found to be suffering from depression. But strangely, overall the pre-treatment mean on the PHQ9 was below the cut off of 10 usually used to denote a case of depression. Further this score is much lower than the initial mean of PHQ9 scores in the IAPT population.  This creates doubts about the level of functional impairment in this population. 
  8. On the surface the study results are remarkable with 70%+ recovering from SAD and avoidant personality disorder. But the primary outcome measure was a composite of loss of diagnostic status and achieving below cut-off scores on several SAD self-report measures. There could be no certainty that the components of the composite were equally important to each patient, nor that the frequencies of say loss of diagnostic status and below cut off scores matched. The hazards of using composites has been highlighted by McCoy (2018).
  9. Patients see recovery as being free of a disorder for a meaningful length of time. Given that persistent SAD is defined in the DSM as having the disorder for at least 6months. It therefore seems reasonable to suggest that a primary outcome measure should have been being free of SAD for at least 6 months. This would have been a real world change.

This study is a salutary tale about the marketing of CBT – the takeaway message in the title is that the cost of one intervention(internet CBT) is half the cost of standard CBT. This is not to say that CBT is not of limited utility with depression and the anxiety disorders (including OCD and PTSD) but we need to assess what is of real world importance to the patient.


Dr Mike Scott