Evidence Tables

The evidence base for many common mental health problems has already been interrogated using a transparent and rigorous process in the production of the various SIGN and NICE guidelines and these published documents form the basis of the evidence tables where possible.

1. The evidence base for any intervention, as currently defined in SIGN and NICE guidelines, will generally tell us one of three things:
   That there is evidence in the literature for the effectiveness of that intervention; AND If this is the case the intervention will then be ranked on the quality of the available evidence.
2. That there is no evidence in the literature for the effectiveness of that intervention. It is recognised that the absence of robust evidence for an approach does not prove that the approach is ineffective - it may simply be that the evidence has not yet been collected, or there may be an effect of publication bias (in which case this is considered through the GRADE publication bias evaluation criteria). However, in an environment where resources are limited it is prudent to focus on where we can have the greatest confidence in the maximum return for our investment.
3. That there is evidence in the literature that the particular intervention is ineffective, or indeed harmful

In specialty areas where interventions are under-researched or where there is an absence of SIGN and NICE guidance, evidence will be drawn from the psychological intervention literature. The processes for this are described here:

• Rapid reviews will identify systematic reviews, meta-analyses and controlled trials.
• In the absence of any of the evidence above, practice guidance summarising ‘expert consensus’ will be considered e.g. recommendation by professional bodies or Delphi panels.
• Evidence will be considered by the representatives involved in the advisory and technical working groups for those topics along with any associated recommendations (Role of the Technical Team and Advisory Group.doc).
• This process should only be followed for areas of practice not covered by NICE or SIGN guidance.

Review of Evidence tables visual summary

Implications from the evidence tables

The Matrix evidence tables summarise the psychological therapies or interventions that have evidence of effectiveness.

• In line with previous iterations of the Matrix guidance, the implications are clear: NHS Boards should provide interventions for which there is good evidence of effectiveness; and clearly, where an intervention has been proven ineffective or harmful, it should not be provided within the NHS.
• This version of the Matrix develops the ability of services to examine and plan their delivery of psychological interventions in two important respects:
  1. Some topics have multiple psychological approaches with good evidence of effectiveness. The hexagon assessment process will help services consider other important factors that may affect the compatibility of the evidenced interventions available, and their service, by including factors such as service staffing, training implications and local needs.
  2. Where little or no evidence has been collected for the effectiveness of interventions with particular groups of people, the hexagon assessment process can help services assess the relative merits of the interventions that they use already, alongside those that are recommended within the topic as a whole. This allows some flexibility of approach as the evidence develops.

In some areas, there are longstanding services which are recognized as being of benefit to patients despite the lack of a tradition of collecting evidence in a way which would be recognised by SIGN or NICE, or in specialist populations where their small numbers mean that less research has been conducted. It is crucial that services collect their own good quality evidence around the effectiveness of any approaches they provide that are not included in the Matrix recommendations. Not only is this essential for good governance, but it has the potential to contribute to the wider evidence base and help ensure that what we invest in is effective in the longer term. The hexagon tool can be used to facilitate local strategic planning allowing identification of the relative merits and limitations of approaches offered. Services need to be able to demonstrate that they are working towards providing evidence-based services in a developmental way.

Definitions used in the tables

• WHO FOR:A description of the population groups that the evidence summary is based on, includes potential level of functioning.
• LIST OF INTERVENTIONSThe interventions are those that are recommended by guideline development groups such as NICE and SIGN and/or meta-analyses, systematic reviews and high quality controlled trials. Details of each intervention are listed in intervention templates.
• TYPE OF PSYCHOLOGICAL PRACTICE Indicates the types of staff most likely to deliver the intervention effectively. These descriptors are expanded here (add link to practice types document).
• LEVEL OF EVIDENCE This is the level of evidence that is reported in published national guidelines and describes the quality of the research on which the recommendation is based. It represents the level of confidence about the estimate of treatment effect. Where this information is not available in national guidelines the evidence is summarised in the ‘Grading the Evidence’ table below.
• LEVEL OF EFFICACY This refers to the degree of symptom/ functional improvement on key outcomes that one might expect from the intervention based on research findings. Level of efficacy is often based on metrics such as “effects size”, which describes the degree of improvement of an intervention group in relation to that of a control condition. It can be seen as a measurement of how well an intervention succeeds in achieving its’ aim. As the quality evidence for psychotherapies grows, this can help distinguish between those interventions that have small, medium and large effects. Levels of efficacy are described as low, medium or high where available. This information may not be available for all interventions at this point (indicated by N/A in the table).

Psychological Interventions

Psychological Interventions included in the Matrix must meet several criteria.  The interventions should based on a psychological theory of change, have an intended outcome of improving health or wellbeing and meet the minimum standards of evidence.  The interventions reviewed in each Matrix topic will be those that have been identified as having sufficient evidence to be listed in the evidence table (link to guidance above). Where the evidence indicates a moderate or high level of confidence in the evidence of efficacy for interventions related to a topic, interventions with low efficacy will not be included in the recommendations.

There are many interventions that improve mental health and wellbeing that are not psychological interventions i.e. psychotropic medication. There are also packages of care where a component is based on a psychological theory, but this psychological component is not sufficient as the main focus is on other components such as education, exercise, relaxation etc. The absence of these interventions in this guidance does not mean they do not have an evidence base. These interventions are beyond the scope of this guidance which is limited to psychological practices.

Psychological interventions criteria: visual summary

Intervention Review Process

To produce and rate the collection of interventions contained in the Matrix, a multi-step method was followed:

• Intervention summaries were completed based on available research findings in peer-reviewed publications, grey literature and interviews with intervention providers.
• Summaries were developed which include descriptions of both intervention indicators and implementing site indicators.
• Technical groups involving experts in the interventions reviewed the intervention summaries, editing content and rating interventions using the Hexagon tool
• Reviewers scored the three programme indicators based on the intervention summaries.
• All programme indicators were scored by two independent coders using the Hexagon Tool Guide.
• Differences in scores were resolved by discussion and a consensus decision being agreed by the review team.
• Practitioner perspectives on scores were gained through advisory groups and, where required, surveys for a sample of interventions.
• Scores and descriptions were revised where required based on practitioners’ input.

Grading the evidence

We are aware that different guidelines use different systems for grading evidence. To be included in the Matrix there should be at least one high quality meta-analysis or systematic review, or more than one randomised controlled trial of high quality aimed at the target population.

Where available, gradings in SIGN or NICE guidelines are used to complete the tables for each disorder.  Where this is not available the technical groups involved in each topic will review the evidence using the GRADE system (click here for more information)