Designing Trials that are Fit for Purpose

Randomised controlled trials (RCTs) are the gold standard for evaluating the relative effectiveness of interventions. However, while RCTs minimise the risk of bias (systematic errors which can be introduced into projects and favour one intervention over another) and threats to internal validity (the study is performed in a way that avoids confounding), they do not always address issues relating to external validity (the extent to which the results of a study can be generalised to other situations and to other people not involved in the trial).  An internally valid trial that has poor applicability, in a wider context, is a lost opportunity to influence clinical practice and healthcare delivery. This has led to calls to design trials with greater applicability to the real world, known as pragmatic trials.

Pragmatic trials aim to influence decisions about practice and policy, whereas explanatory trials aim to gain new knowledge and give interventions the best chance to demonstrate a beneficial effect.  Explanatory trials evaluate efficacy (whether an intervention can work under ideal conditions) and pragmatic trials assess effectiveness (whether an intervention is better than standard of care in routine settings) and often efficiency (whether an intervention extra benefits are worth any extra costs). However, while pragmatic trials are important no-one is suggesting that there should be no further explanatory trials. If an explanatory trial demonstrates that there are harms from interventions then there would be ethical justifications as to why more patients should not be exposed to this intervention in a future pragmatic trial. In addition, if an exploratory trial demonstrates that there are no benefits of an intervention in a highly selected group of patients, treated under ideal conditions, then there is no need for further trials – either pragmatic or explanatory.

There is room for both types of trial and the type selected should depend on the question that you are trying to answer. Information from an explanatory trial is unlikely to provide a satisfactory answer to a pragmatic question, and explanatory questions do not require pragmatic methods. The research question should explicitly specify the purpose of the trial and the design should fit this intention. The PRagmatic Explanatory Continuum Indicator Summary (PRECIS-2) encourages researchers to match their design decisions to their trial intention. In the BMJ paper the PRECIS-2 developers talk about designing trials which are fit for purpose, and which therefore reduce research waste.

The PRECIS-2 tool recognises that there is an explanatory-pragmatic continuum, rather than a dichotomy, with all trials falling somewhere along the spectrum. The tool identifies a number of dimensions on which trials can vary in the degree to which they are more explanatory or more pragmatic. Trials can therefore have elements that are more towards the explanatory end of the spectrum and others that are more towards the pragmatic end.

Photo of Shaun Treweek presenting a worked example of the PRECIS-2 tool at the International Clinical Trials Conference (2017) in Liverpool. Photo by Bronia Arnott (all rights reserved).

A new tool to support researchers was launched on 8th June: PragMagic, which is a decision support tool for assessing pragmatism in trial design. The tool aims to assess the true effect of an intervention in a real world setting and to support trial designers to maximise generalisability of results while also ensuring that the trial is valid and feasible to conduct.

If you would like further support in trial design you can contact the Research Design Service (RDS) North East or the RDS team in your region for those outside of the North East.


This post is by Bronia Arnott, Health Research Methodologist, Research Design Service (RDS) North East





Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s