So I always struggle a little when a clinician asks me whether or not they should be using a certain treatment with their patients. Mainly because whilst I can quote the studies that evaluate that intervention, I struggle with what the results mean at an individual level.
To explain this further, let’s think of the results of a make believe RCT of a treatment that shows a small effect as compared to placebo (eg, pain levels in the treatment group are slightly lower than in the placebo group). This result tells us that when we evaluate this treatment with this certain population, on average, we don’t find that the treatment is much better than placebo. But if you are a clinician and sitting there with one individual in front of you – how do you know exactly how this person will respond? Will they be the one that, for some reason, gets a massive effect from this treatment? Will they be the one that actually gets a little worse? Would that same person get the same effect (as they got with treatment) if they got a placebo? Of course the variance of the outcomes in each group will give us some indication if different patient responses were present (ie, some people did really good and some did really bad with treatment =large variance); however these remain between group differences and don’t tell us how that exact person, say who did well with treatment, would respond if they got the placebo. Unfortunately, cross-over studies that actually look at how an individual would do with both treatments are quite tough to do, particularly in acute conditions.
This is also where stats like number needed to treat (NNT) come in. NNT gives us an idea of the number of patients you would need to treat before 1 patient received a good outcome. Whilst this gives us a general indication of how useful applying this treatment might be (eg, if NNT is big we may be a bit hesitant to use that treatment), but it still doesn’t answer the question…how will this person sitting in front of me do with this treatment? On the other hand, I find number needed to harm (NNH) a bit more helpful, because if the NNH is low, then I’m a bit worried that the chance of my patient having adverse effects with this treatment is higher. And it’s easier to say no to something that might ‘hurt’ a patient than saying no to something that just might not help them that much.
We have attempted to better predict outcome at the individual level by searching for subgroups of patients. That is, what patient factors predict who will respond best to a certain intervention? To date we’re at the baby stages of subgrouping. There are still plenty of methodological issues that are related to subgrouping that make us unsure that the ‘subgroups’ identified are really those that will do best with this type of treatment (see Ref. below for a nice summary). Also, any subgrouping (when performed properly) is specific to the comparison (ie, exercise vs placebo OR exercise A vs exercise B), so heaps of studies have to be performed before we have a good tool kit in our hands.
At the end of the day, I don’t have any better suggestions than using the mean effect of each group of an RCT to characterise how well a treatment ‘works’. However, I’m a bit hesitant to write off all treatments, particularly ones with very few side effects or chances of harming patients, when the effect sizes are small. Statements like this might bring in some heated debate (I’m ready…I think!), but I think it is fantastically important debate. To be clear, I am not for one second saying disregard the evidence (particularly evidence of no effect) and just use whatever you want to for treatment. But I am saying – let’s think a second before we completely discount potentially useful treatments (ie, treatments that demonstrate small effects). Because applying something at an individual level is a whole lot harder and more complex than considering the application of something at the population level.
Hancock MJ, Herbert RD, Maher CG. A guide to interpretation of studies investigating subgroups of responders to physical therapy interventions. Physical Therapy 2009; 89(7):698-704
If you would like to contribute, check out the “About” page and send us an email. We’d love to hear from you!
Enter your email address to subscribe to this blog and receive notifications of new posts by email.
Join 106 other followers
academic misconduct advertising Bec Bruno career Cartoon Chris W cochrane collaboration conferences cost doping evidence-based practice experiences experts fun grants hours interpreting research LBP Forum learning Leo C lifestyle Luciola C nerds networking new skills Nick H PhD placebo presentations profiles publications question of science reporting research methods research translation Sport stats Steve K study quality supervisors Tasha tips for research videos WCPT work-life workload writing Zoe M