Let’s imagine we have a Patient-Reported Outcome Measure (PROM) for surgical outcomes and we give it to patients pre- and post- operatively to see if health status improves after surgery. In common with many questionnaire measures, our PROM has reliability of 0.7.
Let’s also suppose that the surgery is equally effective for everyone, increasing health status by 3 points. In this example, that’s an effect size of d=0.36.
We calculate ‘health gain’ as the difference between pre- and post- operative score:
Health Gain = Postoperative Score – Preoperative Score
We know this should be around 3 points for each person, give or take a bit due to measurement error.
What happens when we plot health gain against preoperative score?
The correlation between preoperative score and health gain is negative and significant: r = -0.46, p<0.05: can we conclude that surgery is more effective for patients with the lowest initial health status?
NO: because we know that everyone improved by the same amount, 3 points. The correlation should be zero.
Why this happens is quite easy to explain but will have to wait until I work out how to do equations in WordPress.
UPDATE: just in case you were wondering if this sort of thing happens in real life, this graph is taken from a BMJ article on PROMs (1) in which it is concluded that “better preoperative health tends to be associated with smaller, not larger, health gains “.
1. J. Appleby, “Patient reported outcome measures: how are we feeling today?” BMJ 344, no. jan11 2 (January 11, 2012): d8191-d8191.