Wednesday, September 13, 2006

A common paradox

About 2-3 years ago I tried to learn some Statistics and got stuck in Simpson's Paradox at the very beginning. This seems to be a very common paradox which still causes problems in various arguments:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=65634
A very simple example is in Pearle's article http://singapore.cs.ucla.edu/R264.pdf
It is this: A drug tried on 30 males and 18 recovered and with no drug in a sample of 10 males 7 recovered.. So the recovery rate is 60% with drug and 70% without the drug.
The drug was tried on 10 females of whom only 2 recovered and in a sample of 30 females with no drug 9 recovered giving success rates of 20% and 30% respectively.
Now if we take the total number of both males and females for the trials with and without drug, it is 40 in both cases. With the drug, 20 out of 40 recovered giving a success rate of 50% and without the drug 16 out of 40 recovered giving a success rate of 40%.
So, even though the recovery rate for both males and females separately is 10% better without the drug, yet it seems beneficial to population as a whole.
Apparently this was first observed by the geneticist Pearson who warned about looking upon correlations as cause and effect in mixed populations.
I could not understand the rest of Pearle's article. If anybody does pl. let me know.

No comments: