Social science, statistics and survival rates
22 April 2001
Email from Matt Berkley
...My project is going rather well. I've been very encouraged by the interest shown by Ravi Kanbur, economist at Cornell, in what I had to say. He was in charge of the Bank's latest World Development Report (which countries follow in their policies) until he resigned last year...... He said to me something like, "Sen looked at capabilities but maybe should have been looking at the point you're raising". This is all very odd, since I can't believe that I am saying something so fundamental about how social science has to be done if it is to give credible conclusions. The statistical effect I talk about - at root, if the poorest die, then average wealth is automatically higher even if no individual gets richer - is clearly possible in theory, and so it's only unimportant in practice if the effect can be proved to be small.
...If Cuba has extended the lives of the poor over 40 years, and child mortality among the income-poor there is drastically different from what it is in another country, this must have a significant effect on the demographics. You just can't expect - for demographic reasons and also for reasons of practical economics (helping the poor costs money) - a country to have the same per capita income if the poor survive. In Uganda, in the poorest 10% by asserts, nearly half the children born in the five years up to 1999 were dead by 1999. Experts say the children get replaced, but rates of early deaths among adults vary too.
I now think I don't necessarily have to prove anything with numbers - I can raise a serious question about the validity of any social science-based policy conclusions which ignore mortality risk.
What I now think is that economists simply haven't realised how different poor countries are from rich countries, in that in poor countries, every policy decision affects life and death, and this affects demographics, and this affects outcome indicators for the living. It's not the level of mortality that's important, it's the variability between countries, and along the dimensions of time and the chosen welfare indicator.
Anyway, if you change the demographic composition of a population you are now looking, to some extent, at different people. If you have no information about the composition of the group (including the effects of mortality rates on composition, then rises in the average of any welfare index of those living at different times tell you nothing about the progress of individuals. The assumption that the rise in the index reflects accurately enough the progress of individuals is merely an assumption. If the poorest die, you are by definition looking at the less-poor the next time you look.
In all of this, there are some problems which are statistical, some methodological, and some conceptual. Some of them are closely linked. The point about composition of groups can be expressed in terms of a logical confusion: economists talk about "the poor" in various different contexts, and sometimes mean "the poor" as a conceptual class (all those who are poor at any one time, which of course comprises, to some extent, different people at different times) while at other times they talk of "the poor" as a group of real people alive at one time. What happens over time to the average welfare of the conceptual class is not the same as the sum of welfare over time of real people who were in the group at the start. But the economists use a rise in the welfare of the conceptual class to justify policies as good for the real poor of today. This causes two big problems.
First, the data give no indication of survival rates, and so gives no idea of survival chances for the future given the same conditions or policies as the period being studied. This is in fairly sharp contrast to what we expect from research on options for ourselves in situations where survival chances are affected by choices - if a doctor gave patients advice based on statistics that ignored mortality rates, the doctor would get the sack.
Second, the average rises if the worst-off die. The extent to which this happens in practice doesn't seem to have been looked at by anyone. I now think that it's not up to someone like me to prove it has a significant effect - it is a potential problem in social science which has to qualify any conclusions. If someone wants to show that something was good for a group of real poor people, they have to show they had good grounds for believing that their measurements reflected the progress of those people during the period.
Actually this is a big can of worms, because it's not just mortality that's relevant to the difference between the conceptual group and the real group. It's demographics. The problems involved in looking at a conceptual group result from variable birth rates in families at different levels of welfare, the changing ratio of adults to children (because sometimes researchers adjust welfare indicators to fit the different needs of children and adults; and give weightings for economies of scale in different household sizes), and who replaces who in the group - the fact that someone is replaced at the same level of welfare does not mean that the average for the conceptual group will reflect the average outcome for real people.
If I am right with my basic point, different statistical effects occur with different statistical methods and groups, but the principle is the same.
Even if it were proved beyond reasonable doubt that the effects are not significant, there is another related question that needs answering. If countries are encouraged to "reduce poverty" and this means reducing the proportion of the population in poverty, what kind of outcomes for individuals are influencing the statistics? Since the internationally-agreed child mortality goal is behind in its progress, and globally the proportion of poor people is reducing according to plan, my contention is this: The further child mortality is from the goal, the more the proportion of poor people is being reduced by excess deaths among the poorest rather than by raised living standards.
Bit repetitive, but I'll send this now, because otherwise I'll keep revising it. There's actually rather a lot more I have to say about these and related subjects, both in terms of political philosophy (after Rawls, what would the average person from the West like to happen in a country if they had to swap places with a random person among the poorest? and if the spirit of the US constitution or UK law were applied to countries where people's life chances/life expectancy are very unequally distributed, what kinds of policies would result?) and in terms of social science methodology, for which I have a number of conditions I think evidence must meet in order credibly to support claims as to what has been good for the poor.