Global statistics on poverty Dear Mr Kirby We spoke a few months ago about global poverty statistics, during the Earth Summit. I think you may be interested in the story below, which perhaps deserves some attention in the media. I realise that this is not your speciality, but perhaps you could guide me to someone who could assess the content for the BBC. There is perhaps not much more to add for the moment. My knowledge extends to many private comments from senior social scientists in academia and international organisations. There are many disturbing things about this story, including the fact that Martin Ravallion, the man in charge of the World Bank numbers for “halving poverty”, has written academic papers about flaws in poverty measurement and then subsequently used the flawed methods. Many thanks for your interest. Yours sincerely, Matt Berkley [....] ....................................................... International development scandal? The chairman of a UK parliamentary committee is to investigate a series of apparent defects in poverty statistics used by DfID. Draft Matt Berkley July 10, 2003 [Letter to Chairman of Select Committee on International Development is reproduced at the end of this draft article.] The author of the complaint, Matt Berkley, claims to have produced a new list of flaws in official statements on world poverty. These claims originate from the World Bank and have been reported through DfID. They concern the monitoring of progress towards the Millennium Goal on poverty, and other claims on which policies have helped poor people. The flaws also appear in work by a number of economists in universities, some funded directly by DfID. The flaws include: 1. Failing to adjust for the fact that changes in survival rates of hungry people influence the economic statistics in the wrong direction. 2. Failing to adjust for the cost of living, or even the general price of rice or wheat. 3. Failing to adjust for food requirements, even though proportions of children vary over time and between countries. 4. Using income data known to be unreliable, and certified as such by senior academic economists. He argues that the apparent seriousness of the flaws, viewed as a whole, has four consequences. 1. Several major claims used as the basis for international development policy are scientifically invalid. These include the World Bank claim that economic growth and trade have helped poor people. The true economic trends for poor people are unknown. 2. Macroeconomists who have failed in these areas have provided an extremely poor service to DfID and to the public. A thorough review is therefore required of DfID’s performance in poverty monitoring and in the funding of research. 3. There is a need for professional organisations of economists and statisticians to review their codes of conduct for practitioners. Misdescription of statistical results is a serious charge in social science. Professional bodies need to review whether any social scientists have done this. 4. Policymakers must, when they have fully understood the flaws, look to other methods to assess progress. A cost-effective method, says the author, is to count survival rates. Governments and international organisations measure the progress of poor people using economic statistics. The main statistic for the Millennium Goal on “halving poverty” is one example. Statistical claims from economists are also used as evidence on what has worked for the poor. Some aspects of macroeconomics are highly complex. But some of the main problems, says Mr Berkley, are easily visible to non-specialists. Substance of the complaints: further detail The author states that there are four problem areas in official economic statistics: 1. Failing to adjust for the “negative” impact on economic statistics of survival rates Simply put, if poor people die earlier, poverty is “halved” faster. This flaw appears in Millennium Goals and the World Bank/IMF Poverty Reduction Strategy Papers for poor countries. A further flaw is to ignore death as a cost to individuals. Economics is in part the science of financial costs and benefits. It has traditionally used statistics which are affected in the wrong direction by survival of poor people. But economic statistics are also used to infer how well or badly poor people have done in general or on average. That implies an assessment of human costs and benefits as well as financial ones. Traditionally, economics has ignored the human cost of dying and the benefit of staying alive. Its practitioners have produced claims as to what has been “good” or “bad” for the poor without any data on survival rates. 2. Failing to adjust for inflation in poor people’s prices Income is only part of the equation for wealth. Prices are an essential component often left out of macroeconomic analysis. Grain prices are essential for any assessment of poverty using income data. Poor people mostly depend on grain to stay alive. 3. Failing to adjust for the falling proportion of children Adults need more food than children. Birth rates have fallen in many countries. So any international “per person” poverty line set in 1990 – such as the “dollar a day” line was claimed to be - must now be too low. A fall in the birth rate results in less total food requirement, but an increase in the statistical average required for the same size of adequate meal. 4. Using income data from household surveys, certified as unreliable by senior academics These data, the Deininger and Squire dataset from staff at the World Bank, have been since 1996 the prime source of claims from macroeconomists in large-scale international poverty comparisons. Scope of the flaws Those four flaws apply to the macroeconomists’ and in particular the World Bank’s major claims in recent years concerning: 1. The effects of globalisation on poor people; 2. The effects of various government policies on poor people; 3. Millennium Goal monitoring on “halving poverty by 2015”. They also apply to the findings of other researchers in universities who have used similar methods. The mortality flaw appears in other Millennium Goals measured by statisticians: the goals on water, slums and education. The failures of the World Bank researchers appear to reflect a lack of adequate professional standards in academic economists and statisticians. Without some minimum professional standards for researchers, they are free to promote conclusions with only part of the required data. Poor value for money to the British taxpayer? A subsidiary question arises as to whether the macroeconomics profession has provided value for money to the taxpayer for work on poverty. The high cost of specialist mathematicians can be compared to the low cost of other available methods such as the measurement of survival rates. The total cost to taxpayers of flawed research is not publicly known. Expenses include those in DfID, the World Bank, the IMF, and the UN. They include expenses on researchers’ wages, administrative costs, travel expenses, publication expenses, and liaison with the media. Such expenses have come from money allocated for international development and the relief of poverty. Total cost of the researchers’ wages is not known. The total cost of the consequences to the poor of the flawed research, on which poverty policies have been based, is not knowable. A wrong policy or a wrong outcome measure can be a matter of life and death to hungry people. Additional background information 1. Accountability of the World Bank to British voters The oversight of the World Bank’s activities is the responsibility of its board of 184 governors and 184 alternate governors. The primary governors are in almost all cases finance ministers. The voting rights of large lenders - such as the US, the UK, France and Japan - are several times higher than those of small lenders. The UK’s voting rights at the World Bank are disproportionately high relative to size of its population. The UK has more influence in the running of the Bank than do the vast majority of countries. The British governors are Baroness Amos, the Secretary of State for International Development, and Gordon Brown, the Chancellor of the Exchequer. As governors, they are responsible to the British taxpayer for the effective use of aid money, and to the British voter for the provision of accurate information on statistical progress. 2. The author Matt Berkley is not an economist. His academic training was in classics and experimental psychology. He has undertaken social research in the UK on outcomes for people with mental health problems, some of whom are among the poorest in society. In 1980 he travelled and lived with destitute people in the United States. In 1983 he spent over a year in Bangladesh in order to investigate the causes of world poverty. Twenty years later, he claims that some of the causes of poverty may lie in the abuse of social science. Mr Berkley has researched, over a three-year period, detailed aspects of the statistical methods of economists in measuring global poverty. The complaints come after numerous exchanges with senior officials and academics, and a review of both the academic literature and official statistical methods. Several hundred academic articles, books and official documents were consulted. He claims to have found some significant differences between the theory and the practice. He thinks that his insights have been possible not because of his academic background but more from his experiences living and working with poor people in real life. The academic training, he thinks, may simply have served as a tool to understand the language used by economists and statisticians. This enabled him to check that his hunches were correct, and that economists had not taken certain real-life variables into account. Mr Berkley is now set to raise further questions as to, firstly, academic standards of economists funded directly from Britain’s aid budget; and secondly the total expenditure by the UK and the World Bank on flawed research. ......................................................................................................... Letter sent by electronic mail to Tony Baldry MP, 25 June 2003 Dear Mr Baldry I am writing to you in your capacity as Chairman of the Select Committee on International Development. I would like, with your permission, to raise some issues for the Select Committee to consider. The thrust of the letter is to make a case for longevity as an outcome measure for hungry people. It necessarily involves criticism of other methods. You may find the contents of this letter as shocking as they are to me. This is a short summary of some points I make in detail in other writings, and I have many supporting documents by others. I am not an economist; I undertake this task because there is a need. There are some serious flaws in economic research on global poverty. The observations below are based on discussions with a wide range of senior economists and officials, and on hundreds of official and academic documents. My own belief is that it is better to face problems head-on rather than to avoid them. By facing the problems, we can see the solutions more clearly. The first error is perhaps the most shocking. It is to fail to count survival as a better outcome. It is to use statistics on living people without counting: the cost of early death or the effects of demographic change (including changes in mortality rates) on the statistics. Economists have measured for example, average income in the poorest fifth, as an abstract segment of the economy. But that cannot tell us how poor people did. Suppose that average income in the poorest fifth rises by 1%. Traditionally, economists think that this is equivalent to “on average people in the poorest fifth had a 1% economic gain”. But the two statements are not equivalent. This is one fundamental mistake in the World Bank document “Growth is Good for the Poor”. For one thing, in a country where more poor people die early, the statistic on the poorest fifth will rise - simply because poor people did worse. I have been telling economists this for three years. I have told several senior members of staff in DFID; my own MP, Andrew Smith, who passed my message on to the then Secretary of State for International Development in 2001; the head of the World Development Indicators project at the World Bank; and several heads of department at UK universities. Prominent economists are now discussing the problem, led by Ravi Kanbur of Cornell University. I had two detailed conversations in 2001 with Professor Kanbur, in which I explained aspects of the theoretical problem to him. It is very encouraging that the United Nations Statistics Division is now considering it. There is more work to be done: Professor Kanbur appears to believe that it is adequate to measure poverty without knowing the price of food, the proportion of adults or the quality of the data (for all of which see below). The mortality problem extends to using the proportion of people in poverty as an outcome measure. This was the subject of a recent paper by Professor Kanbur. There is a serious problem with all Millennium Goals concerned with reducing the proportion of deprived people. We do not want to claim success in “reducing poverty” through early deaths. Where child survival is making bad progress and the proportion of poor people is falling fast, we might suspect some relationship between the two. But we do not know the proportion of poor people anyway from economic statistics on income (see below on food prices, and on data quality). Current statistics on income cannot tell us how many people “rose out of poverty”. Nor could any statistics even on a perfect measure of annual poverty. What is more, survival is the most important outcome for poor people. Traditional “welfare economics” does not count survival as a good outcome (except for those people on higher incomes). A second serious flaw in economic research is this. To be blunt, economists claim income gains or losses to poor people without knowing the price of food. The “income share of the poorest fifth” does not tell us, even if we know survival rates were stable, whether the poor gained or lost. In a country where poor people eat rice, you need to know the inflation rate for rice if you are going to say anything about poverty from income statistics. Economists use the overall inflation rate in a country. Why have they assumed that the price of rice changed at the same rate as non-essential goods? I have no idea. All of the World Bank’s major research conclusions (“Assessing Aid”; “Growth is Good for the Poor”; “Globalisation, Growth and Poverty”; and measures of income inequality) suffer from these flaws. To measure the outcome for the survivors is not to measure outcomes for people during the period. To measure income without knowing the price of staple food is not to measure poverty. Measures of income “inequality” - income ratios or Gini indices - falsely show more inequality if poor people live longer. The extent of this problem is unknown. Fundamentally, the change in average income does not tell us the average gain. Average income falls if the poor live longer. Average income rises if people in the majority on below-mean incomes die earlier. The poor are bad for growth. In the cases of both the mortality flaw and the inflation flaw, the poor can look as if they have done better when they have in fact done much worse, or vice versa. Martin Ravallion, who designed the methodology for the World Bank’s global poverty counts, wrote of the mortality flaw in 1996. Despite this, the World Bank has claimed that their statistics show the degree of benefit to poor people. This is not good enough in any age, but is especially alarming in the age of AIDS. Thirdly, the Select Committee should know that prominent academics do not consider the main World Bank dataset to be reliable. The main dataset for research on world poverty is the Deininger and Squire dataset of 1996. Its critics include Tony Atkinson of Oxford, who is one of the most respected authorities on inequality in the world, and James Galbraith of Texas. The criticisms are eminently sensible for prima facie reasons as well as comparisons with other data. The surveys in different countries are carried out using a wide range of different techniques, and it is unreasonable to expect poor countries’ statistical departments to have carried out rigorous and comparable surveys. After all, these are poor countries. The data are a jumble of gross income, net income and expenditure, all assessed using different methods. In the document “Growth is Good for the Poor” the authors admitted that for some data they did not know whether the income was gross or net. Martin Ravallion is on record as saying that the rich and the very poor do not cooperate with surveys, so that there are gaps in the data. Problems as to the reliability of the data for measuring the numerical values of income are in addition to the problems of making inferences from a) the numerical value of income to b) purchasing power to c) economic gains, which include changes in assets and debts. One difference between my writings and those of economists is that I refer to all the problems, rather than considering one or two at a time. (Economists are forever telling each other to adjust for children’s needs, or to get better purchasing-power parity information. But those don’t solve all the serious problems). Another difference is that I take the commonsense view that without the required data, a social scientist does not have a firm conclusion. That is not to say that social scientists should not speculate, or assume, or infer. The point is that speculation, assumption and inference should be labelled as such. The social scientist might infer a conclusion from general knowledge, but that is not the same as saying that they have calculated the answer. Oddly, it is the tradition in economics to state, for example, that there “is” a relationship between a policy and economic gains to poor people, without the required data on prices. Professor Kanbur, and the economic theorist Kenneth Arrow, have both suggested to me that the problem of food prices is not economists’ fault, because national statistical agencies do not provide those data. They are right in the sense that absence of data is not the fault of economists. However, that is neither the point which I make nor relevant to it. My observation is simply a matter of fact: the vast majority of development economists claim, without the required evidence on food prices, to know the level of economic gains to the poor. Adam Smith noted an observable difference between the inflation rate for the poor and for other people in 1776. It is somewhat surprising that two hundred years later, for all the equations and long words, economists are using more primitive methods than that. To ask around about the cost of rice and get a rough idea would be simple. To get an accurate idea would be complex. A sensible solution is to ask a few basic questions before saying what the answer is. The numbers on the faces of the coins do not tell us whether hungry people were able to eat more or less. Development experts have made similar excuses to me about data availability when I discussed the mortality flaw. In both cases, with all due respect to the experts, their thinking was the wrong way round. To think that the standard assumption is acceptable just because it is standard is not to take a scientific approach. The standard assumptions are merely conventional, with no theoretical or empirical support. I now turn to practicalities. The complexity involved in household surveys is enormous even without price surveys: income surveys can involve up to two hundred questions to a single family. In the case of prices, this second necessary task would also be complex. Those in charge of the International Comparison Programme, hosted by the World Bank to set purchasing-power parity rates, are well aware of the problems. Poor people often pay higher prices because they have to buy in small quantities and cannot buy more when prices are cheap. Further, there are no standard prices for rice: your results will depend on which markets you go to and how many people live in each place. Obtaining a price for a whole country would be complex and expensive. Suppose we had the required data on life length - would that make economic statistics meaningful measures of poverty? Maybe, if we knew about prices, age structure and extra items of necessary expenditure. But let us consider the following. a) Income statistics are not in themselves sufficient to measure poverty. b) Life length statistics are necessary. c) To some extent, life length statistics are sufficient. Amartya Sen has argued for life length as a measure of economic success. Are income statistics (adjusted for prices, age and extra items) necessary? Perhaps not. When we consider the technical difficulties in using income to measure the progress of hungry people, we may decide that measuring life length is a far cheaper, more practical, more transparent and less corruptible option. I am developing a set of minimum criteria for the UN Statistics Division. The criteria are not demanding. They include such things as these: 1) outcome statistics should take survival into account; 2) statements about progress should be based on a credible assessment of the reliability of the data, and 3) income statistics should only be used where prices of staple foods (and any changes in necessary expenditure due for example to the proportion of people in cities, or the changing proportion of adults) are estimated. It would help the effort to feed hungry people - in both the short and long terms, by whatever means are chosen - if the UK Government were to adopt some such guidelines. Otherwise, we will not know what the statistics mean. [Note: Economic statistics on “poverty” in all large-scale studies also ignore the facts that the proportion of adults is rising and adults need more food than children. The World Bank’s method for counting the global poor suffers from this flaw. Other things being equal, the Bank must have overestimated the reduction in the proportion of people living on the original level of per-day consumption considering their age and size. A dollar a day, even if we knew its value in food terms, would not be an appropriate measure where the proportion of children is changing, as now.] Current economic statistics do not meet those minimum criteria. There is a more cost-effective option. The economics of hunger is measured in years. Thank you for taking the time to read this letter. Yours sincerely Matt Berkley mattberkley@dsl.pipex.com SMTP alex.kirby@bbc.co.uk alex.kirby@bbc.co.uk SMTP Normal Normal