Dean Karlan and Jacob Appel, More Than Good Intentions: How a New Economics is Helping to Solve Global Poverty, New York: Dutton, 2011.
Book review by Dr. Dale Adams.
Knowing that Karlan is a proponent of number crunching, I thought the book would be filled with numbers, statistics, equations, tables, footnotes, and other sleep-inducing details. Instead, it is an entertaining and spirited tour through recent random controlled trials (RCT) that document the results of various poverty alleviation programs. I envy the ability of the authors to wade through the two depressing topics (poverty and industrial-strength research methods) while making the tour interesting. Their narrative is sprinkled with anecdotes to snare the interest of readers, and then the authors briefly summarize the findings of RCT studies to buttress the point they raised in the anecdote. This is certainly a lot more honest way to use anecdotes than using them to represent a population as a whole without supporting numbers to show they are representative. Visit the publications of most any NGO for examples of the latter.
About a third of the book reports on microfinance RCT-studies, both credit and savings/deposits.
The work by the nascent RCT industry is starting to filling a gaping void in developmental work. Heretofore, much of the so-called post-project evaluations of these activities were self serving, naive, badly flawed, or nonexistent. I was recently re-reminded of this void after looking through the paperwork associated with about 100 USAID-funded land-titling projects around the world that involved something well worth of a half-billion dollars in US taxpayers' money. I found only one badly-flawed ex-post evaluation of a single project in the mound of paperwork! That makes me cringe as a taxpayer. I also grit my teeth when I see the World Bank and other major donors do captured, in-house evaluations, and I see the cozy relationships between donors and their stables of consultants who do evaluations that are far from being systematic, independent, or objective. There will always be inherent problems in the close tying of the funding of development projects and the funding of evaluations, but it is a major step forward to locate most RCT research in universities that have some independence from the project funders.
Understandably, advocates of RCT see almost unlimited opportunities for their methodology, but I wonder if it can be expected to fill the entire research void? The results reported in the Karlan/Appel book suggest that RTC is more effective on some activities than on others. It is powerful in showing the effectiveness of variants of a development program, for example, various incentive packages to enhance school attendance. It is also dynamite in evaluating programs that yield relatively short-term, measurable results, for example, how de-worming kids affects school attendance. Nonetheless, it is less effective in accessing programs that yield benefits that accumulate gradually over long periods and in forms that are difficult to measure. It seems to me that most microfinance efforts yield benefits that fall into this latter category. Increased access to loans, deposits, micro-insurance, and electronic banking takes the edge of poverty and enhances the quality of life of poor people, without leaving large economic foot prints that can be captured in RCT studies that cover only a year-or-so. Karlan/Appel admit as much when, in their conclusions, they ignore microcredit in their list of seven things that work in resolving poverty. I am happy to see that savings/deposits, however, did make their list of seven. Way to go Dean and Jacob!
The work by the nascent RCT industry is starting to filling a gaping void in developmental work. Heretofore, much of the so-called post-project evaluations of these activities were self serving, naive, badly flawed, or nonexistent. I was recently re-reminded of this void after looking through the paperwork associated with about 100 USAID-funded land-titling projects around the world that involved something well worth of a half-billion dollars in US taxpayers' money. I found only one badly-flawed ex-post evaluation of a single project in the mound of paperwork! That makes me cringe as a taxpayer. I also grit my teeth when I see the World Bank and other major donors do captured, in-house evaluations, and I see the cozy relationships between donors and their stables of consultants who do evaluations that are far from being systematic, independent, or objective. There will always be inherent problems in the close tying of the funding of development projects and the funding of evaluations, but it is a major step forward to locate most RCT research in universities that have some independence from the project funders.
Understandably, advocates of RCT see almost unlimited opportunities for their methodology, but I wonder if it can be expected to fill the entire research void? The results reported in the Karlan/Appel book suggest that RTC is more effective on some activities than on others. It is powerful in showing the effectiveness of variants of a development program, for example, various incentive packages to enhance school attendance. It is also dynamite in evaluating programs that yield relatively short-term, measurable results, for example, how de-worming kids affects school attendance. Nonetheless, it is less effective in accessing programs that yield benefits that accumulate gradually over long periods and in forms that are difficult to measure. It seems to me that most microfinance efforts yield benefits that fall into this latter category. Increased access to loans, deposits, micro-insurance, and electronic banking takes the edge of poverty and enhances the quality of life of poor people, without leaving large economic foot prints that can be captured in RCT studies that cover only a year-or-so. Karlan/Appel admit as much when, in their conclusions, they ignore microcredit in their list of seven things that work in resolving poverty. I am happy to see that savings/deposits, however, did make their list of seven. Way to go Dean and Jacob!
Another issue that gnawed at me while I read the book was the uneasy feeling that the projects receiving RCT attention were mostly nibbling around the edges of the heart of poverty. The most pressing problem that all poor people have is jobs, not having more debt, getting their kids de-wormed, or using better farming techniques. Most budding micro-entrepreneurs or subsistence farmers would gladly abandon what they are doing if they had access to good steady jobs. Unfortunately, RCT studies don't shed light on this important topic. Perhaps there is a need for additional studies that more directly address why jobs are, or are not created, for the poor. (Where is Carl Liedholm and his crew at Michigan State when we need them most?) That type of information won't come from RCT studies that focus on the poor themselves, but rather from studies of the small- and medium-sized firms that provide most of the additional jobs in an expanding economy.
As an aside, I wonder what percentage of all the funds committed to development efforts are expended on serious evaluations of results? . . . Harriet Bet Your Soul.
The SME Research Initiative at IPA, launched last year, will be addressing just this question of job creation and developing new research projects in partnership with policy and practitioner stakeholders - we'd love to hear your ideas! http://poverty-action.org/sme
ReplyDelete