School of Humanities and Social Inquiry

An Appeal to Aid Specialists

Keith Horton

The appeal I am making is (roughly speaking) for aid specialists to do more to help those of us who aren’t aid specialists to arrive at judgements about the effects of the work of (voluntary) aid agencies that we have at least some reason to think accurate. In §1, I say a little about why it is difficult for us to form such judgements at present. In §2, I argue that the fact that it is so difficult for us to do so has certain negative consequences, especially if the effects of (at least some of) the work aid agencies do are in fact good enough to imply that we should give to them. These negative consequences give those who are in a position to make it easier for us to arrive at such judgements strong reasons to do so. In §3, I argue that there are certain measures that aid specialists in particular are in a good position to take that would make it easier for us to arrive at such judgements. Hence my appeal to aid specialists to take such measures. In §4, I articulate and respond to a number of objections that might be made to my argument. I finish, in §5, with a brief review.

1: THE EPISTEMIC PROBLEM

I am not an aid specialist, but a moral philosopher. Like many other moral philosophers, I believe that a strong case can made for thinking that those of us who are relatively well-off, in global terms, should give some of our money to aid agencies such as Oxfam, World Vision, and CARE.1 Certainly, if the effects of the work aid agencies do are anything like as positive as their fundraising literature implies, then the case for thinking that we should give would be very strong indeed. Are the effects as positive as that, though? (‘The effects’, henceforth, is my shorthand for ‘the effects of the work aid agencies do’.) Or if it is a little naïve to hope for that, are those effects at least good enough to support a strong case for giving?

Unfortunately, it is very difficult for those of us who lack any special knowledge or experience of international aid to say.2 On the one hand, there are a number of strong presumptive reasons – stemming from the extreme poverty of many of the people such agencies work with, for example, and the cheapness (by our standards) of the goods and services many of them lack – for thinking that the effects might be very positive indeed. On the other, though, the literature on voluntary aid agencies that has emerged over the last twenty years or so highlights a number of concerns about such agencies and the work they do that at least make one think again.3 Given space constraints, I won’t attempt to summarise those concerns here, but I will briefly indicate a few of them for purposes of illustration.

Some of those concerns relate relatively directly to the effects of the kinds of work aid agencies do. Thus at least some of those kinds of work can apparently give rise to significant unintended negative effects. Humanitarian aid, for example, can help to fuel a war economy,4 while service delivery can undermine the very forces necessary to put the poor in a position to meet their own needs.5 And it appears that little is known at all about the effects of certain other types of aid, such as capacity-building.6 Other concerns relate more directly to aid agencies as organisations. Many NGO specialists accuse such agencies of not investing sufficiently in evaluation and research, for example (see e.g. Smillie 1999: 21-31, Davies 2001, Riddell 2007: 265f); of failing to learn from experience, and consequently repeating the same errors again and again (Edwards 1999: 82-5, Minear 2002: 46f, Terry 2003: 224f, Riddell 2007: 346); of indulging in various forms of dishonesty (Fowler and Biekart 1996: 111, Smillie 1995: 152, de Waal 1997: 144, Smillie 1999: 11-14, 28, Darcy 2005: 3, Rieff 2005: 3-4, Riddell 2007: 278-80); of failing to cooperate with one another or coordinate their activities (Clark 1991: 65, Smillie 1995: 233, Edwards 1999: 105, Biekart 1999: 299, Minear 2002: ch. 2, Riddell 2007: 338-9); and more broadly of giving more attention to ‘institutional’ ends (such as increasing ‘market share’) than to ‘developmental’ ends (such as alleviating poverty) (Edwards 1996, Biekart 1999: 77-92, Dichter 2003, ch. 6; cf. de Waal 1997: 66).

Given such concerns, it is not clear whether it is reasonable simply to assume that the effects of the work aid agencies do are good enough to imply that we should give to such agencies, despite the strong presumptive reasons for thinking that those effects may be very positive. And to make matters even worse, there appears to be a lack of the kind of probative data on the effects that might help to resolve the issue one way or another. As noted above, NGOs apparently tend to give relatively little attention to attempting to measure the effects of their own work. Further, they tend not to publish those evaluations of their work that they do commission or conduct, or indeed any other reliable data on the effects of their activities. 7 And there is no independent body charged with evaluating their activities and reporting back to the public on their findings.8 Furthermore, it appears that many of the evaluations that have been conducted are of very poor quality. According to the authors of the most comprehensive review of evaluations of NGO activities to date, for example (Kruse et al. 1997: §9.4):

    a repeated and consistent conclusion drawn across countries and in relation to all clusters of studies is that the data are exceptionally poor. There is a paucity of data and information from which to draw conclusions about the impact of projects, about efficiency and effectiveness, about sustainability, the gender and environmental impact of projects and their contribution to strengthening democratic forces, institutions and organizations and building civil society. There is even less firm data with which to assess the impact of NGO development interventions beyond discrete projects, not least those involved in building and strengthening institutional capacity. . . .9

Naturally, a lot more could be said about these issues. For now, though, I hope I have said enough to support the claim that it is very difficult for ordinary people – people with no special knowledge of or expertise in international aid – to find out how good or bad the effects of the work aid agencies do are.10 And I will call the fact that it is so difficult for us to do so the ‘Epistemic Problem (for potential contributors to aid agencies)’. Next, I will explain why I think this phenomenon is a problem.

2: WHY THE EPISTEMIC PROBLEM IS A PROBLEM

Why is the fact that it is difficult for us to find out how good or bad the effects are a problem? In part, because it follows that it is also difficult for us to determine whether or not we should give to such agencies. For clearly, whether or not we should give to aid agencies depends on how good or bad the effects of their work are. It is easiest to see this if one focuses on the worst-case scenario – that aid agencies do more harm than good. No one, I take it, would think that we should give to aid agencies if this were so. The effects must have to come up to some standard, then – must be ‘good enough’, as I shall put it – in order for it to be the case that we should give. This isn’t to assume that whether we should to aid agencies depends solely on how good or bad the effects of their work are. Some would endorse that assumption, while others would deny it, arguing that other factors need to be taken into account, in addition to the effects. All would agree that the effects are at least one thing that we need to take into account, though. And all would agree that we should not give to aid agencies if they did more harm than good, and consequently that the effects have to come up to some standard in order for it to be the case that we should give.11

Determining how high that standard should be is likely to require considering a number of complex issues. Some accommodation may need to be made for the fact that achieving positive effects is easier in some environments than in others, for example, and for the fact that some aid activities that have disappointing results may bring indirect benefits, such as the opportunity to learn more about what works and what doesn’t and thus to improve performance in the future. How good one thinks the effects need to be in order to be ‘good enough’ is also likely to depend on what moral theory or (less formally) moral outlook one accepts.

Determining how good the effects have to be in order to be ‘good enough’ (to imply that we should give), then, raises many complex and controversial questions. Fortunately, though, I do not need to resolve those questions in order to tackle the issue that is the main focus of this paper. For whatever standard the effects have to reach in order to be ‘good enough’, the fact that it is difficult to find out how good or bad the effects are means that it will also be difficult to establish whether or not the effects reach that standard. Whatever that standard is, then, one will still face the Epistemic Problem and the difficulties it gives rise to.12

Of course, it would be unrealistic to expect any very precise judgement about the effects. International aid is a complex business, and any such judgements are therefore likely to have to be very rough and probabilistic. And for the same reason it would also be unrealistic to expect anything approaching certainty that any such estimate is correct. If we had literally no idea at all about how good or bad the effects were, though, then we would also have no idea about whether we should give to such agencies, given that the answer to the second question depends on the answer to the first. In order to make a minimally well-informed decision about whether or not to give to aid agencies, then, we do need to form at least some sort of rough estimate about the effects that we have at least some good reason to believe accurate.13

The Epistemic Problem, then, makes it is difficult for us to determine whether or not we should give to aid agencies. And this is awkward, given that we cannot avoid the choice of whether to give to such agencies or not. This, then, is one reason why Epistemic Problem is a problem. As long as it is seriously unclear whether we should give to aid agencies, moreover, we are unlikely to give to such agencies, or at least to give much. For in view of the sacrifices that doing so would impose, and the variety of other ‘good causes’ that one might support, we are likely to hold back unless we are fairly confident that the effects are good enough to imply a moral obligation to give. Whether this is a good or a bad thing depends, of course, on whether the effects are in fact good enough. If they are not, then it may be no disaster (indeed, it may be a good thing) if the difficulty of finding out whether the effects are good enough leads us not to give. If the effects are good enough, though, matters would be much more serious. For in that case, the difficulty of finding out how good the effects are would result in our failing to do what we should do (by hypothesis, give to aid agencies). This would be bad from our point of view, at least if we care whether or not we do what we should do. And much more importantly, it would also be bad from the point of view of the global poor.14 For in order to be good enough, the effects would presumably have to be at least generally positive. And if the effects were generally positive, then increased donations to aid agencies would presumably benefit the global poor.15

Of course, some people would not give even if it were as clear as could be that the effects were good enough.16 As long as there are some people who would give (or give more) if it were easier to find out that the effects were good enough, though, aid agencies would receive more money. And it seems hard to deny that there are some people of this sort. The most common reasons people give for not giving to aid agencies seem to be worries of one sort or another about the effects.17 In some cases, no doubt, such reasons function merely as excuses, but it would seem skeptical to the point of cynicism to suggest that is so in all cases. If it were easier to find out that the effects were good enough, then, it would be easier to see that such worries do not undermine the case for giving. And if it were easier to see this, then some people who currently do not give because of such worries would start to give. Further, some of those who already give to such agencies may give more.18

It seems likely, then, that donations to NGOs would go up if it were easier to find out that the effects were good enough. And as I said above, this would benefit the global poor, given the assumption that the effects are good enough. Given that assumption, then, the Epistemic Problem is a very bad thing indeed, for it prevents the global poor from getting the benefits they would be getting if that problem were less acute – if it were easier to find out that the effects are good enough.

Is that assumption true, though? Are the effects good enough? As a non-expert it is of course difficult for me to say – that is just one instantiation of the Epistemic Problem. For what it’s worth, though, I suspect that they are. I will not try to support this claim here, however.19 Instead, I will just put the point conditionally: if the effects are good enough, then the Epistemic Problem is a very bad thing indeed, for it results in the global poor being worse-off than they would be if that problem were less acute, on the assumption that donations would increase if it were easier to find out that the effects are good enough.20

The Epistemic Problem, then, makes it difficult for us to find out whether or not we should give to aid agencies. This is a bad thing whether the effects are good enough of not – all would agree, I take it, that it would be better if it were easier for us to find out whether we should give to aid agencies or not. If the effects are good enough, moreover – either the effects as a whole, or some identifiable subset of those activities – then the Epistemic Problem also results in aid agencies getting less money than they would otherwise get, given the plausible assumption that people would give more to such agencies if it were not so difficult to find out that the effects of their work were good enough. And this, in turn, deprives the global poor of benefits that they would be getting if it were not so difficult to find out that the effects are good enough.

Given these negative consequences, those who are in a position to make it easier for us to find out how good or bad the effects are have strong reasons to do so. And as I argue in the next section, aid specialists appear to be in a position to do so.

3: ARE AID SPECIALISTS IN A POSITION TO HELP?

When one faces an important decision that involves a complicated subject matter in which one lacks expertise, the normal response is to turn to specialists in that area for guidance – if possible, specialists who do not occupy roles that are likely to bias them or deter them from giving frank and fearless advice. In the case of aid agencies, then, the natural thing is to turn to those I am calling ‘aid specialists’ – academics, consultants, and others who spend their working lives studying and working with such agencies.21

If one does so, though, one is likely to be disappointed. Though there is now quite a substantial literature on NGO aid, that literature is of limited use in helping potential contributors find out how good or bad the effects are. For one thing, much of that literature is aimed mainly or entirely at other aid specialists rather than the general public,22 and as a consequence is not always easy for non-specialists to understand.23 Perhaps because little of that literature is aimed at the general public, moreover, it does not tend to focus on the issues that matter most to potential contributors, such as how good or bad the effects are, whether they are good enough to imply that we should give to such agencies, and (particularly if it is difficult to answer the previous question, or the answer to it is negative) whether there are any particular agencies or types of aid that one can be relatively confident have good enough effects.

Of course, as I pointed out in §1 there is apparently a lack of the kind of reliable data that would enable aid specialists to provide confident judgements about these matters. It doesn’t follow that there is nothing aid specialists could do that would make it easier for us to find out how good or bad the effects are, though. Indeed, aid specialists themselves sometimes write about these issues in ways that imply that they think more could be done. In 1996, for example, Alan Fowler and Kees Biekart wrote that ‘a consumer’s guide to agencies, produced by an independent entity, is long overdue’ (Fowler and Biekart 1996: 131). Absent such a guide, they continue, one is left with ‘a bizarre situation of agencies competing in a donor market-place where buyers cannot reasonably compare the products on offer for their relative value for money; they must still choose by an act of faith’ (ibid.). In the intervening years, however, the situation has not substantially changed.24

What measures could aid specialists take, then, that might make it easier for those of us who aren’t aid specialists to find out how good or bad the effects are? One obvious step would be to set up a body staffed by aid specialists charged specifically with giving guidance to potential contributors about NGOs and their work, and especially the effects of that work. Such a body would ideally be genuinely independent of NGOs, and (as far as possible) free from any other pressures that might distract it from the task of providing clear and honest advice. One could imagine more or less ambitious versions of such a body. The most ambitious would be one with sufficient resources to do its own evaluations of NGOs and their activities. Calls have been made from time to time for such a body to be set up,25 but to date without much effect. One major issue is presumably funding, as of course it would not be cheap to conduct such evaluations. And getting permission from NGOs to evaluate their activities and report back to the public on their findings may also raise certain problems.

It is not clear to me whether these problems could be overcome. Even if not, though – and at least in the meantime, until such a body exists – one might consider

less ambitious alternatives. One such alternative would be a body with a similar remit of reporting to the general public about the work of NGOs, but which restricted itself to data that was already available, rather than conducting its own evaluations. Such a body would of course be a great deal cheaper to fund than one that also conducted evaluations, and at least some of the data in question would be in the public domain and thus not raise any problems of access. Could such a body make a significant impact on the Epistemic Problem? That would depend on the degree to which there is already enough reliable data on the work aid agencies do (whether publicly available or not) to enable such a body to come to well-based judgements about the effects of the work they do. To the degree to which there is enough data, we would already be tantalisingly close to a solution to the Epistemic Problem. All that would be necessary is for a suitable group of aid specialists to get access to that data, draw relevant conclusions based on it, and then pass on those conclusions to the general public.

Is there already enough data of this kind? As an outsider it is difficult for me to judge. At the time the DAC Study was produced (in 1997), the authors evidently didn’t feel that they had sufficient reliable data to make any judgement about the effects of the development work NGOs do as a whole. And though more attention has been given to evaluation since then, it still seems unlikely that there is currently sufficient data for such a body to make such a judgement with any confidence. Even if this is right, though, there might be sufficient data for such a body to form estimates about the effects of the work of some aid agencies. There might be some NGOs which ensure that their own activities are evaluated with sufficient rigour and objectivity to allow those who know how to interpret such data to make reasonably reliable estimates of their effects, for example. If so, then in such cases an independent body would merely have to play the role of an honest broker, interpreting the data in question and reassuring the public of its reliability. And similarly, there might already be enough data about certain types of aid for such a body to offer reasonably confident judgements about their effects.2627

As I pointed out in §2, even such partial information might be of decisive import to potential contributors, if such a body were able to report with a reasonable amount of confidence that at least certain sets of aid activities had effects that were good enough. For then potential contributors could be confident that they should fund such activities, whether or not there is a moral obligation to fund NGO aid in general. And this would solve the key practical question they face: ‘Is there something in this area that it is relatively clear I should do?’28

The kinds of bodies I have been discussing so far would have an ongoing existence. Less ambitiously still, one might set up a commission to perform some of the same functions on a one-off basis, rather like the DAC Study, but this time aimed primarily at the general public, and focused especially on establishing whether there are any aid agencies or kinds of aid that can be said with some confidence to have good enough effects.

Finally, and most simply of all, aid specialists could simply offer their own personal judgements about the effects, based on their experience and their reading of the relevant data – again, either judgements about the effects of NGO aid in general, or about the effects of particular types of aid or of the work of particular aid agencies. Potential contributors may not be able to rely on such judgements as confidently as they could rely on the findings of the kinds of bodies considered above. For while it would presumably be clear that such findings were based on a thorough, impartial review of the available evidence, it would be harder to determine how this or that specialist had arrived at their judgements. In the absence of any such body, however, such informal judgements made by aid specialists might be a lot better than nothing.29 Indeed, if such judgements were sufficiently positive, and were taken to be sufficiently reliable, then they might again be decisive for certain potential contributors, willing to give only if they receive sufficient assurance about the effects.

Currently, though, few aid specialists seem willing to provide such estimates, at least in print. Some aid specialists do address the question of how good or bad the effects are. Those who do so tend to focus mainly on emphasising the lack of reliable data, though, and the consequent difficulty of making such an estimate.30 They don’t then go on to ask the next questions that arise next for potential contributors: ‘Given this lack of reliable data, what is the most reasonable assumption to make about the effects? If one really can’t make any estimate, no matter how rough or qualified, about the effects of NGO aid as a whole, are there at least certain types of aid or aid agencies that one can make more confident judgements about? Are there, in particular, any activities that one can say with at least some confidence have effects that are good enough?’

Of course, aid specialists too may find it very difficult to make such judgements with any degree of confidence, given the lack of reliable data on the effects. However difficult they might find it to make such judgements, though, it would still be less difficult for them to do so than for those of us who are not aid specialists. Given that we need such judgements in order to determine whether or not we should give to aid agencies, then, it would still appear to be very helpful if they told us – tentatively, perhaps, with whatever qualifications are necessary – what they think.

The issues discussed in this section are complex ones, and there is of course a lot more to say about them than I have had space for here.31 If what I have said is even roughly on track, though, it does appear that there are measures that aid specialists could take that would make it easier for those of us who aren’t aid specialists to find out how good or bad the effects are.32 And if what I said in §2 was correct, aid specialists have strong reasons to take such measures. At the very least, doing so would be helpful to those of us who are struggling to determine whether or not we should give to aid agencies. If the effects of at least certain types of aid are good enough, moreover, getting that message out seems likely to lead to higher donations, which would benefit the global poor. Hence my appeal to aid specialists to take such measures.

I finish by articulating and responding to some objections to my argument.33

4: OBJECTIONS

I begin with an objection to the suggestion made in §3 that aid specialists provide judgements or estimates about the effects of the work aid agencies do. Given the complexity of the environments in which aid agencies work and the variability of the effects in different circumstances, one might point out, no such judgement could begin to capture the complex reality of NGO aid and its effects. And for this reason, one might suggest, any such judgement would have to falsify that reality to a greater or lesser extent, and is therefore likely to be more misleading than illuminating. And if that is right, then it would not be helpful for aid specialists to provide such judgements after all.34

There is clearly something in this line of thought. Clearly, no such judgement about the effects could do justice to the complex reality of NGO aid. I do not think it follows that it would not be helpful for aid specialists to supply such judgements, though. There are a number of reasons for this. For one thing, there is the point made in §3 that such judgements might concern a particular type of aid or the work of a particular aid agency, rather than NGO aid in general. Furthermore, the kind of judgements I am thinking of need not be very short and simple. There is nothing to rule out extended judgements that include many qualifications and nuances (in the kind of way that certain legal judgements do). And it seems implausible that no extended judgement about any type or source of aid could avoid being more misleading than illuminating. Admittedly, even judgements of these kinds would not be able to capture the full reality of the phenomena they describe. They might at least provide a more complete and accurate picture than those of us who are not aid experts are currently able to piece together, though, and this might be helpful to us when we try to determine whether to give to aid agencies or not.

In addition, it is not clear to me that one need entirely rule out short and simple judgements, or judgements about NGO aid as a whole, or even short and simple judgements about NGO aid as a whole. A judgement that the effects of NGO aid as a whole were or were not good enough would be very short, for one thing, and simple at least in the sense that one could state it in a few words. In arriving at such a judgement, though, one may take account of any number of complex factors. And such a judgement would be ideal for those who seek to determine whether we should give to aid agencies.35 More broadly, it is important to keep in mind the particular aim that I have highlighted here for providing judgements about the effects of the work aid agencies do. That aim is not to reflect or illuminate the world of aid in all its complexity. (Such an aim may be a perfectly worthy one, of course – it is just not the one at issue here.) That aim is rather to help potential contributors to aid agencies determine whether or not they should give to such agencies. And for this, a certain amount of simplification may be no bad thing. Indeed, a certain amount of simplification may be helpful or even necessary, for it may be very difficult for ordinary people who have limited amounts of time to devote to these matters to digest too much complex information.

For these reasons, it does not seem to me that any judgements or estimates about the effects that aid specialists might provide would have to be more misleading than illuminating. It is worth adding, moreover, that even if I am wrong about this, it wouldn’t undermine my claim that aid specialists are in a position to take measures that would make it easier for us to find out how good or bad the effects are. At most, it would just follow that providing estimates of the effects aren’t one of those measures. As long as there are other measures that would help, then, that claim would stand. And given the scalar nature of that claim (that aid specialists are able to make it easier for us to find out how good or bad the effects are, not necessarily easy) it seems hard to deny. For it would be rather extraordinary if there were nothing aid specialists could do, either individually or collectively, that would make it even a little easier for those of us who aren’t aid specialists to get even a slightly better sense of how good or bad the effects might be.

Next, I consider two reasons to question the claim made in §2 that making it easier to find out that the effects are good enough (assuming that the effects of at least certain types of aid are good enough) would lead to an increase in donations to aid agencies. One such reason stems from the view that those who give to NGOs do so mostly for self-regarding reasons – in order to feel good, for example, or to assuage guilt – and therefore tend not to be much interested in the effects of the work aid agencies do.36 If this is right, one might suggest, making it easier to find out that the effects are good enough may not lead to much of an increase in donations.

This line of argument does not seem to me to be persuasive. For one thing, giving to aid agencies would presumably not make one feel good (or be effective in assuaging guilt) unless one believed that their work had generally good effects. If it were easier to see that this claim were true, then, more people would believe it, and thus more people would be in a position to feel good (or assuage guilt) by giving to such agencies. If it were easier to find out that the effects were good enough, then, it seems likely that more people who are responsive to such motives would give. Even if such motives are very common, moreover, it seems reasonable to suppose that there is nonetheless at least a significant minority of potential contributors who are more directly concerned about the global poor. And if so, these people at least would be more likely to give, or to give more, if it were easier to find out that the effects were good enough.

A related and potentially more troubling worry is this. Any attempt to give a truthful overview of the effects of the kinds of work aid agencies do would be likely to reveal more about the negative aspects of aid than such agencies tend to admit to publicly.37 And for a number of reasons, this might lead some people to stop giving, or to give less, even if such an overview supported the view that the effects are good enough, all things considered. Some people might interpret the information incorrectly, for one thing, because of honest errors or the influence of bias.38 And even those who interpret the information correctly may find that any negative information tends to loom larger in their minds than the positive information, which might disincline them from giving. The media might tend to focus disproportionately on any negative information, moreover, which would help to foster an unfairly negative view of the work NGOs do.39 And similarly, organisations ideologically opposed to NGOs might pounce on any negative information and use it in their campaigns against NGOs.

These concerns seem to me to be serious ones. Such factors may indeed lead some people to stop giving to aid agencies, or to give less, even if the information published supports the view that the effects are good enough, all things considered. On balance, though, these concerns do not seem to me to undermine the case for providing more reliable information about the effects. For one thing, it remains the case that (for the reasons given in §2) some people are likely to give more as a result of such information being published, if that information supports the claim that the effects are good enough. And I suspect that the increase in donations resulting from this is likely to outweigh any reductions in donations resulting from the factors cited in the preceding paragraph. After all, quite a lot of negative claims about NGO aid are published in books or the media currently – some of them much more negative than (I take it) a fair-minded, balanced survey would provide. And yet many people continue to give to aid agencies. This suggests that at least a significant number of people understand that aid isn’t as straightforward as the fundraising material aid agencies typically publish tends to imply, and are still willing to fund such agencies.

Furthermore, providing more reliable information about the effects would have other positive aspects, whatever its effect on the overall level of donations. Doing so would make it easier for potential contributors to determine what they should do, which is a good thing, even if some would fail to make appropriate use of that opportunity. Openness and honesty presumably have some value in themselves, moreover, whatever their effects. Providing more accurate information about the effects might also lead some people to change their giving practices in positive ways, for example by giving more to the activities or agencies that have better effects, and less to the ones that have worse effects. And this would lead to better outcomes for the global poor even if the overall level of donations didn’t go up.

Though the objections discussed in this section raise genuine concerns, then, none seem to me to undermine my main claims. It still appears that there are measures that aid specialists could take that would make it easier for those of us who aren’t aid specialists to find out how good or bad the effects are, and that providing estimates of the effects is one of those measures. And it still appears that aid specialists have strong reasons to take such measures, especially if the effects of at least certain types of aid are good enough.

5: REVIEW

The questions I have discussed in this paper raise many complex issues, and there is of course much more to say about them than I have found space for here. I hope that I have said enough, however, to support my appeal for aid specialists to do more to help those of us who aren’t aid specialists to find out how good or bad the effects of the work aid agencies do are. Doing so would be helpful to those of us who are struggling to determine whether or not we should give to aid agencies. And if the effects of at least certain types of aid are good enough, doing so also seems likely to lead to higher donations, which would benefit the global poor. Given the importance of these ends, I suggest, aid specialists should as a matter of urgency do what they can to make it easier for us to find out how good or bad the effects are.

References

Addison, Tony (2000): ‘Aid and Conflict’, in Finn Tarp and Peter Hjertholm (eds), Foreign Aid and Development: Lessons Learnt and Directions for the Future (Routledge)

Aiken, William and LaFollette, Hugh (eds) (1977): World Hunger and Moral Obligation (Upper Saddle River, NJ: Prentice-Hall)

Aiken, William and LaFollette, Hugh (eds) (1995): World Hunger and Morality (Upper Saddle River, NJ: Prentice-Hall)

Baker, Judy L (2000): ‘Evaluating the Impact of Development Projects on Poverty’ (World Bank: Washington D.C.)

Banerjee, Abhijit V. and He, Ruimin (2003): ‘Making Aid Work’ mimeo, MIT, reprinted in Easterly 2008

Biekart, K (1999): The Politics of Civil Society Building (International Books and Transnational Institute, Utrecht/Amsterdam)

Black, Maggie (2002): No Nonsense Guide To Development (Verso Press)

Chatterjee, Deen K. (ed.) (2004): The Ethics of Assistance: Morality and the Distant Needy (Cambridge University Press)

Clark, John (1991): Democratizing Development: The Role of Voluntary Organizations (London: Earthscan)

Collins, C (1998): ‘Critiques of Humanitarianism and Humanitarian Action’, in Humanitarian Coordination: Lessons Learned (Office for the coordination of Humanitarian Affairs: New York): 12-26

Crocker, David (1996) ‘Hunger, Capability and Development’, in William Aiken & Hugh Lafollette (eds.), World Hunger and Moral Obligation, Prentice-Hall, place, reprinted in Hugh Lafollette (ed.) Ethics in Practice, Routledge, 2nd Edition

Cullity, Garrett (2004): The Moral Demands of Affluence (Oxford University Press 2004)

Darcy, James (2005): ‘Acts of Faith? Thoughts on the effectiveness of humanitarian action, (Overseas Development Institute Discussion Paper, at http://www.odi.org.uk/HPG/papers/Aid_effectiveness.pdf)

Davies, R (2001): ‘Monitoring and Evaluating NGO Achievements’, at http://www.mande.co.uk/docs/arnold.htm

De Waal, Alex (1997): Famine Crimes: Politics and the Disaster Relief Industry in Africa (London: African Rights and the International African Institute)

Dichter, Thomas (2003): Despite Good Intentions (University of Massachusetts Press)

Dower, Nigel (1983): World Poverty: Challenge and Response (William Sessions Limited)

Duflo, Esther and Kremer, Michael (2005): ‘Use of Randomization in the Evaluation of Development Effectiveness’, in Pitman, George Keith; Feinstein, Osvaldo N.; and Ingram, Gregory K. (eds), Evaluating Development Effectiveness (New Brunswick, New Jersey, and London, U.K.: Transaction Publishers), reprinted in Easterly 2008

Edwards, Michael (1996): ‘International Development NGOs: Legitimacy, Accountability, Regulation and Roles’, in Meeting the Challenge of Change: the report of the Commission on the Future of the Voluntary Sector. London: Commission on the Future of the Voluntary Sector

Edwards, Michael (1999): Future Positive (Earthscan, London)

Edwards, Michael And Hulme, David (1996): ‘Introduction’ to Michael Edwards and David Hulme (eds), Beyond the Magic Bullet: NGO Performance and Accountability in the Post-Cold War World (Kumarian Press)

Edwards, Michael And Hulme, David (1996c): ‘Beyond the Magic Bullet? Lessons and Conclusions’, Edwards and Hulme 1996a

Escobar, Artoro (1994): Encountering Development (Princeton University Press)

Fowler, Alan (2000): ‘Civil Society, NGDOs and Social Development: Changing the Rules of the Game’ (Occasional Paper, Geneva: UNRISD), at http://www.unrisd.org/unrisd/website/document.nsf/0/F553495F06F98DCE80256B5E005C9DDC?OpenDocument

Fowler, Alan and Biekart, Kees (1996): ‘Do Private Aid Agencies Really Make a Difference?’, in Sogge 1996

Horton, Keith (2004): ‘Aid and Bias’, Inquiry 47: 545-61

Horton, Keith (forthcoming): ‘The Epistemic Problem: Potential Solutions’, in Horton and Roche (forthcoming)

Horton, Keith and Roche, Chris (eds) (forthcoming): Ethical Questions and International NGOs (Springer)

Jordan, Lisa and van Tujil, Peter (2006): NGO Accountability: Politics, Principles and Innovations (Earthscan)

Korten, David (1987): ‘Third Generation NGO Strategies: A Key to People-Centred Development’, World Development 15 (Supplement): 145-59

Korten, David (1990): Getting to the 21st Century: Voluntary Action and the Global Agenda (West Hartford: Kumarian Press)

Kruse, Sten-Erik; Kyllönen, Timo; Ojanperä, Satu; Riddell, Roger; and Vielajus, Jean (1997): Searching for Impact and Methods: NGO Evaluation Synthesis Study (Finnish Ministry of Foreign Affairs, Helsinki), at http://www.valt.helsinki.fi/ids/ngo/

Lewis, David (2007): The Management of Nongovernmental Organizations (London: Routledge, 2nd edition)

Lindenberg, Marc and Bryant, Coralie (2001): Going Global: Transforming Relief and Development NGOs (Kumarian Press)

Maren, Michael (1997): The Road to Hell: The Ravaging Effects of Foreign Aid and International Charity (New York: Free Press)

Minear, Larry (2002): The Humanitarian Enterprise: Dilemmas and Discoveries (Kumarian Press)

O’neill, Onora (1985): Faces of Hunger (Allen and Urwin)

Pogge, Thomas (2002): World Poverty and Human Rights: Cosmopolitan Responsibilities and Reforms (Polity)

Riddell, Roger (2007): Does Aid Really Work? (Oxford University Press)

Rieff, David (1995): ‘The Humanitarian Trap’, World Policy Journal 12 (4): 1-11

Roche, Chris (forthcoming): ‘The Seeming Simplicity of Measurement’, in Horton and Roche (forthcoming)

Singer, Peter (1972): ‘Famine, Affluence, and Morality’, Philosophy and Public Affairs 1 (3): 229-243

Singer, Peter (1993): Practical Ethics (Cambridge University Press, second edition)

Singer, Peter (2009) The Life you can Save (Yale University Press)

Sinnott-Armstrong, Walter (2006): ‘Consequentialism’, in the Stanford Encylopedia of Philosophy, at http://plato.stanford.edu/entries/consequentialism/

Smillie, Ian (1995): The Alms Bazaar: Altruism Under Fire – Non-Profit Organizations and International Development (London: IT Publications)

Smillie, Ian (1999): ‘At Sea in a Sieve?’ in Ian Smillie and H. Helmich, Stakeholders: Government-NGO Partnerships for International Development (Paris: OECD)

Sogge, David (ed.) (1996): Compassion and Calculation: The Business of Private Aid Agencies (London: Pluto Press)

Temkin, Larry (2004a): ‘Thinking about the Needy, Justice, and International Organizations’, Journal of Ethics 8: 349-395

Temkin, Larry (2004b): ‘Thinking about the Needy: A Reprise," Journal of Ethics 8: 409-458

Tendler, J (1982): Turning Private Voluntary Organizations into Development Agencies: Questions for Evaluation (USAID Programme Evaluation Discussion Paper No. 12, Washington, D.C.)

Terry, Fiona (2003) Condemned to Repeat? The Paradox of Humanitarian Action (Cornell University Press)

Unger, Peter (1996): Living High and Letting Die (Oxford University Press)

Weiss, Thomas G (1999): ‘Principles, Politics, and Humanitarian Action’, Ethics & International Affairs 13: 1-22

1

Acknowledgements

I’d like to thank Emma Rooksby and an anonymous referee at Development Policy Review for helpful written comments on an earlier draft of this paper, and Chris Roche for discussion about related issues. All remaining errors and shortcomings are of course entirely my own responsibility.

In this paper, ‘aid agencies’ refers to Non-Governmental Organisations (NGOs) working in the field of international aid, taking ‘aid’ broadly to include both development aid and humanitarian aid. I shall also sometimes refer to them simply as ‘NGOs’. For work by philosophers on this issue, see e.g. Singer 1972, 1993: ch. 10, and 2009, Aiken and LaFollette 1977 and 1996, Dower 1983, O’Neill 1985, Unger 1996, Pogge 2002, Chatterjee 2004, Cullity 2004, and Temkin 2004a and b.

2 Henceforth, when I use the terms ‘us’ or ‘we’ I will mean people who are in a position to give to aid agencies – that is, those who are relatively well-off by global standards.

3 There has been an extensive literature on official aid (aid given by governments) for several decades, but a similar literature on NGOs only began to emerge in the 1980s (see e.g. Tendler 1982 and Korten 1987), and didn’t really develop momentum until the 1990s. I give some references immediately below.

4 For brief overviews of humanitarian aid that discuss such concerns, see e.g. Collins 1998, Edwards 1999: ch. 5, Weiss 1999, Lindenberg and Bryant 2001: ch. 3, Addison 2000, and Riddell 2007: chs 18-19. For more detailed studies, see de Waal 1997 and Terry 2003.

5 For discussion of some of the problems with service delivery and other forms of development aid, see e.g. Korten 1990, chs 9-10, Clark 1991, Smillie 1995, Sogge 1996, Edwards 1999, Fowler 2000, Lewis 2007: ch. 3, and Riddell 2007: chs 16-17.

6 According to Roger Riddell, ‘of all the different ways that NGO aid is allocated and spent, least is known about the overall impact of capacity-building initiatives’ (Riddell 2007: 283). Indeed, he continues, it ‘is extremely rare for NGOs separately to assess even the direct impact of such activities, never mind the wider effects on the lives of beneficiaries’ (ibid.).

7 As Michael Edwards and David Hulme put it, ‘Internal evaluations are rarely released, and what is released comes closer to propaganda than rigorous assessment’ (Edwards and Hulme 1996: 6). In recent years, some NGOs (such as Oxfam, CARE, and BRAC) have begun making more evaluation material of one sort or another public (see the relevant websites). The great majority of NGOs still do not publish any such material, however. It is often hard for the non-expert to interpret the material that is published, moreover, and to tell how reliable it is, and so even such initiatives are of limited use in enabling non-experts to find out how good or bad the effects are.

8 Again, in recent years a variety of organisations have been created that in one way or another provide information about NGOs and their work. Examples include the Global Accountability Project of the One World Trust, the Active Learning Network for Accountability and Performance in Humanitarian Action, and the Humanitarian Accountability Partnership. (See again the relevant websites, and for discussion of such initiatives, Jordan and Van Tujil 2006, and Roche (forthcoming).) While such initiatives are welcome, however, none provide the kind of information that would substantially affect the basic point I am making here: that it is very difficult for people with no special knowledge of or expertise in international aid to find out how good or bad the effects of the work aid agencies do are. (For further discussion of the issues raised in this and the preceding note, see Horton (forthcoming).)

9 This was a study commissioned by the Development Assistance Committee (DAC) of the Organisation for Economic Cooperation and Development, which drew on much of the data on NGO development work then available. This work – the ‘DAC Study’ – was published in 1997, but the situation regarding evaluations of NGO aid appears not to have changed very much since then. For a recent overview, which includes some qualifications to this claim, see Riddell 2007, esp. 268.

10 I would not expect much dispute from aid specialists about this claim, in any case, for one of the points such specialists often emphasise is how hard it is for anyone (even aid specialists) to form judgements about the effects with much reason for confidence, given the lack of reliable data on the effects. For a little more on this, see §3 below.

11 For this reason, it is not the case that in this paper I assume a ‘consequentialist’ moral theory according to which moral questions are determined exclusively by reference to consequences or effects. (For an introductory account of consequentialism, see Sinnott-Armstrong 2006). I assume merely that how good the effects are is one issue that needs to be taken into account. And for the reasons just given in the main text, this is something that no-one would deny. (As Onora O’Neill puts it, in the most sustained discussion of these questions to date from a Kantian point of view, ‘When so much hangs in the balance for so many it would be frivolous to depend on ways of reasoning which are not at all concerned with results’ (O’Neill 1985: 97).)

12 When I write about whether the effects are ‘good enough’ in this paper, then, I will mean ‘good enough to imply that we should give’, leaving open how good those effects have to be in order to be ‘good enough’ in this sense.

13 For ease of exposition, I shall continue sometimes to write about ‘finding out’ or ‘establishing’ how good or bad the effects are, but these phrases should be interpreted with the points just made in mind; that is, as shorthand for ‘arriving at an estimate about the effects that one has at least some minimally good reason to believe correct’.

14 The ‘global poor’ will be my shorthand for the people such agencies seek to benefit, though of course this group of people won’t coincide exactly with any of the groups picked out by any of the standard definitions of poverty.

15 Such an increase in donations may or may not be large enough to make a statistically significant impact on global poverty as a whole. Even if it is not, though, some of the global poor would benefit, and this is important enough.

16 And conversely, some people do give now despite the fact that it is difficult to find out whether the effects are good enough, though very few a significant proportion of their income, as some philosophers have argued they should (see e.g. Singer 1972, esp. 238-9, Unger 1996, esp. ch. 6, and Cullity 2004, esp. ch. 10).

17 As David Crocker puts it, ‘what challenges aid to distant peoples is not so much skepticism concerning moral foundations as pessimism about practical results’ (Crocker 1996).

18 Some may give a little money now more in hope than in confidence that the effects are good enough, and thus be disposed to give more if it became clearer that the effects are good enough.

19 In part, because doing so would take a lot of space, and in part because I am far from sure that I would be competent to do so, even given lots of space.

20 Unless otherwise indicated, by ‘the effects’ I mean the effects of the work aid agencies do in general, or as a whole. It is important to note, though, that the same argument would apply even if it were merely the case that a certain identifiable subset of the work aid agencies do – the work of a particular agency or group of agencies, for example, or of a certain kind of aid – had good enough effects, as long as that type of aid had not already absorbed all the money it usefully could. For again, if it were easier to find out that that subset of activities had good enough effects, then people would presumably give more to support those activities, and this would benefit the global poor.

21 One may also turn to those who work for such agencies, of course, though one would naturally wonder if such people would tend to find it harder to be completely unbiased, open, and honest than those who occupy more independent roles.

22 Exceptions include Edwards 1999, Black 2002, and Riddell 2007, though each of these works discusses aid as a whole, rather than focusing in more detail on NGO aid.

23 The DAC Review referred to in §1 (Kruse et al. 1997) does not appear to have been aimed at the general public, for example, as it contains a lot of aid jargon that is not explained.

24 A number of organisations (such as Guidestar) provide information about NGOs, but such information tends to be limited to the kind of thing one would find in NGOs’ annual reports. To my knowledge, none of these organisations conducts rigorous evaluations of NGOs or their work.

25 Michael Maren, for example, writes, ‘What is really required is a truly independent agency . . . to look after the interests of the targets of development and relief, a.k.a. the needy. The organization should be staffed by professionals who have the time and resources to produce detailed analyses of what these organizations [NGOs] are doing for the poor of the Third World. Those that do effective work should be singled out so that “customers” know where to spend their money’ (Maren 1997: 269). See also Clark 1991: 72.

26 Perhaps the growing stock of findings from randomized trials of development activities would be able to support such judgements? See e.g. Banerjee and He 2003 and Duflo and Kremer 2005.

27 As this paper goes to press, I read in Singer 2009 that an organisation that roughly fits this description has been created (though it is not clear if those who run it may be called ‘aid specialists’). It is called ‘GiveWell’: see http://www.givewell.net/ and Singer 2009, ch. 6.

28 If, on the other hand, it turns out that no kind of aid has effects that are good enough, then it would of course also be helpful to be told that.

29 Of course, some aid specialists would be in a better position than others to give such guidance, depending on such factors as how much experience of NGO aid in particular they have, how much familiarity with whatever data is available, and so on.

30 See e.g. Fowler and Biekart 1996, Biekart 1999, esp. ch. 3, Fowler 2000: 11-20, and Riddell 2007, chs 16-19.

31 I say a little more about some of those issues in Horton (forthcoming).

32 As I said in §2, ‘finding out’ how good or bad the effects are is my shorthand for ‘arriving at an estimate about the effects that one has at least some minimally good reason to believe correct’.

33 I tried out some of the ideas in this paper in a slightly different form in a workshop, International Non-Governmental Organisations and Ethical Questions, held at Melbourne University in July 2007, and would like to thank the participants for putting forward some of those objections.

34 I thank Chris Roche in particular for pressing this objection to me. See also Riddell 2007: 307-8.

35 My point here is merely that the fact that short and simple judgements about NGO aid as a whole are short and simple (and thus neglect many complexities and qualifications) may not mean that they are misleading or unhelpful. I am not of course claiming that those aid specialists who feel that making such judgements would be misleading or unhelpful should nevertheless feel obligated to make them. Nor am I claiming that only such very simple and general judgements would be helpful. As I said explicitly both in §3 and in the second paragraph of §4, tentative judgements with many qualifications may also be helpful, as may judgements about particular NGOs or types of NGOs or particular types of aid.

36 This possibility is put forward and discussed briefly in Riddell 2007, at 156-7. And indeed, some aid specialists accuse aid agencies of pandering to such desires, providing conscience-salving gimmicks rather than measures that might really lead to lasting solutions. See e.g. Smillie 2000: 121.

37 As Riddell writes, ‘There are only a handful of NGOs today which put information into the public domain which draws attention to failures and problems as well as successes’ (2007: 268).

38 For discussion of some biases that seem likely to affect our thinking about whether or not we should give to aid agencies, see Horton 2004.

39 This danger is illustrated by the experience of the Disasters Emergency Committee (DEC) in the UK, an umbrella organisation representing 13 UK aid agencies. For several years, DEC made independent evaluations of the work of their agencies publicly available. The media tended to focus exclusively on the negative elements of such reports, however, and in response DEC now commissions two reports, one for public consumption and the other for insiders only.