1. What is DARA?
2. What is the HRI?
3. Why does the HRI focus on government donors?
4. What is the foundation of the HRI?
5. How does the HRI measure the adherence of donors to the GHD principles and practices?
6. Why did DARA decide to create the HRI?
7. What experience does DARA have in this area?
8. Why does the HRI use a ranking?
9. What are the advantages of using a ranking?
10. Doesn’t a ranking detract from a spirit of collaboration amongst donors?
11. Won’t a ranking be a disincentive to new donors?
12. Donors already have indicators to assess the application of GHD, like the peer review process. Isn’t this enough?
13. Other reports look at donor’s humanitarian assistance. Isn’t this a duplication of efforts?
14. How can different stakeholders use the results?
15. What do the rankings tell us after three years?
16. How have government donors responded to the HRI?
17. What difference has the HRI made since it was launched in 2007?
18. How is the HRI financed?
19. Which crises are studied in the HRI?
20. Who do we ask to answer the field survey questionnaire?
21. What kind of questions do we ask?
22. What other data do we use?
23. Has the methodology used in the HRI 2009 changed compared with previous years?
24. Given these changes, can this year’s findings be compared with previous years?
25. How reliable are the HRI results?
DARA – Development Assistance Research Associates – is an independent, international, non-profit organization, which works to improve the quality and impact of development and humanitarian interventions. We do this through research, evaluations, promoting learning and knowledge sharing. Founded in 2003 and with offices in Washington DC, USA and Madrid, Spain, DARA has carried out evaluations in more than 40 countries across five continents for a variety of organizations, including United Nations (UN) and European Union (EU) agencies. The organization also produces its own self-promoted independent initiatives, such as the Humanitarian Response Index (HRI) and the Disaster Risk Reduction Initiative (DRRI), which seeks to forge partnerships for risk mitigation and adaptation.
Created by DARA in 2006, the Humanitarian Response Index (HRI) is the world’s first independent instrument for measuring the individual performance of humanitarian donors. Produced annually, the index ranks the performance of the 22 donor countries of the Organisation for Economic Cooperation and Development- Development Assistance Committee (OECD-DAC) plus the European Commission in funding and supporting humanitarian action. The HRI’s assessment criteria are based on the principles of Good Humanitarian Donorship, which government donors have themselves signed up to as part of their commitments to good practice. The HRI assesses how well donors are applying these commitments in practice in different situations of conflicts and disasters.
3. Why does the HRI focus on government donors?
The Index is based on the premise that, since government donors fund around 85% of all international humanitarian aid, they have a powerful and influential role to play in promoting positive changes in the way the humanitarian sector provides assistance to those who need it the most. Holding governments accountable for making efficient and effective use of public and taxpayers´ money is critical for the survival and wellbeing of some 250 million people affected by disasters and conflicts every year. This is even more important in times of economic uncertainty and increasing humanitarian needs.
4. What is the foundation of the HRI?
The HRI is based on the 23 principles and practices of Good Humanitarian Donorship (GHD), a political commitment developed in 2003 and since then signed up to by all the OECD/DAC government donors included in our index (see www.goodhumanitariandonorship.org). The GHD outlines a series of measures for government donors’ to align their humanitarian funding and assistance based on humanitarian principles and apply good practice in the way they work with their partners. The HRI developed performance indicators for the GHD principles to assess how well individual donors are applying the GHD in practice. The results are tabulated into comparative scores and into a ranking of 23 of the principal government donors that mainly fund and support the international humanitarian system.
5. How does the HRI measure the adherence of donors to the GHD principles and practices?
The Humanitarian Response Index (HRI) uses both quantitative and qualitative data to build 60 indicators, organized into five Pillars, which capture the essence of the Principles of Good Humanitarian Donorship. The Pillars correspond to five basic questions:
The index uses both qualitative and quantitative data to measure the performance of donors on these five pillars.
The qualitative data comes from structured interviews and questionnaires completed by members of implementing organizations working in humanitarian crises around the world. This year, field research was conducted in 13 crisis and more than 2000 responses were collected on donor behavior (1589 about OECD-DAC donors ranked in the HRI and 503 other donors and funding sources). To construct the Index several quantitative humanitarian sources of reference are also consulted, such as the databases of the OECD DAC, UN OCHA’s Financial Tracking System, ECHO’s point system and other relevant databases and information published by governments.
6. Why did DARA decide to create the HRI?
There are many different initiatives within the sector to improve the quality, effectiveness and accountability in humanitarian action. However, the majority of these initiatives focus on operational actors – the organizations that actually implement aid programs on the ground - and not on government donors, the main funders of humanitarian action. DARA created the HRI in response to this gap. The HRI provides a means to assess the performance and promote greater accountability of government donors, a major component of the humanitarian system. As an independent exercise, the HRI provides a neutral, impartial and objective view of donor performance to interested stakeholders.
7. What experience does DARA have in this area?
DARA has conducted evaluations of humanitarian and development assistance in more than 40 countries across five continents for a variety of organizations, including United Nations (UN) and European Union (EU) agencies. Of particular significance was DARA’s involvement in the studies on donor funding as part of the Tsunami Evaluation Coalition (TEC). These studies showed that donors lacked specific, practical guidance and benchmarks to assess how they are implementing their commitments to the principles of good practice contained in the Good Humanitarian Donorship (GHD). DARA developed the HRI to help bridge the gap between donors’ political commitments to improve the quality of their assistance, and actual donor practice.
8. Why does the HRI use a ranking?
DARA chose a ranking and scoring approach for the HRI as a means to stimulate public debate and awareness about the quality of governments’ humanitarian assistance and to promote actions to improve aid effectiveness and accountability. Indeed rankings and indices are increasingly used in the public sphere to help convey complex information in a simplified manner. The HRI is therefore similar to other annual assessments such as the UNDP’s Human Development Index or the Center for Global Development’s Commitment to Development Index. A parallel example for OECD/DAC humanitarian donors is the OECD’s PISA annual evaluation of members’ states educational system. The PISA study systematically assesses and ranks the quality and impact of member states educational system. The findings often provoke a healthy debate within national governments and civil society on how to improve educational results. The HRI similarly hopes to promote greater debate on how to improve the way that donor governments support humanitarian action.
9. What are the advantages of using a ranking?
An annual ranking allows governments, humanitarian organizations and the public to track the overall progress and evolution of donors’ performance in the humanitarian sector. More importantly, it helps stimulate public debate and reflection on what defines good practice. In addition, because it is based on a comprehensive set of performance indicators, the ranking offers a wealth of information for analysis, pinpointing strengths and weaknesses in donors’ responses to different crises contexts, and in different areas of donor practice, as well as offering a platform to compare where donors are collectively doing well and where they are performing poorly. The HRI’s individual rankings also help uncover the top and poorest performers, thereby addressing the problem of ‘free-rider’ behavior, where poorest performers benefit from the performance of others, and where the efforts of good donors may go unacknowledged. This can lead to greater collaboration between donors to learn from each other, and may serve as may serve as an example and incentive to government donors to work towards applying their commitments to good practice, and to pressure those who don’t.
10. Doesn’t a ranking detract from a spirit of collaboration amongst donors?
Collaboration largely depends on the willingness of donors to work with each other and with their partners and stakeholders to improve performance and accountability. By systematically providing a solid base of evidence of how donors are performing on a wide range of indicators of good practice and highlighting areas for improvement, we think the HRI can actually be used as a tool to promote greater collaboration and information sharing. We believe that the HRI provides an incentive for individual and collective efforts to improve the humanitarian system – the very aims that underlie the GHD principles.
11. Won’t a ranking be a disincentive to new donors?
Like more established donor governments, new and non-traditional donors want to do their best when it comes to providing aid effectively. Rather than act as a disincentive, the HRI may actually help new donors to better understand what good practice is and how it can be measured and assessed. This may help new donors to subsequently avoid many of the problems and issues in providing aid effectively that donors themselves identified when the GHD principles were drafted. In fact, this year the HRI has systematically begun to gather information on new and non-traditional donors, as well as specific examples of good and poor donor practices in different crisis settings. This information can be a valuable, practical tool to help all donors to better understand what works and what doesn’t work in terms of effective donor practice.
12. Donors already have indicators to assess the application of GHD, like the peer review process. Isn’t this enough?
Donors should be applauded for their efforts to track progress in implementation of the GHD principles and for using mechanisms like peer reviews to help agencies improve. However, these mechanisms tend to focus more on internal donors processes and do not always provide the level of participation and transparency that other stakeholders in the humanitarian sector would like to see. Furthermore, the HRI goes much further than any of these tools by looking at a wider set of both quantitative and qualitative indicators, and conducting field research to verify how the GHD principles are being applied in reality.
13. Other reports look at donor’s humanitarian assistance. Isn’t this a duplication of efforts?
There are relatively few reports that focus on the issue of donors´ humanitarian assistance. Most of these reports focus tracking funding flows using financial data from donors, and not the quality or impact of that funding. In contrast, the HRI has a much broader scope of analysis including both quantitative and qualitative indicators for other important GHD principles such as implementation of international laws, conventions and quality standards. More importantly, the HRI is one of the few annual research processes that include extensive field research in different crisis contexts. This data is generated through a large-scale field survey of humanitarian actors working in 10-14 different crises each year. The field survey provides a “reality check” on how well donors are responding in different crisis contexts. This approach makes the HRI the largest and most comprehensive review of how donors and the international humanitarian system are responding to meeting humanitarian needs.
14. How can different stakeholders use the results?
The comprehensive research process provides government donors, humanitarian organizations and the general public with a rich body of data and analysis to help uncover good and poor practice and work towards improving donors’ performance and accountability.
15. What do the rankings tell us after three years?
After three years, the HRI findings show that the world’s most powerful and wealthy donor nations are still far from meeting the aims of the Good Humanitarian Donorship (GHD) initiative to which they committed themselves. No donor scores above 8 out of 10 on the HRI’s consolidated score, and nearly half fail to achieve scores of 6 out of 10, showing that there is great room for improvement amongst all donors. Since 2006, some donors have showed continuous progress and improvement which is reflected in incremental changes in their rankings. Others donors, on the other hand, have consistently done poorly when assessed against the GHD principles and show no sign of improvement. This suggests that the GHD principles are not a priority for these donors, and raises questions about the level of commitment amongst the OECD/DAC group of donors to the GHD. The HRI research also shows how little the GHD principles are known to donors’ main stakeholders. In 2009, only one in five people interviewed for the HRI survey considered themselves to be “very familiar with the GHD”, less than the number reported in last year’s survey, which was around a third of respondents. If donors’ principal partners are unfamiliar with the GHD, this implies they do not know what they can expect from donors in terms of good practice, nor do they have knowledge of the extent of donors’ responsibilities and accountabilities when responding to crises. These findings highlight the need to revisit the GHD principles, originally drafted in 2003, to clarify any contradictions and take into consideration changes within the humanitarian system. There is a need to engage in a wider dialogue on what good donor practice is in today’s context and how to apply it.
16. How have government donors responded to the HRI?
Government donors have expressed some concern about the HRI rankings and the methodology used to generate the scores, and have stated that the rankings might detract from a collaborative spirit between donor agencies to improve performance. DARA has taken these concerns into consideration and has continually refined and improved the methodology, and has attempted to engage with donors in a constructive dialogue on how to use the HRI as a tool for identifying and promoting good practice. Privately, more and more staff of donor agencies have told DARA that the HRI provides them with information that can be used for internal lobbying within their agencies to encourage and apply good practice. The GHD group has also adapted their own collective indicators to measure the progress of implementation and there is now more alignment with HRI indicators. This suggests that the HRI can be compatible with donors’ own efforts to measure and improve their performance.
17. What difference has the HRI made since it was launched in 2007?
The HRI continues to raise awareness of the importance of measuring and benchmarking donor performance. But the annual reports also serve to highlight several key issues facing the humanitarian sector, such as limited capacity to adapt and respond to increasing needs, and the constraints and challenges of applying the GHD principles in different crises contexts. Some of these issues, such as improving the quality and use of needs assessment, are now being taken seriously within the sector and donors are contributing to finding solutions to them. Another way that the HRI is having an influence on the sector is that the research process itself has opened up a previously non-existent channel for humanitarian organizations to reflect on the quality of their relationships with the donors that fund them, as well as an opportunity to voice their opinions in an impartial and confidential way. At the same time, the HRI findings have provided evidence which can be used for internal lobbying to highlight inconsistencies of governments in their different approaches to different crisis and leading to an improvement of the humanitarian system as a whole.
The HRI is privately funded and receives no funding from government donors in order to protect the independence and integrity of the research process. During the first two years, seed money was provided by private philanthropists to help start up the HRI. As of 2009, the HRI council, a small group of foundations, has contributed to the costs of producing the HRI. Many different humanitarian agencies also support the HRI by providing in-kind goods, services and logistics support during the HRI missions and to promote and disseminate the findings to a wider audience.
19. Which crises are studied in the HRI?
This year, we have assessed the donor response to crises in 13 countries: Afghanistan, Chad, China, the Democratic Republic of Congo, Ethiopia, Georgia, Haiti, Myanmar, the Occupied Palestinian Territories, Somalia, Sri Lanka and Timor Leste, and collected over 2000 survey responses. Over the three years that the HRI has been in existence we have carried out field research in a total of 33 humanitarian crises and collected about four and half thousand survey responses. (In addition to repeat visits to those visited this year, other countries have included: Bangladesh, Central African Republic, Lebanon, Mozambique, Nicaragua, Niger, Pakistan and Sudan.)
20. Who do we ask to answer the field survey questionnaire?
As part of the HRI research process, we interview representatives from humanitarian organizations working in different crises about how donors are supporting the response. In particular we speak to people who have some kind of relationship with donors. Normally this is the head of mission or the person in charge of preparing funding requests or reports to donors. We aim to interview representatives from all the different humanitarian organizations working in the crisis, such as UN agencies, the Red Cross/Red Crescent, and local and international NGOs. When possible, we also interview donor representatives in the country.
21. What kind of questions do we ask?
Survey respondents are asked to answer a series of 40 questions and statements on how well they feel each of their donor supports their work and if they believe donors are applying key concepts of good practices from the GHD, using a five-point Likert scale. Each question is linked to core concepts contained in the GHD, such as timeliness of funding or donor support to facilitate safe humanitarian access. The survey also includes several open-ended questions to allow the interviewer and respondents to clarify and expand on any of the answers. All responses are confidential in order to ensure more candid answers and to protect the often delicate relationship between funders and recipients of aid money.
22. What other data do we use?
In addition to information collected through the field survey and interviews, DARA also makes extensive use of data published by a wide variety of sources. This includes data on donor funding flows from the OECD/DAC, UN OCHA’s Financial Tracking Service, and other UN agencies, and information from the IFRC and ICRC. This quantitative data helps to complement the field data, and allows for a more comprehensive and reliable picture of donor behavior. All the information is collected and analyzed using standard statistical measures, and then converted into scores and rankings for each of the HRI’s indicators of donor practice.
23. Has the methodology used in the HRI 2009 changed compared with previous years?
DARA takes every edition of the HRI as an opportunity to continue to refine and improve the methodology based on feedback and lessons learnt from the previous year. In 2009, the survey design was streamlined and improved, and the scope of the survey was expanded in order to assess non-GHD donors such as governmental INGOS, UN and Pooled Fund mechanisms, and private foundations. This was to address a trend noted in previous HRI reports of the growing importance of non governmental donors. The survey also includes new and emerging government donors, such as Saudi Arabia, Korea and others. With over 500 survey responses this makes a good platform to explore how these actors perform as donors.
We also increased the number of crises from 11 in 2008 to 13, obtaining more survey responses than the previous year (1400 to 1600). In addition, the statistical formulation of many of the indicators was improved, and data sources reviewed to ensure accuracy of information collected.
We continue to refine and improve the HRI methodology and process based on lessons learned from previous years and through an ongoing process of consultation and feedback. The organization continues to work with other stakeholders to build a common approach to defining standard data sources and performance indicators, and has initiated a wide consultation process to understand concepts of good donor practice beyond what is stated in the GHD Principles in order to incorporate into the HRI’s future analysis.
24. Given these changes, can this year’s findings be compared with previous years?
Yes. The basic foundation of the HRI remains the same, as do many of the 60 different indicators, results can be broadly compared from one year to the next. The statistical methods used to generate the index scores and rankings also allow DARA to carry out more sophisticated comparisons between certain indicators, by donor and by crisis. Our intention is to begin to use the data generated over three years to produce additional reports on trends and patterns in the system. However, it is important to note that while the scores are comparable, the rankings are based on comparison among GHD donors; therefore a change in the ranking of a country may be partially influenced by changes in another country’s performance.
25. How reliable are the HRI results?
One of the reasons DARA developed the HRI was to provide the humanitarian sector with an empirical evidence base to assess donor performance, avoiding informal and anecdotal information. The HRI methodology and process represents the “state of the art” in terms of performance measurement in the humanitarian sector. The indicators capturing the GHD Principles and the research methodology used to construct the HRI were developed in wide consultation with experts from the humanitarian sector and other sectors, with the aim of ensuring transparency, consistency and rigor in the approach. The HRI attempts to ensure the findings are as accurate and reliable as possible through the following measures:
Representative crises:
Standardized survey design:
Survey respondents:
High survey response rate:
Application of standard statistical analysis:
Peer review:
©2010 DARA All rights reserved. Disclaimer - Recruitment - Contact us