28.08.2020

By Brenda de Farias Oliveira Cardoso, Honorary mentioned in The HUMLOG Institute’s Best Thesis Award 2020

Supervisor: Adriana Leiras | Co-supervisor: Tharcisio Cotta Fontainha

I am a researcher at the HANDs Lab (Humanitarian Assistance and Needs for Disasters) which is a laboratory of the Industrial Engineering Department at the Pontifical Catholic University of Rio de Janeiro (PUC-Rio) focused on research in Humanitarian Logistics and Operations Management in disasters, crises, and emergencies.

In the development of my master thesis, I saw an opportunity to combine two important research topics: performance evaluation and humanitarian logistics. Nevertheless, I decided to follow a novel perspective, and I ended up looking at how a humanitarian operation can be considered successful from the perspective of the beneficiaries. The supervision of Professor Adriana Leiras and co-supervision of Professor Tharcisio Cotta Fontainha were essential throughout the process and receiving an Honorable Mention at the HUMLOG Best Master Thesis Award 2020 contributes to enhancing the acknowledgement of the research relevance for a broader audience. I hope you find the time to read some of my thoughts on the topic.

As a contextual factor, we see that the discussion about how to respond to a disaster is gaining more and more attention due to the increasing occurrence and consequences of disasters worldwide. In 2019, for instance, natural disasters affected more than 95 million people and caused losses of 130 billion dollars. These numbers reinforce the importance of humanitarian logistics and mainly the disaster response when decisions must be designed to minimize the impacts of these events on the lives of the affected population. Consequently, academic and professional communities still demonstrate a constant interest in the development of metrics to evaluate and pursue improved response operation.

Despite the challenges encountered in disaster response operations, performance evaluation proves to be an essential tool for directing organizations according to strategic objectives. The occurrence of disasters causes donors, the media and the beneficiaries themselves to monitor the efficiency and speed with which organizations can operate in operation. With data structured from a well-established evaluation system, organizations can disseminate their results with more transparency and assertiveness.

However, the traditional performance evaluation perspective in humanitarian logistics focuses on the view of stakeholders providing aid to the population, such as humanitarian organizations, governments, and donors. Indeed, these aid providers need to understand the real situation to pursue improved operations. Nevertheless, not always do the performance evaluations include the perspective of the main clients who, in the humanitarian context, are the victims or beneficiaries (the users of the service). Generally, the organizations are evaluated by their funders which may cause some conflicts of interest, such as biased analysis without reporting the real problems that exist in an organization. This problem brought the following research question: how to assess the response to a disaster from the perspective of the beneficiary?

This discussion was the central theme of my master research. Performance evaluation in humanitarian operations is a growing research topic, not only due to the need for transparency in operations but also due to the focus has not been on the beneficiary. Therefore, from a systematic literature review of the, my master thesis proposes a framework with a step by step to measure the performance of operations considering the perspective of the beneficiary. The validity of the proposed framework is tested through two empirical research methods: survey and case study.

The proposed framework is based on other frameworks found in the literature, as well as on the analysis of papers related to performance evaluation in humanitarian operations. The five steps in the framework are (1) identification of organizational objectives, (2) processes selection, (3) categories selection, (4) data collection, and (5) assessment results. The organizational objectives include efficiency, effectiveness, delivery, flexibility, productivity, quality, cycle time, and adaptation. The second step is the selection of processes and involves the distinction between pre-disaster and post-disaster categorization, which affects the remain steps. The selection of categories step takes into consideration the pre-disaster and post-disaster with their respective six areas of analysis:

  • Health: availability of medicines, medical care and treatments, and food.
  • Housing: quality assessment of the facilities and housing security.
  • Education: educational services, such as access to schools and educational courses or lectures for children, youth and adults.
  • Assistance: satisfaction of the services received in general, such as, for example, response time, reliability of information and quality of products or services.
  • Socioeconomic factors: the relationship with the community in the pre-disaster and post-disaster, assessing, for example, access to the labor market, financial situation, engagement with the local community.
  • Disaster risk management: the existence of management activities, such as lectures, courses, programs and information dissemination.

After selecting the categories to be evaluated, we identify the instrument for data collection (interviews, documents, records and questionnaires) to determine the level of satisfaction of the beneficiary. From the collection and analysis of the results obtained, it is possible to direct the operations in search of efficiency, effectiveness, and equity.

In the survey method, the unit of analysis is the response operation after a disaster in Brazil and in this method the beneficiaries’ satisfactions and dissatisfactions are identified, as well as the strengths and weaknesses related to post-disaster action. In the case study method, the objective is to validate the results found in the RSL and investigate the results found in the survey, where the unit of analysis is the organization responsible for providing services to beneficiaries. In this method, it is possible to understand clearly, from the organization, each important aspect pointed out in the survey, and the main dissatisfactions and negative points are treated with priority in an interview with the person responsible for the case. The combined approach of the two techniques allows complementary results.

The study contributes to the academic literature with relevant results on the topic, including a performance evaluation framework from the beneficiary perspective and its initial validation through an empirical study, serving as a basis for new theories and practical applications. Besides, these deliverables may serve for considerations of professionals from organizations in the sector that aim to improve their services and processes looking at the beneficiary view.