The Case of Many Indices – A Study of Disaster Risk Reduction Indices

We live in an age of information. Every day, we are collecting even more data that gives us more insight, and when combined with the modern computation power, helps us improve our understanding of our world. Indices leverage this information to standardize previous measurements, and in some cases, even measure that which previously could not be measured, allowing us to make sense of the abstract.

Indices play an extremely important role in combating climate change. From measuring the risks posed to a country because of climate change to tracking the readiness measures adopted, these indices guide us in our efforts to combat climate change. These indices help governments and communities in understanding the risks and building resilience.

Let’s consider two indices: ND Gain – Country Index, and The World Risk Index. Both the indices measure a country’s vulnerability to climate change and study that country’s preparedness in response to its vulnerability. Based on these two components, the two indices calculate the final score for each country. We limit our analysis to the Oceania countries, since that lies within the scope of our broader project, using the scores to assign a rank to countries. Table 1 shows the country ranks.
Table 1

Missing Data

As we can see from the table, every country in the Oceania region has a different relative rank as per the two indices. While that is not something unexpected, as each index measures vulnerability and adaptability using slightly different methodologies, the surprising finding is the extent of variation in the country rank across the two indices. While Fiji is ranked the best by ND Gain, the country stands at the fourth position according to the World Risk Index. Similarly, Micronesia, which fares poorly as per the ND Gain Index (6th out of 7), does extremely well according to the WRI rankings (2nd out of 8). The only exception is the nation of Samoa, that has a similar rank (2nd and 1st) for both the indices.

So how do we reconcile the differences between the indices? More importantly, how do we ensure that we are getting accurate and consistent insights into disaster risk preparedness.

For starters, indices should be more transparent about their meta-data and their methodology. Much of the discrepancies in indices is due to missing data. At least that’s what the experts tell us.

The Disaster Risk Reduction team, working on the Oceania project, investigated the two indices for missing data. We studied the meta-data that informs these indices. We found that some of the sub-components were indeed missing data for a few countries. This was primarily because these indices source their data from global datasets like the World Bank, the World Health Organization, etc. For example, both the indices use World Bank data to inform their economic and development indicators. Any discrepancies, including missing data, in the World Bank dataset, creeps into these indices, leading to in However, this still does not explain the difference in scores for countries that have data available for all of the indicators.

Differing Methodology

The most probable explanation is the difference in methodologies. In this case, both the indices – ND Gain and The World Risk Index, lay out their methodology in detail. Both the indices are formulated by researchers who have years of experience working on the issue of climate change and resilience. On a broader level, the two indices follow a similar methodology – assess the climate vulnerability of a country, study the preparedness, often termed as readiness, and then analyze the gap between the two to calculate the index score.

But that does not resolve the issue. The variation stems from how indices define vulnerability and preparedness. The ND Gain index defines vulnerability as a combination of multiple components, ranging from Ecosystem, Health, to Exposure and Sensitivity – a total of 9 sub-components, whereas the World Risk Index measures vulnerability in terms of Adaptation, Coping, and Susceptibility. This ultimately results in a discrepancy in the final score.

The other discrepancy that we observed during our preliminary study of the indices is the consistency – or lack thereof, in following the methodology. In my next article, I will try to recreate the scores using the Indices’ data and methodology. Retracing the steps, I will explain in detail the discrepancies between the ND Gain and The World Risk Index scores for countries in the Oceania region.

Harshal Zalke is a graduate student at the Lyndon B. Johnson School of Public Affairs. He is pursuing a Masters in Global Policy Studies and is interested in the interaction between International Trade and Politics. Before entering the policy world, Harshal completed a Masters in Operations Research from the Cockerell School of Engineering at The University of Texas at Austin. In the summer of 2017, he worked as a GRA at the Energy Institute studying the adoption rate of clean technologies in the world.

Leave a Reply

Your email address will not be published. Required fields are marked *