Methodology: Inside Bloomberg’s Covid Resilience Ranking

(Bloomberg) -- Everyone is battling the same coronavirus, but quality of life, control of the pathogen’s spread and progress toward reopening look vastly different across the world.

Bloomberg’s Covid Resilience Ranking scores the largest 53 economies on their success at containing the virus with the least amount of social and economic disruption. As more places deepen their reopening efforts amid rising vaccinations, the progress of normalization in global travel routes and flight capacity remains a key indicator, having been taken into account since June 2021.

The Winners and Losers From a Year of Ranking Covid Resilience

In devising the Ranking -- which launched on Nov. 24, 2020 -- we considered many datasets, indicators and indexes produced by organizations around the globe and applied three fundamental criteria in whittling the list down to the 11 components that make up the current measure:

• How complete is the data? Many relevant indexes and databases -- for instance, measures on trust in government -- cover only a small number of places. We focused on indicators that cover the vast majority of the 53 economies in the Ranking, filling in the gaps with substitutions where reasonable.
• How current is the data? All datasets have a lag, some of up to a few years. Because of the pandemic’s pace and transformational impact, we chose to use the most up-to-date datasets where possible, with the maximum lag of about one year at the start of the Ranking. Eight of our indicators are refreshed daily, one is weekly, and two are annual figures.
• Who collects the data? We decided to only use data compiled by Bloomberg or indicators from reputable third-party organizations with a track record of collation and analysis.

Why rank only 53 places?

We decided for brevity and relevance to limit the Ranking to economies valued at more than $200 billion prior to the pandemic.

How often do you update?

The Ranking story is updated monthly, usually in the last week of every month.

How is the Ranking aggregated?

Each of the 11 data indicators are aggregated through the “max-min” method, which is used to convert metrics expressed in different scales into a common one, while maintaining the relative distance between values.

All the indicators are scored on a 0-100 scale, with 100 (blue) indicating the best performance and zero (orange) the worst. The rest fall in between, scaled by their distance from one another. The final Bloomberg Resilience Score is the average of a place’s performance across the indicators, equally weighted.

The final score given to each place is a relative measurement on a given date. That score shouldn’t be compared in isolation to the economy’s previous scores as the max-min ceiling and floor values change in every update.

What is the Reopening Progress tab about?

Where infections, deaths and the capacity of a health-care system to cope were once paramount, now the ability to essentially turn back the clock and return to open, pre-Covid times is also taking on significance.

Until early 2021, being in a place that had stamped out local virus transmission in part by shutting out the world was one of the best outcomes. But with more and more places taming serious disease and fatalities through vaccination, progress in safely lifting restrictions on social and business activity -- including global travel and tourism -- is now an important factor in an economy’s pandemic performance.

Hence in the June 2021 Ranking we introduced the Reopening Progress tab, with four indicators that help measure a place’s path toward opening up to the wider world. The bucket includes two metrics that have been in the Ranking for some time -- Vaccination Progress and Lockdown Severity -- and two brand-new datasets, Flight Capacity and Vaccinated Travel Routes.

What is measured with Flight Capacity?

Bloomberg publishes a flight tracker based on data from flight-monitoring firm OAG. The new Ranking metric draws from this, calculating the provision of plane seats in each place in the latest available four-week period compared with the same period prior to the pandemic between February 2019 and January 2020.

The data combines domestic and international flights, rewarding larger economies where local travel has made up for the plunge in global air traffic. Mainland China was the top scorer in this indicator, until its variant-driven outbreaks at the end of 2021 and into 2022, with strong domestic travel momentum as the population was prevented from vacationing overseas.

Conversely, small, travel-reliant economies like Hong Kong and Singapore which don’t have domestic air travel markets are typically among the worst scorers.

How is Vaccinated Travel Routes calculated?

It’s not an easy task to quantify the openness of a country in a fast-moving pandemic. Bloomberg collaborated with data provider Sherpa, whose collation of information on travel restrictions is used by companies like Wego Pte Ltd., Expedia Group Inc and American Airlines Group Inc.

We used Sherpa’s underlying database to create a measurement of how many open travel routes -- both outbound and inbound -- a place has with the rest of the world for vaccinated people. Routes where quarantine-free travel is possible count as a full route, while those which require quarantine count as half.

Some places still bar all travelers and only allow residents to enter their borders, which we don’t consider an open route.

In the scoring, we only look at entry restrictions on the destination end to determine the openness of a route. Some places have exit restrictions barring residents from leaving the country, but we don’t take these into account as the indicator aims to track non-resident movement. 

Outbound and inbound travel are considered two separate routes, underscoring the complexity of navigating travel to an open place if one lives where borders are still closed. Travel restrictions are also not always reciprocal, with tourists from low-infection places welcomed in many places, though the place itself bars the vast majority of travelers.  

The scoring does not take into consideration testing requirements, as pre- and post-flight testing is now commonplace and easily accessible, unlike quarantine and isolation mandates. It also does not factor in vaccine types that are currently required by various jurisdictions, given that governments are moving toward standardized recognition of the most widely-used vaccines globally. 

With 242 jurisdictions tracked in Sherpa’s database, this indicator’s maximum theoretical score is 482 routes -- for a place which allows all quarantine-free vaccinated travelers to enter, and whose vaccinated travelers are welcomed by every other place. None of the ranked places have ever reached the maximum score.

What are you measuring with the Vaccine Doses Per 100 People indicator?

This metric that captures vaccine access has been repeatedly refined as progress on the ground becomes clearer. In November 2021, the People Covered By Vaccines indicator was replaced by Vaccine Doses Given Per 100 in accordance with the shift in Bloomberg’s Vaccine Tracker. 

The previous indicator approximated the percentage of a country’s population that is fully inoculated by taking into account total doses administered and the mix of vaccine types. Given that some vaccines require three or two doses and some just one, this method of calculating coverage allowed dosage differences to be recognized where data is available. 

Yet as more countries start rolling out booster shots, this method of calculation risked marking those getting their third shots as newly vaccinated. Thus, the metric of Vaccine Doses Per 100 People became a better mirror of the real-world situation. Given the limited use of single-dose vaccines in the world, the change of methodology basically doubled the figure for most countries without affecting differences in scale. The metric shift therefore had no material impact on the Ranking’s consistency. 

In earlier versions, the Ranking also included an indicator tracking vaccine supply agreements, named Access to Covid Vaccines. In March 2021, we retired this metric, given that distribution is proving far more important than deals signed. Below is a table capturing how the vaccine access indicators have changed since the Ranking’s debut in November 2020.

What does Lockdown Severity measure?

This indicator is based on an index produced by the University of Oxford, which assesses the number and strictness of government policies that limit people’s social contact as a way of containing spiraling outbreaks.

We’re interpreting restrictive government policies as a negative in the Ranking as the stricter the lockdown, the more disruption people are experiencing. A higher score on Lockdown Severity denotes a less advantageous performance.

Two years into the Covid crisis and with vaccination having a game-changing impact, the need to impose lockdowns reflects a failure to progress, and so we’re scoring it accordingly. 

Stringent restrictions also correlate with the mental and economic toll of the virus on a population. Social disruption and isolation have been linked to higher suicide rates in some places, while school closures have raised concerns over child development and increases in hunger and drop-out rates among disadvantaged families.

The index was updated in March 2022 by the University of Oxford to evaluate restrictions for the vaccinated and unvaccinated separately, with the two scores then being aggregated based on each country or region’s immunization progress sourced from Our World in Data. Prior to this change, the Lockdown Severity indicator showed only the strictest rules applied in each economy, which often only targeted the unvaccinated.

Does Lockdown Severity reflect conditions across an entire economy?

The Oxford University index reflects the most stringent conditions in place anywhere in a given economy. That means a lockdown in one city or area can be the basis for an economy’s overall score.

This naturally penalizes large, expansive countries like China and the U.S., where conditions can vary greatly from region to region and city to city. It also affects smaller places with a devolved system of government: for example, the U.K.’s conditions were still scored in the middle range in July’s Ranking despite England’s “Freedom Day” on July 19, 2021, because restrictions remain in Scotland and Wales.

Similarly, China’s conditions have generally been scored as stringent in most editions of the Ranking because some cities or districts were under targeted lockdowns as local officials fought virus flareups. But the majority of China’s 1.4 billion-strong population face barely any limits on their movement until the omicron variant entered its borders.

While imperfect, we view Oxford’s