Human rights data for everyone: Introducing the Human Rights Measurement Initiative (HRMI)

Abstract

Cross-national quantitative data measuring human rights practices have existed for about four decades. However, those data have yet to gain much traction in the public sphere, leaving human rights advocacy a largely data-free enterprise. We argue that the underutilization of these data is due to (1) a heavy reliance on public documentation alone as an information source, (2) incomplete coverage of the entire list of internationally recognized rights, and (3) the many barriers to data access and understanding faced by those outside of academia. In response to these problems, we introduce the Human Rights Measurement Initiative, a global collaborative project committed to generating the best cross-national, comprehensive human rights data possible through a process of codesign across disciplines and professions and presenting those data in ways that are accessible and useful to academics, practitioners, and everyday people alike.

Introduction¹

Quantitative measures of human rights are not new. Researchers were attempting to use quantitative data to measure the degree to which civil and political human rights factored into the distribution of US foreign aid as early as the late 1970s and early 1980s (e.g., Chomsky and Herman 1979; Schoultz 1981; Stohl, Carleton, and Johnson 1984; Cingranelli and Pasquerello 1985). Indeed, over the last few decades, two data projects—the Political Terror Scale (PTS) (Gibney Cornett, Wood, Haschke, Arnon, and Pisanò 2017) and the CIRI Human Rights Data Project (Cingranelli, Richards, and Clay 2014a)—have been used widely to produce a large corpus of quantitative human rights studies. Indeed, based on Google Scholar searches, it appears that PTS and CIRI have each been mentioned in at least two thousand scholarly items.

However, if we focus on mentions outside of academia, the story is quite different. Using Nexis Uni to explore the number of news articles that mention CIRI, one comes up with an estimate of about seventy-six articles; similarly, the PTS appears to have been mentioned one hundred and fifteen times.² Perhaps even more striking, a search of a corpus of human rights country reports produced by Amnesty International, Human Rights Watch, the Lawyers Committee for Human Rights, and the US Department of State turns up zero mentions of either PTS or CIRI (Fariss, Linder, Jones, Crabtree, Biek, Ross, Kaur, and Tsai 2015).³ This is not intended as a slight; compared to most academic projects, both CIRI and PTS have had a significant impact outside of academia. Governments, activists, and journalists alike used these projects’ work to make important decisions worldwide. Nevertheless, compared to the impact these projects have had in establishing a great deal of academic knowledge, one would hope that a similar effect would have been seen in the public sphere. However, large-scale use of social scientific human rights data in the public sphere has never materialized.

There are four main reasons why human rights data have largely been underutilized in global human rights practice and activism. First, governments do not produce accurate data on many human rights issues themselves, and they are unlikely to endorse data produced by others. Second, many of the previous measures of civil and political rights have relied heavily on publicly available documentation, opening those measures to questions of bias and missing information. Third, there has been incomplete coverage of the entire list of internationally recognized human rights, which serves to undermine their status as rights, each equal to all of the others in status and importance. For instance, many economic and social rights have been excluded from human rights datasets, and those studying and promoting those rights have been forced to rely on inappropriate data that do not directly connect with the obligations set forth by the international human rights regime. Finally, the data themselves have not always been presented in clear ways that are easy for those outside of academia to understand, which reduces the perceived usefulness of the data. As we will discuss in further detail, these issues do not only affect the degree to which practitioners are willing to embrace quantitative data, they have also served to weaken the social scientific study of human rights.

How do we overcome these problems to produce data that are simultaneously useful for academics, policymakers, practitioners, and everyone else? In order to answer this question, we introduce the Human Rights Measurement Initiative (HRMI), a global collaborative project that is focused on providing human rights data that are useful to practitioners, researchers, journalists, and everyday people worldwide. We believe HRMI overcomes the aforementioned problems by (1) producing data on economic and social rights, as well as civil and political rights, while aspiring to eventually produce data on every single internationally recognized human right; (2) producing the best data possible via a process of collaboration and codesign with human rights experts across disciplines and professions; and (3) producing and presenting those data in a way that is palatable and useful for everyone.

Here, we will briefly review other global human rights data sources, the issues we believe have prevented these data from being useful and widely used by individuals outside of academia, and the ways in which those issues have served to weaken the academic study of human rights. We will then discuss in detail how HRMI confronts these problems, before concluding with a few thoughts about the future of the project.

Why are human rights data missing from the public conversation?

The first question many people ask when confronted with the limited range of global human rights data sources is often, “Why does the United Nations not produce those data?” or, “Why do governments not produce human rights data on themselves in the same way they produce economic data?” These are good questions. Sadly, however, we do not live in a world in which governments can be fully trusted to produce accurate, honest, and transparent data on their human rights practices. When we consider that human rights data are a way of holding governments accountable for violations of their international legal obligations, their hesitance is unsurprising. As it stands, most governments attempt to conceal their rights violations and often outright challenge accounts that point out the violations in which government agents have engaged.

One need look no further than the US government’s treatment of the information about its own use of torture during the war on terror to have this point exemplified. When the Senate Select Committee on Intelligence developed a six thousand-page report detailing abuses committed by the CIA from 2001 to 2006, only a small portion could be released, and the rest remains classified (Senate Select Committee on Intelligence 2014). Indeed, there is reason to suspect that some governments may even attempt to misrepresent the most basic of economic indicators, like gross domestic product (GDP; Martinez 2018). Given that their measurement is widely publicized, open to external review, and subject to well-known, well-defined methodologies for their collection, the attempted manipulation of GDP and similar publicly available socioeconomic statistics is largely limited to the margins; the same cannot be said for human rights data. As such, the idea that we could possibly rely on governments to produce a reliable global data set on all human rights seems out of reach in the current political climate. Furthermore, the reluctance of governments to monitor many human rights has extended to intergovernmental organizations, like the United Nations, as the members of those institutions are the states themselves.

Given this reality, most of the widely used civil and political rights data sets are produced by academics and other nongovernment entities. For instance, PTS and CIRI are two of the most widely used human rights data projects in political science, and both were founded and maintained by academic social scientists employed at universities in the United States (e.g., Wood and Gibney 2010; Cingranelli and Richards 2010). The same is true for many other projects, like the Ill-Treatment and Torture (ITT) Data Collection Project (Conrad, Haglund, and Moore 2013, 2014) and the Human Rights Protection Scores (Fariss 2014; Schnakenberg and Fariss 2014). Each of these projects is dependent on publicly available information to produce its data. Both PTS and CIRI have relied heavily on information from the US State Department’s Annual Reports on Human Rights Practices and the Amnesty International Annual Report to produce their data (e.g., Wood and Gibney 2010; Cingranelli and Richards 2010).⁴ ITT relies on public Amnesty International allegations of torture, whereas Fariss (2014) combines several different sources of human rights data, all of which rely on public accounts of violations.

These projects have done a great deal for the academic study of civil and political rights, producing reliable data that allow for cross-national comparisons and providing scholars with the ability to make inferences about the various causes and consequences of rights violations. However, the information sources on which these data are based also carry well-known limitations. As Conrad, Haglund, and Moore (2014) discuss, the reports produced by large international human rights organizations (HROs), like Amnesty International and Human Rights Watch, contain many allegations, but fall well short of capturing the full corpus of human rights abuses. And, as Hill, Moore, and Mukherjee (2013) demonstrated in their study of Amnesty International “naming and shaming,” these reports generally have high credibility standards and rarely exaggerate the level of abuse in a country. Although this is a useful standard to ensure trust in the reports themselves, it also means that there are many abuses human rights practitioners know about that are never reported. As such, any attempt to count abuses on the basis of HRO public documentation will result in an undercount of the actual level of abuse.

Furthermore, it is highly likely that the undercount of human rights abuses from public HRO documentation is uneven across countries. HROs have limited resources and, as both a practical and ethical necessity, use the resources they have in the locations where they are likely to produce the best results (e.g., Hendrix and Wong, 2014; Barry, Bell, Clay, Flynn, and Murdie 2015). This means that their public documentation tends to focus on some countries more than others, yielding a more severe undercount of abuse resulting from those reports in some areas than others—that is, a biased undercount (Conrad et al. 2014). There are many proposed ways of correcting for these problems, including generating latent variables on the basis of the several variants of related data (Fariss 2014; Schnakenberg and Fariss 2014) or using statistical techniques to account for the undercount before one draws inferences from secondary analyses (e.g., Conrad et al. 2014; Bagozzi, et al., 2015). However, these solutions largely set aside the question of whether better data can be collected directly.

Beyond these problems, human rights data projects have rarely attempted to measure more than a limited subset of civil and political rights, failing to cover the full range and breadth of international human rights law. For instance, PTS specifically measures the violation of the subset of civil and political rights known as “physical integrity rights” (Haschke 2017). These can be defined as “the entitlements individuals have in international law to be free from arbitrary physical harm and coercion by their government” (Cingranelli and Richards 1999: 407), and data on these rights often focus on the rights to be free from torture, ill-treatment, extrajudicial killing, political imprisonment, and disappearance. Like PTS, the Human Rights Protection Scores and ITT also only cover physical integrity rights. The ITT focuses on torture and ill-treatment (Conrad et al. 2013, 2014), and the Human Rights Protection Scores attempt to capture a measure of physical integrity rights as a whole (Fariss 2014; Schnakenberg and Fariss 2014).

Likewise, CIRI has a combined measure of physical integrity rights; however, CIRI also includes disaggregated measures of the violation of different physical integrity rights and several measures of other civil and political rights, such as the freedoms of speech, press, religion, and electoral self-determination, among others (Cingranelli, Richards, and Clay 2014b). Furthermore, CIRI includes several measures of women’s rights, as well as a measure of worker rights. Other projects also capture parts of the international human rights regime. For instance, Mosley and Uno (2007) captured the rights to associate, collectively bargain, and strike in the workplace, and Hathaway (2002) created a measure for torture. However, data like these are largely produced for single academic works and, as such, are not updated and maintained.

Thus, until relatively recently, we were missing quantitative data on large sections of the international human rights regime. In particular, with the exception of a few labor rights, the economic, social, and cultural rights were long unmeasured as rights, and those new measures that have been developed to fill this gap are not yet well known. As a result, researchers have long relied on weak proxies for the degree to which governments are living up to their obligations to these rights in international law. Examples include the UN Development Programme’s (2016) Human Development Index (HDI)—which combines data on life expectancy, adult literacy, and GDP per capita—and the Physical Quality of Life Index (PQLI), which brings together indicators of infant mortality, life expectancy, and adult literacy (Morris 1979).

Although components of these measures, especially the components of the PQLI, may serve well as indirect indicators of population-level enjoyment of a select set of economic and social rights (e.g., the rights to education and the right to healthcare), they fail to connect directly to any specific rights listed in the international human rights regime. Indeed, because these measures were never intended to measure human rights in the first place, the particular state obligations attached to economic, social, and cultural rights are ignored in their construction. As stated in Article 2.1 of the International Covenant on Economic, Social, and Cultural Rights (ICESCR), each party to the covenant is obligated “to achieve progressively the full realization” of the rights listed in that document, “to the maximum of its available resources” (1966; 1976). As such, when attempting to determine if a state is meeting its obligations with regard to the rights listed in that document, we must take into account the state’s resources and its current ability to fulfill those rights. Fukuda-Parr, Lawson-Remer, and Randolph (2015) have pioneered a new methodology for measuring economic and social rights that takes this issue into account, while also directly connecting individual measures to rights contained in the ICESCR. These data are a huge step forward in the study of economic and social rights and we have incorporated them into the HRMI dataset. We will discuss these data in greater detail below.

There have also been data gaps in other areas of the international human rights regime. Although proxy measures and indirect indicators may give some insight into the state of adherence to the rights listed in the Convention on the Rights of the Child, the International Convention on the Rights of Persons with Disabilities, or the Convention on the Elimination of All Forms of Discrimination against Women, as well as several other international human rights instruments, there are few direct measures that treat the content of those documents as international law suggests they should be treated. This absence not only leaves a great many rights unmeasured but sends the signal that some rights, particularly civil and political rights, are somehow more important than others, undermining the unmeasured rights’ status as rights. Furthermore, it has likely deterred greater use of human rights data in the public sphere since, when a practitioner or activist has gone looking for data that treats these rights as rights, they have been left wanting and forced to rely on suboptimal, indirect indicators.

Indeed, failing to capture the entire list of internationally recognized human rights in our data has also probably harmed the academic study of human rights by yielding overstated inferences. Often, researchers claim that some factor affects respect for “human rights” in some way, when what they really mean is that the factor affects respect for “physical integrity rights.” The difference between those two phrases is important. Although the entire list of internationally recognized human rights is interdependent and indivisible (Donnelly 2013; Fariss and Schnakenberg 2014), that does not mean that policies or actions that improve respect for one set of rights automatically improve respect for all other rights.

Indeed, human rights come into conflict all the time. For example, recent work by Barry, Cingranelli, and Clay (2018) showed that, in the area of labor rights, our conclusions have largely relied only on what improves respect for the rights to associate and collectively bargain, ignoring other labor rights, like the right to the reasonable limitation of working hours, the right to an adequate minimum wage, or the right to a healthy and safe workplace. The causes of respect for these different sets of rights appear to be different, and under conditions of low state capacity, increased respect for one set of rights might in fact lead to lower respect for another. We suspect there are many cases in which the same is happening in the broader study of human rights. Most of our knowledge is based on what is good for improving respect for physical integrity rights. By not looking at the entire list of human rights holistically, our conclusions may lead to improvements in respect for physical integrity at the expense of respect for other rights. Without research that attempts to understand how to mediate or moderate these tradeoffs, research that aims to provide insights into how to improve respect for human rights globally may be actively doing harm to the enjoyment of at least some of those rights.

Finally, the existing human rights data have not always been presented in ways that are easy to understand and accessible by audiences beyond academia. Currently, the vast majority of human rights datasets are primarily available in the form of downloadable Excel, pdf, or text files (e.g., Cingranelli, Richards, and Clay 2014a; Fariss 2014; Schnakenberg and Fariss 2014; Fukuda-Parr et al. 2015). Few of these data projects have any kind of visualizations of the data outside of their academic publications, and when they do, they mostly take the form of minimally interactive global maps (Gibney et al., 2017; Human Rights Atlas 2018). This is not a criticism of the producers of these data; the production of these data is a commendable task in and of itself. However, the lack of an interface with which one can easily and quickly interact with the data and get answers to pressing human rights questions serves as yet another barrier between high-quality social scientific human rights data and the general public.

Thus, until recently, human rights data have been piecemeal, often based on tertiary sources that underestimate the global level of human rights abuse, and largely inaccessible by the public. This state of affairs is harmful to the academic research that relies on these data and has left global human rights practice as a nearly data-free zone. What can be done to improve this situation? In the next section, we introduce the Human Rights Measurement Initiative (HRMI), our attempt to overcome these problems with a new framework for producing data and a new platform for accessing those data.

The Human Rights Measurement Initiative

HRMI aims to overcome the problems discussed above by taking a comprehensive approach to human rights measurement, with a strong emphasis on independence, cross-disciplinary collaboration, and human-centered design. HRMI’s goal is to reinvent the way human rights data are both produced and used to help bring about improved global enjoyment of the entire list of internationally recognized human rights. Although unabashedly prohuman rights, HRMI is not an activist organization. Its goal is to work with others to produce the best possible data and to help codesign ways for the data to be used for impact, but not to actively engage in lobbying for changes to laws, policies, and practices in countries around the world. To ensure that HRMI data are accessible to the public and civil society, rather than just academics and other quantitative researchers, HRMI puts a strong emphasis on the public presentation of the data, including through the development of an easily understood interactive data visualization website. In the remainder of this section, we elaborate on each of these key points.

Comprehensive coverage of all human rights

HRMI’s goal is to measure country-level progress on every single human right as defined in international human rights law. The 2019 data set provides metrics on twelve different areas of international human rights. To begin to overcome the lack of attention that economic and social rights have received in past work, we adopt the measurement approach pioneered by Fukuda-Parr et al. (2015) for five of those rights specified in the International Covenant on Economic, Social, and Cultural Rights (ICESCR): the rights to food (Article 11), education (Articles 13 and 14), health (Article 12), housing (Article 11), and decent work (Articles 6 and 7). Unlike the previous indirect and proxy measures used in the past, these data treat these outcomes as rights and take seriously the state obligations laid out in Article 2.1 of the ICESCR—that every party to the treaty must progressively realize the rights “to the maximum of its available resources.” As such, every country has a different level of rights enjoyment that the state is responsible for fulfilling at a given time, based on what could be reasonably achieved with the state’s current resources.

To incorporate these varied obligation levels into their data, Fukuda-Parr, Lawson-Remer, and Randolph (2009, 2015; see also Randolph, Fukuda-Parr, and Lawson-Remer 2010) set an achievement possibilities frontier representing the best observed practice at each per capita income level. This frontier benchmarks the state’s obligation at each per capita income level. As such, the HRMI economic and social rights measures are presented as a percentage of the feasible achievement obtainable by the country given its per capita income level. Thus, a 100 percent score does not necessarily mean that everyone in the country is enjoying the right but, rather, that the state is doing as well as can be expected on that right given its resources.

Of course, there are other economic, social, and cultural rights beyond these five. Likewise, if we aspire to capture all of the human rights contained in the UN human rights regime, HRMI will have to develop measures of the rights contained in the International Covenant on Civil and Political Rights (ICCPR), the International Convention on the Elimination of All Forms of Racial Discrimination (ICERD), the Convention on the Elimination of All Forms of Discrimination against Women (CEDAW), and several other international human rights treaties. How did we develop the measures of the other seven rights included in our data? By what process do we intend to design future measures? In the next section, we go into greater detail about the process we used to develop seven new measures of civil and political rights.

Collaboration and codesign

Earlier human rights datasets have been highly influential at building academic knowledge, but much less successful in terms of influencing public opinion and helping practitioners to bring about change. In part this is not surprising, given that many of these data projects were developed in low-resource environments by academics working alone, absent strong input from the human rights practitioner community. Although HRMI also draws strongly on the expertise of academics, it uses human-centered design techniques to also harness the expertise of in-the-field human rights researchers and practitioners. One of HRMI’s criteria for success in producing good data is that those data be used by both academics and human rights practitioners alike.

Human-centered design (also known as codesign) is a process for design that hinges on the idea that, as long as the designers gain empathy for and stay connected to the behaviors and needs of the people for whom they are designing, their ideas will evolve in the right direction (IDEO.org 2015). HRMI credits this approach for the successful completion of its initial studies aimed at measuring civil and political rights. In this case, the key user group was in-country human rights researchers and practitioners who may be working for large human rights organizations such as Amnesty International or Human Rights Watch, or for smaller regional or domestic NGOs. When seeking feedback from some of these practitioners about existing civil and political rights datasets, a common complaint was that those datasets were often drawing on publicly available information that was incomplete and not compiled for the purposes of quantification. When asked what alternatives would be acceptable, practitioners often responded, “Why don’t you just ask us?”

Taking that challenge seriously, HRMI’s civil and political rights metrics team designed an early prototype of an expert survey. Using codesign techniques at a March 2017 workshop run by HRMI at the University of Georgia, workshop participants worked together in pairs or groups of three, observing one another take the survey prototype. Observers would ask survey takers to speak out loud as they progressed through the survey, commenting on how they were feeling (e.g., frustrated, intrigued, confused), and what they were thinking (e.g. “I am having trouble answering this question because I don’t know what role the government plays in crime-related disappearances”). Using this information, the team quickly revamped the survey and retested, with this process being repeated several times, including over the following months, via online video-calls with human rights experts in countries around the world.

The resulting survey produced a dataset containing information on the type of abuse, the frequency of that abuse, and the attributes that placed people at the perceived highest risk of abuse in each country. To generate cross-nationally comparable measures of the frequency of each type of abuse, the HRMI civil and political rights team used a method that had respondents answer questions about a series of anchoring vignettes—that is, hypothetical human rights scenarios. These responses allowed the team to ensure that the statistical model used places all respondents on the same scale and overcomes any differential understanding of the scale across respondents. Likewise, as demonstrated by our discussion of Australia’s treatment of indigenous people, refugees, and asylum seekers below, our data provide more information about the distribution of multiple forms of civil and political rights abuse than any previously existing global data set.⁵ These data are discussed in greater detail below, but for even more information on how these data were produced and how they compare to existing human rights data, please see Brook, Clay, and Randolph (2018) and Clay, Bakker, Brook, Hill, and Murdie (2018), as well as the methodology section at the HRMI website (https://humanrightsmeasurement.org/).

It is too early to evaluate how successful this approach has been at encouraging practitioners, journalists, and others to embrace human rights data in their work. As of the time of this writing (May 2019), HRMI’s civil and political rights data are so far available only for nineteen countries and are not yet well known. However, we believe that once more complete country coverage has been achieved the data are likely to gain profile, aided by a strong emphasis on dissemination and accessibility, which is discussed below.

Making data accessible and usable

HRMI has made available downloadable data on twelve human rights.⁶ Seven of those rights are civil and political rights: the rights to opinion and expression, assembly and association, participation in government, freedom from execution, freedom from torture and ill-treatment, freedom from disappearance, and freedom from arbitrary or political arrest and imprisonment. For each of these rights, HRMI provides estimates of the overall level of respect for the right on a scale from 0 to 10, in which higher scores represent greater government respect (and fewer government violations) of the right in question, as well as a standard deviation and an 80 percent credible interval for each estimated overall score. Further, HRMI includes information on which populations our respondents claimed were at particular risk for violations of each civil and political right, providing the proportion of respondents that told us that each of thirty-two listed groups was at risk in their country. With additional disaggregated data on death penalty executions and extrajudicial executions, the 2019 HRMI data set contains nearly three hundred variables on civil and political rights alone, with coverage of nineteen pilot countries and plans to expand country coverage with each subsequent annual data release.⁷

The other five rights included in the currently available HRMI data are the economic and social rights mentioned above: the rights to food, education, health, housing, and decent work. As discussed by Fukuda-Parr et al. (2015), data available for high-income countries often differ from those available for the rest of the world. As a result, two different assessment standards are included in the existing data: the high-income assessment standard and the low- and middle- income assessment standard. Table 1 displays the overall economic rights indicators and subindicators included in the 2019 HRMI data, as well as the assessment standards to which each subindicator contributes. These social and economic data are available for five rights and seventeen separate subrights covering one hundred and twenty to one hundred and eighty countries, depending on the right.⁸

Table 1. Economic and social rights data in the 2019 HRMI data release.

Rights and subrights	Assessment standard
	Low- and middle-income country	High-income country
Right to food score
% Children not stunted	✓
% People food secure (Food Insecurity Experience Scale)		✓
Right to education score
Net Secondary school enrolment	✓	✓
Adjusted Net Primary school enrolment	✓
PISAscience % > level 2		✓
PISAmath score % > level 2		✓
PISAreading score % > level 2		✓
Right to health score
% Children surviving to age 5	✓	✓
% People surviving to age 65	✓	✓
% Couples (15–49) using Modern Contraception	✓
% Newborns not low birthweight		✓
Right to housing score
% People with basic sanitation	✓
% People with water on premises	✓
% People with safely managed sanitation		✓
Right to work score
% People not absolutely poor (>3.20 2011 PPP$ per day)	✓
% People not relatively poor (>50% median income)		✓
% Unemployed not long-term (>12 months) unemployed		✓

HRMI wants to produce data that are used by human rights advocates and academics alike. Moreover, HRMI also hopes its data will be used to help educate the public about human rights. As a result, given the large amount of complex data available in the merged HRMI dataset, HRMI puts a strong emphasis on presenting data in a way that is both easily accessible and understood and that uses human-centered design techniques to help steer the development of the data visualization website, much as was done for the development of the civil and political rights expert survey.

The results of this process can be viewed in their entirety from HRMI’s website (https://humanrightsmeasurement.org/). First, the data visualization site gives the viewer the ability to select between exploring the data by country or by right. The country option allows the viewer to get a snapshot of a country’s practices across all the rights that are currently in the HRMI dataset. For instance, Figure 1 shows Australia’s and Mozambique’s scores for calendar year 2018 across all the HRMI measures in the 2019 data set. In these graphs, Australia’s economic and social rights metrics are calculated according to the high-income assessment standard, whereas Mozambique’s scores are calculated according to the low- and middle-income assessment standard.

Figure 1. Human Rights Performance in Australia & Mozambique in 2018. Image Source: 2019 HRMI dataset, data.humanrightsmeasurement.org (last accessed on June 6, 2019).

Although the graphs in Figure 1 provide an overview of the human rights practices of these two countries, they also likely invoke questions that merit deeper analysis. For instance, Australia scores reasonably poorly on freedom from torture and freedom from arbitrary or political arrest and imprisonment. How do those low scores compare to what we observe in the rest of the world? To begin to answer that question, one could explore the HRMI data by right to look at a graph that looks like Figure 2. This figure shows Australia’s score, along with that score’s 80 percent credible interval, on the right to be free from torture and ill-treatment alongside the performance of the other eighteen countries in the 2019 HRMI civil and political rights data. Australia’s score is better than most of the other countries in our sample, but it is still significantly worse than South Korea, New Zealand, and the United Kingdom and is basically indistinguishable from those observed in Liberia and Nepal.

Figure 2. Freedom from torture and ill-treatment in 2018. Image Source: 2019 HRMI dataset, data.humanrightsmeasurement.org (last accessed on June 6, 2019).

Why would a high-income democracy perform poorly on this right?⁹ Previous research (e.g., Englehart 2009; Cingranelli, Fajardo-Heyward, and Filippov 2014; Clay and DiGiuseppe 2017) has argued that states with high levels of resources and state capacity should be less likely to violate physical integrity rights. Likewise, a large literature (e.g., Poe, Tate, and Keith 1994; Davenport and Armstrong 2004; Conrad and DeMeritt 2012) has demonstrated a strong relationship between democracy and physical integrity rights, with some suggesting an overlap between the two concepts at their core (Hill 2016). As such, it is surprising to see Australia performing similarly to countries at lower levels of both economic development and democracy.

The expert respondents to the HRMI survey have provided some explanation. On the HRMI website, the viewer can get more information on a country’s score for any of the civil or political rights, including a word cloud showing which kinds of attributes, traits, identities, groups, and other factors most placed one at risk for violation of that right by state actors. Figure 3 displays the word cloud from the 2019 data set for Australia’s torture and ill-treatment practices in 2018. As shown, indigenous people, refugees, and asylum seekers were the people our respondents most pointed to as being at risk for torture and ill-treatment. In the qualitative information provided by our respondents, survey respondents largely pointed to the treatment of Aboriginal and Torres Strait Islander people, particularly women and those arrested and in detention, as well as the treatment of asylum seekers, refugees, and other immigrants who attempt to reach Australia by boat or who have been detained, especially in offshore detention on Manus Island, Papua New Guinea, and Nauru.¹⁰

Figure 3. People most at risk of torture and ill-treatment, Australia, 2018. Percentage of expert survey respondents who selected each group is shown in parentheses. Image Source: 2019 HRMI dataset, data.humanrightsmeasurement.org (last accessed on June 6, 2019).

Returning to Figure 1, another standout figure is Mozambique’s low score on the right to work. How does that compare to other countries? Figure 4 shows a bar graph from the rights section of the HRMI data visualizations that compares Mozambique’s right to work score with those observed for all other low- and middle-income countries in sub-Saharan Africa. As shown here, Mozambique performs relatively poorly among its regional peers, with only fourteen of the forty-two low- and middle-income countries in Sub-Saharan Africa—like Madagascar, Malawi, and Nigeria—performing worse. The low- and middle-income standard for the right to work relies strictly on the percentage of the population with an income over $3.20 per day in 2011 dollars adjusted for purchasing power parity (World Bank 2019). As such, this score tells us that Mozambique is only achieving about 33 percent of what should be feasible, given its level of economic development. Overall, the amount of wealth in Mozambique suggests that many more people should be receiving an income over $3.20 per day if the government was taking seriously the right to decent work as outlined in Articles 6 and 7 of the International Covenant on Economic, Social, and Cultural Rights. Perhaps unsurprisingly, as of May 2019, Mozambique is among the twenty-four eligible states that have neither signed nor ratified that document (OHCHR 2019).

Figure 4. The right to work in low- and middle-income states in sub-Saharan Africa, 2018. Image Source: 2019 HRMI dataset, data.humanrightsmeasurement.org (last accessed on June 6, 2019).

Of course, there are many more questions that can be investigated and answered using HRMI’s data visualization platform. However, the key goal in the development of these visualizations, as well as HRMI’s continued development moving forward, is to ensure that their existence makes human rights data accessible to every person who may ever find a use for it. As such, although we find HRMI’s data platform to be a vast improvement over previous attempts to make human rights data accessible, we suspect there will be many additions, changes, and improvements to it in the years to come.

Conclusion

The HRMI emerged out of a perceived need, by its cofounders and others, for comprehensive human rights data that could be easily used and understood by an audience wider than academics alone. Earlier attempts at producing human rights data were plagued by limitations on the list of rights analyzed, bias in their sources of information, and impediments to understanding by those outside of academia. HRMI attempts to overcome these issues by moving toward comprehensive coverage of all internationally recognized human rights, using codesign techniques aimed at reducing bias and increasing the number of information sources on which the data are based, and producing data visualizations that make the messages present in the data clear to a broad audience. Although we are still in the early stages of this project, we are encouraged by HRMI’s progress so far.

Nevertheless, much remains to be done. First, although HRMI aspires to comprehensive coverage of rights and has displayed its commitment to that goal by pursuing measures of civil, political, economic, and social rights in its first phase, there are many more rights to cover before HRMI comes close to that goal. We will have to overcome limited and missing data in the sphere of economic and social rights in order to cover all of the rights in the ICESCR; furthermore, we would like to expand to highlight inequalities in the enjoyment of these rights within countries.¹¹ Likewise, HRMI will need to expand its coverage of civil and political rights to cover all of the rights in the ICCPR, alongside making many more connections with human rights practitioners worldwide to ensure global coverage on those metrics. Beyond the spheres of these existing categories of rights, HRMI will need to recruit additional researchers to develop methodologies for measuring adherence to the rights listed in the International Convention on the Elimination of All Forms of Racial Discrimination, the Convention on the Elimination of All Forms of Discrimination against Women, the Convention on the Rights of the Child, and all of the other internationally recognized treaties that make up the international human rights regime.

Likewise, HRMI will have to continue to develop ways to improve its existing metrics and visualizations, as well as considering new potential routes for expansion. In the fall of 2018, we conducted a codesign workshop in Johannesburg, South Africa, with human rights practitioners from twenty-five countries around the world. This workshop exposed us to new information about how practitioners would like to see our economic and social rights data visualized, as well as exposing us to interesting insights into how we can further develop our civil and political rights data with additional sources of information. The visualizations in this article are a direct result of some of that feedback. However, the work of design will never be done. HRMI will always be striving to improve its data and presentation to make it as useful as possible for all audiences around the world. Furthermore, as new human rights law and issues arise, HRMI’s process of collaboration and codesign with human rights practitioners should enable it to be at the forefront of data collection in those areas. Of course, the continuation and expansion of this work, as envisaged, is conditional on HRMI being able to attract sufficient and ongoing funding.

HRMI data and visualization tools will be good for academics, practitioners, and the public alike. For academics, we hope it provides better data than has previously been available on a much larger number of rights. Furthermore, by encouraging users to focus attention on a holistic list of human rights, we hope HRMI encourages more research on the linkages between different rights. For practitioners, HRMI aims to provide outputs that are useful in human rights advocacy and policymaking. We hope its data provide the opportunity to analyze the impact of multiple interventions and to determine if things are getting better or worse over time. Likewise, through comprehensive coverage of rights, we also hope that HRMI minimizes the risk that efforts to improve one category of rights come at the expense of another.

Finally, for the general public, we hope that HRMI’s metrics and visualizations are easy enough to comprehend so that people will find them useful for understanding what their rights are, how they are being respected, and how their fulfillment might be improved. Furthermore, if we have met that goal, we hope journalists and other communications experts are able to leverage HRMI’s efforts to bring more attention to human rights issues globally. If and when all of this is achieved, then HRMI will have truly made human rights data for everyone.

Acknowledgments

The authors would like to thank Ryan Bakker, Scott Edwards, Caroline Faggion, Sakiko Fukuda-Parr, Timo Franz, Russ Gaskin, Elizabeth Heritage, Daniel Hill, Terra Lawson-Remer, David Matsinhe, Amanda Murdie, Vincent Nolette, Matthew Rains, Ute Reisinger, Ceridwyn Roberts, Brian Root, Thalia Kehoe Rowden, John Stewart, Dominic White, Edith Woischin, and Carlos Zazueta for their many contributions to HRMI’s work. We would also like to thank all of the survey respondents who have given their time to share their knowledge with us, as well as the HRMI ambassadors, who helped connect us with potential survey respondents for their countries. Further, we are grateful to the participants of the 2015, 2017, and 2018 HRMI codesign workshops, as well as the many others who have given of their time to help us design and translate all of the components of our work.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

Notes

1 All data and graphs discussed and used in this article are available for download at https://humanrightsmeasurement.org/ (last accessed on June 3, 2019). This research was supported in part by a grant from the Open Society Foundations.

2 Based on a search conducted on August 26, 2018, using the search terms “Cingranelli-Richards” OR CIRI AND “human rights” for CIRI and “political terror scale” for PTS.

3 This search was conducted by indexing in Windows 7 the text of all of the files produced by Fariss et al. 2015 and using the terms “Cingranelli,” “Political Terror,” “CIRI,” and “PTS.” None of the text matching these search terms referred to the CIRI or PTS human rights datasets.

4 In recent years, PTS has also relied on Human Rights Watch’s Annual Report.

5 By comparison, CIRI (Cingranelli, Richards, and Clay 2014a) only disaggregates its civil and political rights measures slightly by providing additional data on the enjoyment of women’s political rights. PTS (Gibney et al. 2017) considers some information about whether abuse is politically targeted or indiscriminate in determining whether a country receives a higher score on its scale (see Haschke 2017), but it does not provide any further information on the range of abuse. The Human Rights Protection Scores (Fariss 2014; Schnakenberg and Fariss 2014) provide no information on the distribution of abuse in the population. The Ill-Treatment and Torture data (Conrad et al. 2013, 2014) provide information about the identities of the victims of ill-treatment and torture, but they provide no information about any of the other civil and political rights.

6 We have also made the complete merged HRMI dataset released in 2018 available as replication data associated with this article.

7 The thirteen countries included in the initial 2017/2018 HRMI pilot survey were Angola, Australia, Brazil, Fiji, Kazakhstan, Kyrgyzstan, Liberia, Mexico, Mozambique, Nepal, New Zealand, Saudi Arabia, and the United Kingdom. Country coverage for the 2019 survey (collecting data for calendar years 2018 and 2017) was expanded to include a further six countries: Democratic Republic of the Congo, Jordan, South Korea, the United States, Venezuela, and Vietnam.

8 More detailed information on the construction of the economic rights data can be found in the HRMI Methodology Handbook (Brook et al. 2018), particularly in Chapter 4 (Randolph, Fakuda-Parr, Lawson-Remer, and Reisinger 2018).

9 It should be noted that Australia does not have the worst score on freedom from torture among high-income democracies included in the 2019 HRMI civil and political rights sample. The United States’ mean score of 3.2 falls well below Australia’s mean of 5.5.

10 Summaries of the qualitative responses provided on the HRMI pilot civil and political rights survey can be viewed alongside the information about people at risk in the HRMI data portal at https://data.humanrightsmeasurement.org/ (last accessed on June 6, 2019).

11 The 2019 data also include information from survey respondents about the people most at risk of lack of enjoyment of their economic and social rights. However, we currently only have this feature for the nineteen countries included in the civil and political rights sample, leaving out many countries for which we have other economic and social rights data.