Identifying Biases Using Data Analysis
How bias affects policy
All policies have unintended or unacknowledged consequences. Unforeseen outcomes can happen for several reasons, including faulty or incomplete assumptions and planning that is not sufficiently rigorous, but a major source of unintended consequences is bias.
While biases may be explicit, the result of identifiable assumptions by policymakers, they are more likely to be implicit; they have not been identified or their existence is unknown and therefore they have not been accounted for when predicting outcomes.
Biases often arise from incomplete data, inaccurate or unreliable data, or outdated data. Collecting and analysing all relevant data using precise and impartial methods is essential for identifying biases and for the accurate measuring of the success (or otherwise) and impact of policies.
The 2017 Lammy Review identified biases in the criminal justice system that mean that people from Black, Asian, and Minority Ethnic (BAME) communities are disproportionately affected by the negative consequences of our criminal justice system.
“BAME communities make up 14% of the population of England and Wales, but 25% of adult prisoners and 41% of under 18s in custody. If people from BAME backgrounds were in prison in proportion to the wider population, we would have over 9,000 fewer prisoners – the equivalent of 12 prisons. The economic cost of BAME over-representation throughout the CJS is estimated to be at least £309 million a year.” (David Lammy, open letter to the prime minister, 2017)
Using data and analysis to identify biases
The review was critical of the fact that different sections of the criminal justice system record data about the ethnicity of people within the system in different ways. This inconsistent approach to recording data makes it impossible to conclude whether differing outcomes for BAME communities are arising from, say, prosecution or sentencing, and thus it is impossible to identify the biases which need to be addressed. The review recommended that data about ethnicity are collected uniformly within the different sections of the criminal justice system.
Recording accurate data is an important part of the process of identifying biases, but just as important is the analysis of those data.
It may not always be immediately obvious how to complete this type of analysis. An approach that GtD might take would be to consider what biases are already present and to then split the data according to those biases.
For instance, splitting data about youth justice by demographic group (ethnicity, gender, disabilities) while controlling for other factors such as offence type and age, at each stage of the process from arrest to court sentencing, allows the relative likelihoods of punitive outcomes for different demographic groups to be more clearly seen. Further analysis of differing outcomes can then identify demographic biases which are not immediately apparent.
(For more information on this topic see our article: How youth offending services can address over-representation of BAME young people in custody | CYP Now)
The effects of bias
The Lammy Review addresses the effect that biases can have upon policies, how biased policies have life-altering consequences for individuals, communities, and whole sections of our population, and how those consequences affect our society.
Identifying biases is essential for an equitable society, and accurate and detailed data collection and analysis is essential to identifying biases. Analysis of that data can reveal the source of biases, and what factors may be contributing to them (systemic bias, overt bias, socio-economic or educational factors, historic and present-day inequalities, etc.). Once these factors have been identified policymakers can tailor policies that account for biases, address inequalities within systems, target affected populations, produce more equitable outcomes, and utilise organisational resources more efficiently.
Implicit biases are often the result of many factors and failing to account for them can cause unpredictable outcomes. They can be both markers of and contributors to systemic inequalities within a given system.
