On today’s mobile phone screens, we are dealing with algorithms almost every minute: watching short videos, reading news, doing exercises, looking for jobs, and applying for loans. Many people think that algorithms are just “mathematical formulas that calculate a little faster”, but more and more studies remind us that these systems can systematically favour some people and harm others. This is algorithmic bias.
For instance, COMPAS, a risk assessment tool widely used in the US judicial system, was found to have labeled black defendants as “high-risk” almost twice as much as white defendants when predicting “who is more likely to reoffend”, and white defendants are more likely to be labeled as “low-risk”. In the field of facial recognition, Joy Buolamwini from MIT discovered that some commercial systems have an error rate of less than 1% when identifying light-skinned men, but the error rate for dark-skinned women is as high as over 30 percentage points. That is to say, even when it comes to “having machines recognise people”, the accuracy achieved by different groups of people varies completely.
The problem is not merely that the training data is “not diverse enough”. There is a whole set of data power structures behind it, which many scholars call data colonialism. Nick Couldry and Ulises Mejias pointed out that platform companies continuously collect various behaviours in daily life as “raw materials” to train models, precisely target advertisements, and optimise business decisions. This is a kind of data extraction that continues the colonial logic: People’s lives have been transformed into data resources, yet they have almost no control or distribution rights.
To make these AI systems “look smart”, many invisible labourers are still needed to do data annotation, content review, voice transcription and other work. Mary Gray and Siddharth Suri call this kind of work “ghost work”: Consumers think it is done automatically by AI, but in fact, behind it are micro-task workers distributed in places like India, the Philippines, Kenya, etc. They clean up the algorithm for a few cents per order, but receive extremely low pay and unstable contracts.
Putting all these together, we can see that algorithmic bias is not the carelessness of one or two engineers, but a concrete manifestation of data colonisation in the contemporary Internet. On one end, there is asymmetric data collection and labour exploitation on a global scale; on the other end, there are unequal outcomes in facial recognition, judicial scoring, and recruitment screening. The former provides the raw materials of “who is seen and how is recorded”, while the latter packages these differences into seemingly neutral “ratings”, “labels” and “recommendations”.
For us, ordinary users, perhaps we can’t change the entire system immediately, but at least we can ask one more question: For whom was this algorithm designed? Whose data does it use? Who works behind the scenes? Only after seeing these problems clearly will “fairer AI” and “responsible digital society” not be empty words but realities that can be constantly approached.
References：
Angwin, Julia, et al. “Machine Bias.” ProPublica, 23 May 2016, www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
Buolamwini, Joy, and Timnit Gebru. Proceedings of Machine Learning Research 81:1–15, 2018, proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf. Accessed 1 Dec. 2025.
Couldry, Nick, and Ulises A. Mejias. The Costs of Connection: How Data Is Colonising Human Life and Appropriating It for Capitalism, Stanford University Press, Stanford, CA, 2019.
This post really captures the complexity of algorithmic bias and the hidden inequalities that come with it. It is so easy for people to assume that algorithms are objective or neutral, but the reality is far from that. The examples involving COMPAS and facial recognition are shocking but not surprising, especially when we consider how deeply entrenched these biases are in the data on which they are trained (by humans).
Data colonialism: the extraction and exploitation of data from vulnerable communities without any real benefits to them. It goes hand in hand with the concept of “ghost work”. Invisible labor that keeps unequal systems running, at the expense of people who are paid next to nothing and not considered for proper emotional or mental health support for the traumatic content they are exposed to while training AI algorithms. It is a harsh reminder of how technology is being built on systems of inequality. As you well mentioned, as users, we need to keep asking: Who benefits from this? Who is being left out or harmed? And, the more we question, the more we push for transparency…
They don’t just calculate faster; they reflect existing social inequalities and power structures. From biased risk assessments in courts to facial recognition errors and exploitative “ghost work,” it’s clear that algorithmic systems can advantage some groups while harming others. As ordinary users, we may not be able to fix the system overnight, but asking critical questions, like who the algorithm is designed for, whose data it uses, and who works behind the scenes, can help us push toward fairer and more responsible digital technologies.
This was a captivating post, straight from the get go. The real world example you used is shocking to think about, and highlights a massive problems with algorithms, that being that they’re not completely accurate. It’s also interesting to look at the process behind the creation of algorithms, involving exploitation of labourers as seen in Venezuela, and I think it’s valid to argue, are algorithms worth people being exploited for, and I think the answer is no. AI is still developing, however I think it’s causing more problems than it will solve, especially with algorithms.