Profiling
What is it?
Profiling refers to the use of data to predict or infer something about a person—for example, their behavior, preferences, finances, or trustworthiness. With artificial intelligence and access to vast amounts of data, it has become possible to create highly detailed and accurate profiles of individuals, even based on seemingly harmless information.
AI systems can analyze purchasing patterns, browsing history, social media activity, health records, and financial behavior—then predict who is likely to buy a certain product, commit tax fraud, be a good job candidate, or even pose a potential threat. But this kind of profiling raises a number of ethical and societal concerns.
If the profiles are highly accurate, they can give tremendous power to those who own and use them—whether commercial companies or public authorities. They can be used to personalize marketing or identify social benefit fraud—but may also verge on mass surveillance, where people are controlled or penalized based on statistical predictions rather than actual behavior.
Conversely, flawed or biased profiles can have devastating consequences for individuals. When people are misprofiled, it can lead to unfair loan rejections, unwarranted suspicion, loss of social benefits, or even punishment and imprisonment.
Examples:
One of the most well-known and serious cases of failed AI profiling occurred in the Netherlands in 2021, when thousands of citizens were wrongfully accused of social benefits fraud based on an AI algorithm. The system disproportionately targeted people with non-Dutch ethnic backgrounds and low socioeconomic status. The scandal led to the resignation of the entire government—and became an international wake-up call about the dangers of algorithmic profiling when not subject to sufficient human oversight.
At the time of writing, a legislative proposal is being considered that would allow the Danish Intelligence Service to use AI to profile citizens based on health data, social media activity, and movement patterns—without concrete suspicion. Critics warn of mass surveillance and a weakening of legal protections.
What to consider?
Always reflect on how and why you are creating profiles with AI. Ask yourself: What data am I using? Is it accurate? Are decisions being made directly based on the profiles—or are they merely tools to support human judgment?
If your organization uses AI to segment, prioritize, or predict human behavior, you should ensure that the profiling does not discriminate or misrepresent, and that affected individuals—whether citizens or customers—have access to insight and a right to appeal.
AI profiling can be effective—but it requires humility, transparency, and accountability if we are to prevent technology from becoming a tool for digital abuse of power.4o