Imagine submitting your resume for your dream job — and your application gets rejected before a human ever sees it. Or you apply for a loan, and a computer says “no”… without explanations, just based on some numbers in a database.

Artificial intelligence makes decisions about our lives every day: it analyzes resumes, approves loans, shapes what we see on social media, and even assists courts. It sounds logical — algorithms are objective, after all. They have no emotions or personal preferences. But here lies the paradox: AI doesn’t just copy our mistakes — it amplifies them.

Where AI learns what to do

To train artificial intelligence, it is shown enormous amounts of examples: texts, photos, records of past decisions, conversation logs. There is just one problem — all this data was created by humans. And humans, as we know, are far from perfect.

Think about it: if a model is trained on resumes of successful employees in a company that has hired mostly men for the past 20 years, what will it learn? Exactly — that men are a “better fit.” AI doesn’t understand that this may be the result of unfair practices or historical circumstances. It simply sees: “There are more men in leadership positions in this data. That must be how it’s supposed to be.”

The same applies to court cases. If certain groups of people have historically received harsher sentences, the algorithm will treat this as the norm. It doesn’t feel injustice — it just calculates.

How we “teach” AI discrimination

Bias seeps into AI at every stage:

History works against us. If women were less likely to be promoted to leadership roles in the past (and they were), AI interprets this as a pattern: “Women rarely become leaders = women are bad leaders.”

Data can be one-sided. A facial recognition system trained mostly on photos of Europeans will simply perform worse on Asian or African faces — not out of malice, but because it was not shown enough diverse examples.

People make mistakes too. Developers choose datasets, label them, and decide what counts as “success.” At each of these stages, their own unconscious biases can creep in.

What kinds of bias AI picks up

Artificial intelligence can be unfair in many different ways:

Gender stereotypes: for AI, a “programmer” is more often male, while a “nurse” is female — simply because the data reflects that distribution.

Racial bias: credit scoring systems may assign worse ratings to people of certain ethnic backgrounds, even when their financial indicators are identical.

Age discrimination: recruiting AI systems may “dislike” older candidates, even if age is never explicitly stated. The algorithm just notices a pattern: people who graduated from university longer ago seem “less successful.”

Class bias: where you live, where you studied, your postal code — all of this can influence how AI evaluates your chances.

When things go wrong: real-world stories

Amazon was hiring… but only men

Amazon once decided to automate hiring. The idea sounded great: feed the system resumes of all successful employees from the past 10 years and ask AI to find similar candidates. The problem? Most of those successful employees were men.

What did AI do? It automatically downgraded resumes containing the word “woman” or names of women’s colleges. The system effectively learned to discriminate against women — even though no one explicitly taught it to do so.

Amazon identified the issue and shut the project down. But the story showed that even tech giants can unintentionally create discriminatory AI.

A chatbot that became racist in one day

In 2016, Microsoft launched a Twitter chatbot named Tay. The idea was simple: let it talk to people and learn from live conversations. Sounds harmless, right?

Within 24 hours, Tay became racist and sexist, posting offensive content that is hard to even quote. Microsoft shut it down within 48 hours. The lesson was clear: when AI learns from unfiltered internet content, it quickly absorbs the worst of it.

Courts trusting biased algorithms

In the United States, courts used systems designed to predict whether a person would reoffend. It seemed like a useful tool for judges deciding on parole.

However, an investigation by ProPublica revealed a disturbing pattern: the algorithm systematically overestimated risk for Black defendants and underestimated it for white defendants. It was wrong about Black people almost twice as often. Why? Because it was trained on data from a justice system that was already racially biased.

The AI wasn’t programmed to be racist. It simply reproduced what it saw in the data.

Uber raises prices when you need it most

Uber uses AI for dynamic pricing: when demand increases, prices automatically rise. From a business perspective, this makes sense.

But what happened during a terrorist attack in Sydney? People were desperately trying to flee the danger and requested rides en masse. The algorithm saw a surge in demand — and raised prices by four times.

Uber later added exceptions for emergency situations. But the deeper issue remains: dynamic pricing means services become inaccessible to lower-income people precisely when they are needed most. This, too, is a form of discrimination.

Why bias can’t just be “turned off”

A seemingly obvious solution is to remove data about gender, race, and age — and the problem disappears. If only it were that simple.

AI detects hidden signals. Even if an algorithm doesn’t know your gender, it can infer it from your name, university, hobbies listed on a resume, or even writing style.

History remains in the data. Suppose women were historically paid less for the same work. A salary prediction algorithm will reproduce this injustice even without knowing the candidate’s gender. It simply sees: “People with these characteristics were historically paid less” — and continues the trend.

Objectivity doesn’t really exist. Even choosing what counts as a “good outcome” is a human decision loaded with values. What matters more: maximum accuracy, or fairness across all groups? Different answers lead to entirely different algorithms.

Who is responsible for AI now?

Since the launch of the AI Act on August 1, 2024, responsibility for artificial intelligence in the European Union is distributed across multiple levels. At the EU level, the key role is played by the European AI Office — a dedicated body within the European Commission responsible for overseeing and enforcing the AI Act, particularly for general-purpose AI models. It works alongside the European Artificial Intelligence Board (a coordination and advisory body), as well as the Scientific Panel and Advisory Forum, which provide technical and expert guidance. National competent authorities in each country are responsible for supervision and enforcement within their jurisdictions. Full applicability of the law is expected by August 2, 2026, with phased requirements already in effect.

Three roles: who is responsible for what

The law divides all actors into three categories:

Providers — those who develop AI systems. They bear primary responsibility for ensuring their products are safe.

Deployers — companies that take ready-made AI and integrate it into their products, such as adding a chatbot to a website. Even if you didn’t build the AI yourself, you still have obligations:

• assess risks before deployment
• document how these risks are mitigated
• follow the developer’s instructions
• inform users that they are interacting with AI

End users are not responsible for AI failures. If something goes wrong, accountability lies with the provider and the deployer.

Four levels of risk

The AI Act categorizes AI systems based on potential harm:

Minimal risk: AI that doesn’t make significant decisions — generating presentations, applying photo filters. Almost no restrictions apply.

Limited risk: assistants and support chatbots. They affect user experience but don’t make critical decisions. The main requirement is transparency: people must know they’re interacting with AI.

High risk (this is serious): AI in healthcare, justice, education, recruitment, and credit approval. These systems require thorough pre-deployment checks and continuous oversight.

Unacceptable risk: technologies for manipulation, social scoring of citizens, and mass surveillance. These are banned outright.

What to do if your AI is “high risk”

Pre-deployment assessment. Before activating the system, you must:

• identify all potential risks
• document mitigation measures (anonymization, encryption)
• verify compliance with developer instructions

Only then can the system be launched.

Transparency. People must know when they are interacting with AI. This can be as simple as a label: “Our chatbot uses AI to provide faster assistance,” or an introduction at the start of a conversation.

The right to know “why.” If AI makes a decision affecting you, you can ask: “Why was I not approved? What data was checked? What didn’t match?”

The right to a human. If you disagree with an AI decision, you must be able to request human review. This is critical — AI can be wrong, and people must be able to challenge its verdicts.

Continuous monitoring. Set it and forget it is not an option. Systems must be regularly reviewed for errors, drift, and behavioral changes.

Team training. People working with AI must understand:

• how to use it correctly
• its limitations
• when updates are released and how to apply them
• the need for ongoing learning

NovaTalks: avoiding AI mistakes in contact centers

Despite all the challenges, artificial intelligence remains a powerful and effective tool for contact centers — if implemented and controlled properly. NovaTalks has years of experience deploying AI solutions for customer service, and we know how to avoid common pitfalls.

Why AI works reliably at NovaTalks

Controlled environment. Unlike cases such as Amazon or Microsoft Tay, where AI learned from uncontrolled data, NovaTalks’ AI tools operate within clearly defined boundaries. We do not allow systems to form biases from random sources.

Continuous human oversight. AI assistants at NovaTalks support operators in real time — correcting errors, translating, adjusting tone, summarizing conversations. But the final decision always belongs to a human. This is the critical difference from fully automated systems.

Transparent automated quality assessment. Our system analyzes 100% of conversations (compared to the 5% typically reviewed manually). However, we never use it for automatic dismissals or punishments. It is a tool to highlight areas that require human attention.

Certified security. NovaTalks is officially certified under ISO/IEC 27001:2022 — the global standard for information security management. This ensures that all data processed through our platform is securely protected.

Our experience shows: AI can be fair

Through years of working with banking, financial, and telecom sectors, we have developed principles that help prevent discriminatory errors:

• Diverse training data: datasets covering different customer groups, regions, languages, and communication styles
• Regular audits: periodic bias checks to ensure consistent performance across all user segments
• AI Act compliance: full responsibility for quality and fairness when operating in the European market
• Team education: continuous training for clients and their teams on correct AI usage and timely human intervention

At NovaTalks, AI does not replace humans — it augments them. Chatbots and automated replies provide 24/7 support, while complex cases are always escalated to live operators. The system preserves full interaction history and customer data, enabling personalized service without bias.

Our BI system and text analytics deliver objective insights into contact center performance, supporting data-driven decisions.

AI in contact centers can — and must — be both efficient and fair. NovaTalks proves this every day by helping companies automate routine tasks without risking discrimination or losing control over service quality.

What all of us should do

Bias in AI is not a bug that can be quickly fixed. It’s a challenge that requires collective effort.

Developers must work more carefully with data, regularly audit systems for bias, and involve diverse teams in AI creation and testing.

Companies are responsible for how they use technology. They must assess risks and be transparent with users.

Regulators must set clear rules that protect people without stifling innovation.

If we want AI to be fairer than humans, it’s not enough to give it more data. We must teach it our values — not just our statistics.

AI cannot be objective while we remain biased. But acknowledging the problem is already the first step toward solving it. And that is a step we can take today.